How to improve query performance through denormalization of this schema or other techniques?












0















My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.



It looks a little bit like:
a very simplified version of my schema



There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.



Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.



The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"



SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID 
LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
LEFT JOIN itemRes3Table igt ON i.id = igt.itemID

... similar left joins for other restrictions ...

AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']

... similar ands for other filters ...

UNION
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID

... repeated joins from above to get the items without the restriction ...


My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.



The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)



I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.



I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.









share









New contributor




Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

























    0















    My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.



    It looks a little bit like:
    a very simplified version of my schema



    There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.



    Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.



    The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"



    SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID 
    LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
    LEFT JOIN itemRes3Table igt ON i.id = igt.itemID

    ... similar left joins for other restrictions ...

    AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']

    ... similar ands for other filters ...

    UNION
    SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID

    ... repeated joins from above to get the items without the restriction ...


    My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.



    The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)



    I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.



    I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.









    share









    New contributor




    Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.























      0












      0








      0








      My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.



      It looks a little bit like:
      a very simplified version of my schema



      There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.



      Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.



      The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"



      SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID 
      LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
      LEFT JOIN itemRes3Table igt ON i.id = igt.itemID

      ... similar left joins for other restrictions ...

      AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']

      ... similar ands for other filters ...

      UNION
      SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID

      ... repeated joins from above to get the items without the restriction ...


      My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.



      The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)



      I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.



      I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.









      share









      New contributor




      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.












      My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.



      It looks a little bit like:
      a very simplified version of my schema



      There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.



      Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.



      The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"



      SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID 
      LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
      LEFT JOIN itemRes3Table igt ON i.id = igt.itemID

      ... similar left joins for other restrictions ...

      AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']

      ... similar ands for other filters ...

      UNION
      SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID

      ... repeated joins from above to get the items without the restriction ...


      My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.



      The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)



      I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.



      I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.







      mysql schema query





      share









      New contributor




      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.










      share









      New contributor




      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.








      share



      share








      edited 1 min ago







      Stephen













      New contributor




      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 7 mins ago









      StephenStephen

      1012




      1012




      New contributor




      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      Stephen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "182"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });






          Stephen is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f228805%2fhow-to-improve-query-performance-through-denormalization-of-this-schema-or-other%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          Stephen is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          Stephen is a new contributor. Be nice, and check out our Code of Conduct.













          Stephen is a new contributor. Be nice, and check out our Code of Conduct.












          Stephen is a new contributor. Be nice, and check out our Code of Conduct.
















          Thanks for contributing an answer to Database Administrators Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f228805%2fhow-to-improve-query-performance-through-denormalization-of-this-schema-or-other%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          الفوسفات في المغرب

          Four equal circles intersect: What is the area of the small shaded portion and its height

          جامعة ليفربول