How to improve query performance through denormalization of this schema or other techniques?
My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.
It looks a little bit like:
There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.
Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.
The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
LEFT JOIN itemRes3Table igt ON i.id = igt.itemID
... similar left joins for other restrictions ...
AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']
... similar ands for other filters ...
UNION
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
... repeated joins from above to get the items without the restriction ...
My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.
The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)
I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.
I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.
mysql schema query
New contributor
add a comment |
My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.
It looks a little bit like:
There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.
Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.
The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
LEFT JOIN itemRes3Table igt ON i.id = igt.itemID
... similar left joins for other restrictions ...
AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']
... similar ands for other filters ...
UNION
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
... repeated joins from above to get the items without the restriction ...
My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.
The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)
I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.
I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.
mysql schema query
New contributor
add a comment |
My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.
It looks a little bit like:
There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.
Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.
The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
LEFT JOIN itemRes3Table igt ON i.id = igt.itemID
... similar left joins for other restrictions ...
AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']
... similar ands for other filters ...
UNION
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
... repeated joins from above to get the items without the restriction ...
My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.
The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)
I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.
I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.
mysql schema query
New contributor
My database attempts to describe some items, their sources and a variety of restrictions on who may use them as well as a number of intrinsic properties about the items.
It looks a little bit like:
There are 17 of these restriction tables and relationships. I put them there as some or none of them may apply to a particular item, in an almost arbitrary fashion.
Some of the restriction tables might have as few as 3 different rows where others are around 20-30. Not giants by any stretch. Each restriction is completely independent of the others.
The problem I have is that when I establish a filter to say "show me all the items not affected by restriction 1, 3, and 7" I wind up with rather ridiculous amounts of joins which I know I should avoid. What I think I'm doing is saying "gather all the items for which the restrictions provided by the user do not apply, gather all the items that should be included by checking each restriction relationship (left joins) and also gather all the possible items where no such restrictions exist at all (union)"
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, iaft.affectID, iaft.amount, NULL AS affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
LEFT JOIN itemRes2Table iaft ON i.id=iaft.itemID
LEFT JOIN itemRes3Table igt ON i.id = igt.itemID
... similar left joins for other restrictions ...
AND (igt.itemID IS NULL OR igt.gID = '.$_SESSION['gID']
... similar ands for other filters ...
UNION
SELECT DISTINCT i.id, iwt.wlID, i.sdesc, NULL AS affectID, NULL AS amount, iafbt.affID FROM item i INNER JOIN itemRes1Table iwt ON i.id=iwt.itemID
... repeated joins from above to get the items without the restriction ...
My limited research shows me that mySQL, which I'm working in, does not necessarily support automatically optimizing inefficient or unneeded joins, not that I'm certain it'd be helpful here anyway.
The list of available restrictions within a list very rarely update, but can be updated. I had considered doing something like storing a bit flag value in an integer within the item to eliminate the tables altogether and programatically calculating which restrictions were applied .... moving the logic out of the database and into the controller, but then filtering must also fall to the controller unless I can dream up a way to query if a value is stored within a stored integer (for example 2 is clearly in 2048 but not in 1023, power of 2 factors are valid and use and to join them bitwise)
I considered converting many to enums, but I'm not sure how to then include the idea that some or none of the enum values may apply, this seems exactly the problem relationships are trying to address.
I'm getting long in the tooth. I'm sure people have tackled this issue before, so even a point in the right direction on improving this monstrous looking schema would be appreciated.
mysql schema query
mysql schema query
New contributor
New contributor
edited 1 min ago
Stephen
New contributor
asked 7 mins ago
StephenStephen
1012
1012
New contributor
New contributor
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Stephen is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f228805%2fhow-to-improve-query-performance-through-denormalization-of-this-schema-or-other%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Stephen is a new contributor. Be nice, and check out our Code of Conduct.
Stephen is a new contributor. Be nice, and check out our Code of Conduct.
Stephen is a new contributor. Be nice, and check out our Code of Conduct.
Stephen is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f228805%2fhow-to-improve-query-performance-through-denormalization-of-this-schema-or-other%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown