How to influence files (pre)allocation steps (max growing value) in MongoDB 3.4 with WiredTiger
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
We have in environment deployed some really small DBs (5GB) as directory per DB.Under dirs is LVM with XFS, where we have implemented logic for "brake" when DB space should be full-filled (all is due to some strict disk quotas - not really interesting at all I think...).
Our solution runs on MongoDB v3.4.19 on CentOS 7.5. Everything looks ok.
Some of DBs has more than 60% used disk space by single collection and in our scenario we encountered that, while restoring 2,3GB dump into 5GB database dir/=disk, the preallocation after 2GBs asks OS for another space in (I think) power of 2 strategy (or something very similar).
This situation hits ours triggers, and eventually should lead to lock of DB (which is still better than fall of whole cluster because of no space left).
Basically same situation we have with 10GB DB. (And in some way we can say that similar situation can occur with literally any capacity...)
My point of question is: Am I able to influence steps (maximum size lower than ~2GB) of (pre)allocation growing of Mongo/WT for restoring (only) or at all?
(Performance impact is not highest prioritized in this case.)
I've found some promising possibilities through db.adminCommand and configStrings for storage engines, but I was not able to affect or list some of possible parameters nor change something to turn things to good...
Especially file_extend parameter of WT through
db.adminCommand( { setParameter: '1', wiredTigerEngineRuntimeConfig: { <parameter> } } )
looked promising but I didn't find way to pass it to cluster...
Some of tries ended with error 22 - Invalid argument, but I wasn't able to find more about...
The other found, seemed suitable for MMAP engine only - newCollectionsUsePowerOf2Sizes=false.
This makes restoring of dumps (eventually backups) into cluster much more bothersome...
Hope that issue is clearly explained and any hint/tip would be appreciated...
If some additional info needed, let me know.
Thank you.
Zdenek
mongodb wiredtiger
asked 8 mins ago
Zdenek M.Zdenek M.
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
We have in environment deployed some really small DBs (5GB) as directory per DB.Under dirs is LVM with XFS, where we have implemented logic for "brake" when DB space should be full-filled (all is due to some strict disk quotas - not really interesting at all I think...).
Our solution runs on MongoDB v3.4.19 on CentOS 7.5. Everything looks ok.
Some of DBs has more than 60% used disk space by single collection and in our scenario we encountered that, while restoring 2,3GB dump into 5GB database dir/=disk, the preallocation after 2GBs asks OS for another space in (I think) power of 2 strategy (or something very similar).
This situation hits ours triggers, and eventually should lead to lock of DB (which is still better than fall of whole cluster because of no space left).
Basically same situation we have with 10GB DB. (And in some way we can say that similar situation can occur with literally any capacity...)
My point of question is: Am I able to influence steps (maximum size lower than ~2GB) of (pre)allocation growing of Mongo/WT for restoring (only) or at all?
(Performance impact is not highest prioritized in this case.)
I've found some promising possibilities through db.adminCommand and configStrings for storage engines, but I was not able to affect or list some of possible parameters nor change something to turn things to good...
Especially file_extend parameter of WT through
db.adminCommand( { setParameter: '1', wiredTigerEngineRuntimeConfig: { <parameter> } } )
looked promising but I didn't find way to pass it to cluster...
Some of tries ended with error 22 - Invalid argument, but I wasn't able to find more about...
The other found, seemed suitable for MMAP engine only - newCollectionsUsePowerOf2Sizes=false.
This makes restoring of dumps (eventually backups) into cluster much more bothersome...
Hope that issue is clearly explained and any hint/tip would be appreciated...
If some additional info needed, let me know.
Thank you.
Zdenek
mongodb wiredtiger
asked 8 mins ago
Zdenek M.Zdenek M.
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
We have in environment deployed some really small DBs (5GB) as directory per DB.Under dirs is LVM with XFS, where we have implemented logic for "brake" when DB space should be full-filled (all is due to some strict disk quotas - not really interesting at all I think...).
Our solution runs on MongoDB v3.4.19 on CentOS 7.5. Everything looks ok.
Some of DBs has more than 60% used disk space by single collection and in our scenario we encountered that, while restoring 2,3GB dump into 5GB database dir/=disk, the preallocation after 2GBs asks OS for another space in (I think) power of 2 strategy (or something very similar).
This situation hits ours triggers, and eventually should lead to lock of DB (which is still better than fall of whole cluster because of no space left).
Basically same situation we have with 10GB DB. (And in some way we can say that similar situation can occur with literally any capacity...)
My point of question is: Am I able to influence steps (maximum size lower than ~2GB) of (pre)allocation growing of Mongo/WT for restoring (only) or at all?
(Performance impact is not highest prioritized in this case.)
I've found some promising possibilities through db.adminCommand and configStrings for storage engines, but I was not able to affect or list some of possible parameters nor change something to turn things to good...
Especially file_extend parameter of WT through
db.adminCommand( { setParameter: '1', wiredTigerEngineRuntimeConfig: { <parameter> } } )
looked promising but I didn't find way to pass it to cluster...
Some of tries ended with error 22 - Invalid argument, but I wasn't able to find more about...
The other found, seemed suitable for MMAP engine only - newCollectionsUsePowerOf2Sizes=false.
This makes restoring of dumps (eventually backups) into cluster much more bothersome...
Hope that issue is clearly explained and any hint/tip would be appreciated...
If some additional info needed, let me know.
Thank you.
Zdenek
mongodb wiredtiger
asked 8 mins ago
Zdenek M.Zdenek M.
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
We have in environment deployed some really small DBs (5GB) as directory per DB.Under dirs is LVM with XFS, where we have implemented logic for "brake" when DB space should be full-filled (all is due to some strict disk quotas - not really interesting at all I think...).
Our solution runs on MongoDB v3.4.19 on CentOS 7.5. Everything looks ok.
Some of DBs has more than 60% used disk space by single collection and in our scenario we encountered that, while restoring 2,3GB dump into 5GB database dir/=disk, the preallocation after 2GBs asks OS for another space in (I think) power of 2 strategy (or something very similar).
This situation hits ours triggers, and eventually should lead to lock of DB (which is still better than fall of whole cluster because of no space left).
Basically same situation we have with 10GB DB. (And in some way we can say that similar situation can occur with literally any capacity...)
My point of question is: Am I able to influence steps (maximum size lower than ~2GB) of (pre)allocation growing of Mongo/WT for restoring (only) or at all?
(Performance impact is not highest prioritized in this case.)
I've found some promising possibilities through db.adminCommand and configStrings for storage engines, but I was not able to affect or list some of possible parameters nor change something to turn things to good...
Especially file_extend parameter of WT through
db.adminCommand( { setParameter: '1', wiredTigerEngineRuntimeConfig: { <parameter> } } )
looked promising but I didn't find way to pass it to cluster...
Some of tries ended with error 22 - Invalid argument, but I wasn't able to find more about...
The other found, seemed suitable for MMAP engine only - newCollectionsUsePowerOf2Sizes=false.
This makes restoring of dumps (eventually backups) into cluster much more bothersome...
Hope that issue is clearly explained and any hint/tip would be appreciated...
If some additional info needed, let me know.
Thank you.
Zdenek
mongodb wiredtiger
mongodb wiredtiger
asked 8 mins ago
Zdenek M.Zdenek M.
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 8 mins ago
Zdenek M.Zdenek M.
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 8 mins ago
Zdenek M.Zdenek M.
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 8 mins ago
Zdenek M.Zdenek M.
1
asked 8 mins ago
Zdenek M.Zdenek M.
1
1
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Zdenek M. is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Zdenek M. is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f234301%2fhow-to-influence-files-preallocation-steps-max-growing-value-in-mongodb-3-4%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Zdenek M. is a new contributor. Be nice, and check out our Code of Conduct.
Zdenek M. is a new contributor. Be nice, and check out our Code of Conduct.
Zdenek M. is a new contributor. Be nice, and check out our Code of Conduct.
Zdenek M. is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f234301%2fhow-to-influence-files-preallocation-steps-max-growing-value-in-mongodb-3-4%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
