Getting past corrupted binary log “Error in Log_event::read_log_event():”
I have a binary log that mysqlbinlog chokes on with the error in the title.
The file itself has much more activity after the cited position.
Doing some basic confirmation it's not all garbage by running it through the strings command shows theres legit traffic until the end of the file when it got rotated.
I've seen a similar post about using hexdump to get past an error related to event too large, but in my case mysqlbinlog chokes to continue to get further information. I'm not familiar enough with the binary format to look for what might be a position of a next event it would recognize.
It gives a starting position it can't get past so I have a script running to basically mysqlbinlog --start-position=X incrementing X by one until it returns with a 0 exit code but that looks like it's going to take a month to completely get through everything at this rate.
I tested the POC of this idea on "good parts" by starting it at weird offsets and it returned correctly at the next one it found w/o error.
I'm running percona 5.6.20 for this instance.
I realize this report might be lacking in information needed to answer the question so I'm happy to edit with comment requests as needed.
mysql replication mysql-5.6 disaster-recovery
bumped to the homepage by Community♦ 19 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a binary log that mysqlbinlog chokes on with the error in the title.
The file itself has much more activity after the cited position.
Doing some basic confirmation it's not all garbage by running it through the strings command shows theres legit traffic until the end of the file when it got rotated.
I've seen a similar post about using hexdump to get past an error related to event too large, but in my case mysqlbinlog chokes to continue to get further information. I'm not familiar enough with the binary format to look for what might be a position of a next event it would recognize.
It gives a starting position it can't get past so I have a script running to basically mysqlbinlog --start-position=X incrementing X by one until it returns with a 0 exit code but that looks like it's going to take a month to completely get through everything at this rate.
I tested the POC of this idea on "good parts" by starting it at weird offsets and it returned correctly at the next one it found w/o error.
I'm running percona 5.6.20 for this instance.
I realize this report might be lacking in information needed to answer the question so I'm happy to edit with comment requests as needed.
mysql replication mysql-5.6 disaster-recovery
bumped to the homepage by Community♦ 19 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a binary log that mysqlbinlog chokes on with the error in the title.
The file itself has much more activity after the cited position.
Doing some basic confirmation it's not all garbage by running it through the strings command shows theres legit traffic until the end of the file when it got rotated.
I've seen a similar post about using hexdump to get past an error related to event too large, but in my case mysqlbinlog chokes to continue to get further information. I'm not familiar enough with the binary format to look for what might be a position of a next event it would recognize.
It gives a starting position it can't get past so I have a script running to basically mysqlbinlog --start-position=X incrementing X by one until it returns with a 0 exit code but that looks like it's going to take a month to completely get through everything at this rate.
I tested the POC of this idea on "good parts" by starting it at weird offsets and it returned correctly at the next one it found w/o error.
I'm running percona 5.6.20 for this instance.
I realize this report might be lacking in information needed to answer the question so I'm happy to edit with comment requests as needed.
mysql replication mysql-5.6 disaster-recovery
I have a binary log that mysqlbinlog chokes on with the error in the title.
The file itself has much more activity after the cited position.
Doing some basic confirmation it's not all garbage by running it through the strings command shows theres legit traffic until the end of the file when it got rotated.
I've seen a similar post about using hexdump to get past an error related to event too large, but in my case mysqlbinlog chokes to continue to get further information. I'm not familiar enough with the binary format to look for what might be a position of a next event it would recognize.
It gives a starting position it can't get past so I have a script running to basically mysqlbinlog --start-position=X incrementing X by one until it returns with a 0 exit code but that looks like it's going to take a month to completely get through everything at this rate.
I tested the POC of this idea on "good parts" by starting it at weird offsets and it returned correctly at the next one it found w/o error.
I'm running percona 5.6.20 for this instance.
I realize this report might be lacking in information needed to answer the question so I'm happy to edit with comment requests as needed.
mysql replication mysql-5.6 disaster-recovery
mysql replication mysql-5.6 disaster-recovery
asked Feb 10 '17 at 2:16
atxdbaatxdba
3,89842654
3,89842654
bumped to the homepage by Community♦ 19 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 19 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Well much to my surprise the script I mentioned did find a good starting position and I was able to successfully recover a large portion of the binary log after the problem point.
Not the most elegant script but in case it's of use to anyone else. Note it is very slow restarting a new process to scan byte by byte but got me results.
And yes I recognize the potential of infinite looping by never finding anything, might want to add additional check based on start position vs filesize
#!/bin/bash
BINLOG=$1
START=$2
RET=1
while [ $RET -eq 1 ]; do
# This is relying on the mysqlbinlog found in your path
# You'll want to make sure this resolves to a binary version matching the mysqld that generated the binlog
mysqlbinlog --start-position=$START $BINLOG > /dev/null 2>&1
RET=$?
START=$(( START + 1 ))
if [[ $(( START % 1000 )) -eq 0 ]]; then
echo $START
fi
done
GOODSTART=$(( START -1 ))
echo "FOUND GOOD START AT $GOODSTART"
# Saving it to a file in case youre running it overnight, script is very slow.
echo $GOODSTART > ${BINLOG}.goodstartposition
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f163834%2fgetting-past-corrupted-binary-log-error-in-log-eventread-log-event%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Well much to my surprise the script I mentioned did find a good starting position and I was able to successfully recover a large portion of the binary log after the problem point.
Not the most elegant script but in case it's of use to anyone else. Note it is very slow restarting a new process to scan byte by byte but got me results.
And yes I recognize the potential of infinite looping by never finding anything, might want to add additional check based on start position vs filesize
#!/bin/bash
BINLOG=$1
START=$2
RET=1
while [ $RET -eq 1 ]; do
# This is relying on the mysqlbinlog found in your path
# You'll want to make sure this resolves to a binary version matching the mysqld that generated the binlog
mysqlbinlog --start-position=$START $BINLOG > /dev/null 2>&1
RET=$?
START=$(( START + 1 ))
if [[ $(( START % 1000 )) -eq 0 ]]; then
echo $START
fi
done
GOODSTART=$(( START -1 ))
echo "FOUND GOOD START AT $GOODSTART"
# Saving it to a file in case youre running it overnight, script is very slow.
echo $GOODSTART > ${BINLOG}.goodstartposition
add a comment |
Well much to my surprise the script I mentioned did find a good starting position and I was able to successfully recover a large portion of the binary log after the problem point.
Not the most elegant script but in case it's of use to anyone else. Note it is very slow restarting a new process to scan byte by byte but got me results.
And yes I recognize the potential of infinite looping by never finding anything, might want to add additional check based on start position vs filesize
#!/bin/bash
BINLOG=$1
START=$2
RET=1
while [ $RET -eq 1 ]; do
# This is relying on the mysqlbinlog found in your path
# You'll want to make sure this resolves to a binary version matching the mysqld that generated the binlog
mysqlbinlog --start-position=$START $BINLOG > /dev/null 2>&1
RET=$?
START=$(( START + 1 ))
if [[ $(( START % 1000 )) -eq 0 ]]; then
echo $START
fi
done
GOODSTART=$(( START -1 ))
echo "FOUND GOOD START AT $GOODSTART"
# Saving it to a file in case youre running it overnight, script is very slow.
echo $GOODSTART > ${BINLOG}.goodstartposition
add a comment |
Well much to my surprise the script I mentioned did find a good starting position and I was able to successfully recover a large portion of the binary log after the problem point.
Not the most elegant script but in case it's of use to anyone else. Note it is very slow restarting a new process to scan byte by byte but got me results.
And yes I recognize the potential of infinite looping by never finding anything, might want to add additional check based on start position vs filesize
#!/bin/bash
BINLOG=$1
START=$2
RET=1
while [ $RET -eq 1 ]; do
# This is relying on the mysqlbinlog found in your path
# You'll want to make sure this resolves to a binary version matching the mysqld that generated the binlog
mysqlbinlog --start-position=$START $BINLOG > /dev/null 2>&1
RET=$?
START=$(( START + 1 ))
if [[ $(( START % 1000 )) -eq 0 ]]; then
echo $START
fi
done
GOODSTART=$(( START -1 ))
echo "FOUND GOOD START AT $GOODSTART"
# Saving it to a file in case youre running it overnight, script is very slow.
echo $GOODSTART > ${BINLOG}.goodstartposition
Well much to my surprise the script I mentioned did find a good starting position and I was able to successfully recover a large portion of the binary log after the problem point.
Not the most elegant script but in case it's of use to anyone else. Note it is very slow restarting a new process to scan byte by byte but got me results.
And yes I recognize the potential of infinite looping by never finding anything, might want to add additional check based on start position vs filesize
#!/bin/bash
BINLOG=$1
START=$2
RET=1
while [ $RET -eq 1 ]; do
# This is relying on the mysqlbinlog found in your path
# You'll want to make sure this resolves to a binary version matching the mysqld that generated the binlog
mysqlbinlog --start-position=$START $BINLOG > /dev/null 2>&1
RET=$?
START=$(( START + 1 ))
if [[ $(( START % 1000 )) -eq 0 ]]; then
echo $START
fi
done
GOODSTART=$(( START -1 ))
echo "FOUND GOOD START AT $GOODSTART"
# Saving it to a file in case youre running it overnight, script is very slow.
echo $GOODSTART > ${BINLOG}.goodstartposition
answered Feb 11 '17 at 1:11
atxdbaatxdba
3,89842654
3,89842654
add a comment |
add a comment |
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f163834%2fgetting-past-corrupted-binary-log-error-in-log-eventread-log-event%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown