Dictionary infinite loop exiting unexpectedly












11















I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}

for i in x:
del x[i]
x[i+1] = None # Value doesn't matter, so I set it to None
print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0
1
2
3
4
5
.
.
.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0
1
2
3
4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?










share|improve this question




















  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    1 hour ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    50 mins ago






  • 2





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    43 mins ago
















11















I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}

for i in x:
del x[i]
x[i+1] = None # Value doesn't matter, so I set it to None
print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0
1
2
3
4
5
.
.
.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0
1
2
3
4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?










share|improve this question




















  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    1 hour ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    50 mins ago






  • 2





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    43 mins ago














11












11








11


1






I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}

for i in x:
del x[i]
x[i+1] = None # Value doesn't matter, so I set it to None
print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0
1
2
3
4
5
.
.
.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0
1
2
3
4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?










share|improve this question
















I was experimenting with various ways of creating an infinite loop in Python (other than the usual while True), and came up with this idea:



x = {0: None}

for i in x:
del x[i]
x[i+1] = None # Value doesn't matter, so I set it to None
print(i)


On paper, I traced out the way this would infinitely loop:




  1. I loop through the key's value in the dictionary

  2. I delete that entry.

  3. The current counter position in the loop + 1 will be the new key with value None which updates the dictionary.

  4. I output the current counter.


This, in my head, should output the natural numbers in a sort of infinite loop fashion:



0
1
2
3
4
5
.
.
.


I thought this idea was clever, however when I run it on Python 3.6, it outputs:



0
1
2
3
4


Yes, it somehow stopped after 5 iterations. Clearly, there is no base condition or sentinel value in the code block of the loop, so why is Python only running this code 5 times?







python dictionary infinite-loop






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 25 mins ago









benvc

4,6571321




4,6571321










asked 1 hour ago









Suraj KothariSuraj Kothari

686416




686416








  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    1 hour ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    50 mins ago






  • 2





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    43 mins ago














  • 4





    Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

    – jasonharper
    1 hour ago






  • 1





    @jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

    – user2357112
    50 mins ago






  • 2





    You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

    – juanpa.arrivillaga
    43 mins ago








4




4





Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

– jasonharper
1 hour ago





Mutating an object that you're iterating over is pretty much guaranteed to give you unpredictable results. In your specific case, I would guess that the 4 went into a hash bucket that had already been iterated over.

– jasonharper
1 hour ago




1




1





@jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

– user2357112
50 mins ago





@jasonharper: CPython 3.6 is when they introduced the new compact dict implementation, so bucket locations wouldn't have this effect on 3.6. This looks like the dict needed a rebuild after the 5 was inserted, which would have re-compacted the compact array of entries and shifted the entry for the 5 to the left of the iterator position.

– user2357112
50 mins ago




2




2





You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

– juanpa.arrivillaga
43 mins ago





You are making unwarranted assumptions about the dictionary iterator. The only assumption that you should make is that it isn't guaranteed to work if you add/remove from the dictionary while iterating over it. What actually ends up happening will depend on various implementation details.

– juanpa.arrivillaga
43 mins ago












2 Answers
2






active

oldest

votes


















15














There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count

for i in count():
print(i)
# don't run this without some mechanism to break the loop, i.e.
# if (i == 10):
# break

# OUTPUT
# 0
# 1
# 2
# ...and so on





share|improve this answer


























  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    40 mins ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    35 mins ago





















3














I just tested your code in python2 and python3



python3 output
0,1,2,3,4
python2
0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer


























  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    1 hour ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    1 hour ago











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54316557%2fdictionary-infinite-loop-exiting-unexpectedly%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes









15














There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count

for i in count():
print(i)
# don't run this without some mechanism to break the loop, i.e.
# if (i == 10):
# break

# OUTPUT
# 0
# 1
# 2
# ...and so on





share|improve this answer


























  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    40 mins ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    35 mins ago


















15














There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count

for i in count():
print(i)
# don't run this without some mechanism to break the loop, i.e.
# if (i == 10):
# break

# OUTPUT
# 0
# 1
# 2
# ...and so on





share|improve this answer


























  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    40 mins ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    35 mins ago
















15












15








15







There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count

for i in count():
print(i)
# don't run this without some mechanism to break the loop, i.e.
# if (i == 10):
# break

# OUTPUT
# 0
# 1
# 2
# ...and so on





share|improve this answer















There is no guarantee that you will iterate over all your dict entries if you mutate it in your loop. From the docs:




Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.




You could create an "enumerated" infinite loop similar to your initial attempt using itertools.count(). For example:



from itertools import count

for i in count():
print(i)
# don't run this without some mechanism to break the loop, i.e.
# if (i == 10):
# break

# OUTPUT
# 0
# 1
# 2
# ...and so on






share|improve this answer














share|improve this answer



share|improve this answer








edited 24 mins ago

























answered 1 hour ago









benvcbenvc

4,6571321




4,6571321













  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    40 mins ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    35 mins ago





















  • Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

    – Suraj Kothari
    40 mins ago






  • 1





    @SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

    – benvc
    35 mins ago



















Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

– Suraj Kothari
40 mins ago





Thanks for the tip on dictionaries, but the code below was sort of unnecessary. I didn't really want an infinite loop in my code, just a fun project of coming up with odd ways, but turns out mines didn't work. Thanks anyways.

– Suraj Kothari
40 mins ago




1




1





@SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

– benvc
35 mins ago







@SurajKothari - added the example in case someone else ends up looking at your question in an effort to generate an "enumerated" infinite loop (or more usefully an indefinite loop where the breakpoint is determined by some other operation at runtime) similar to your initial attempt.

– benvc
35 mins ago















3














I just tested your code in python2 and python3



python3 output
0,1,2,3,4
python2
0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer


























  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    1 hour ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    1 hour ago
















3














I just tested your code in python2 and python3



python3 output
0,1,2,3,4
python2
0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer


























  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    1 hour ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    1 hour ago














3












3








3







I just tested your code in python2 and python3



python3 output
0,1,2,3,4
python2
0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.






share|improve this answer















I just tested your code in python2 and python3



python3 output
0,1,2,3,4
python2
0,1,2,3,4,5,6,7


One thing comes to mind that could be going on. Either there is only a certain amount of memory being allocated in your dictionary when you create the first key value and when you delete the key value we do not allocate any memory or deallocate the memory you are just removing the value. Once all the allocated memory is used it exits. Because if you run without that del you will get this error



RuntimeError: dictionary changed size during iteration


So python creates enough memory for that key value and a few more and once it is used up theres no more memory allocated for your dictionary.







share|improve this answer














share|improve this answer



share|improve this answer








edited 1 hour ago

























answered 1 hour ago









turtlesalldayturtlesallday

271112




271112













  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    1 hour ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    1 hour ago



















  • Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

    – Suraj Kothari
    1 hour ago






  • 1





    I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

    – Suraj Kothari
    1 hour ago

















Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

– Suraj Kothari
1 hour ago





Are you sure Python 3 gives 8 iterations? Mines gave 5 (0 to 4) in Python 3.6.1

– Suraj Kothari
1 hour ago




1




1





I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

– Suraj Kothari
1 hour ago





I just tested it and perhaps you have the versions the other way around. Python 2.7 gives (0 to 7) and Python 3 gives (0 to 4)

– Suraj Kothari
1 hour ago


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54316557%2fdictionary-infinite-loop-exiting-unexpectedly%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

SQL Server 17 - Attemping to backup to remote NAS but Access is denied

Always On Availability groups resolving state after failover - Remote harden of transaction...

Restoring from pg_dump with foreign key constraints