While using new_list = my_list
, any modifications to new_list
changes my_list
every time. Why is this, and how can I clone or copy the list to prevent it?
new_list = my_list
just assigns the name new_list
to the object my_list
refers to.
new_list = my_list
doesn't actually create a second list. The assignment just copies the reference to the list, not the actual list, so both new_list
and my_list
refer to the same list after the assignment.
To actually copy the list, you have several options:
You can use the builtin list.copy() method (available since Python 3.3):
new_list = old_list.copy()
You can slice it:
new_list = old_list[:]
Alex Martelli's opinion (at least back in 2007) about this is, that it is a weird syntax and it does not make sense to use it ever. ;) (In his opinion, the next one is more readable).
You can use the built in list() function:
new_list = list(old_list)
You can use generic copy.copy():
import copy
new_list = copy.copy(old_list)
This is a little slower than list()
because it has to find out the datatype of old_list
first.
If you need to copy the elements of the list as well, use generic copy.deepcopy():
import copy
new_list = copy.deepcopy(old_list)
Obviously the slowest and most memory-needing method, but sometimes unavoidable. This operates recursively; it will handle any number of levels of nested lists (or other containers).
Example:
import copy
class Foo(object):
def __init__(self, val):
self.val = val
def __repr__(self):
return f'Foo({self.val!r})'
foo = Foo(1)
a = ['foo', foo]
b = a.copy()
c = a[:]
d = list(a)
e = copy.copy(a)
f = copy.deepcopy(a)
# edit orignal list and instance
a.append('baz')
foo.val = 5
print(f'original: {a}\nlist.copy(): {b}\nslice: {c}\nlist(): {d}\ncopy: {e}\ndeepcopy: {f}')
Result:
original: ['foo', Foo(5), 'baz']
list.copy(): ['foo', Foo(5)]
slice: ['foo', Foo(5)]
list(): ['foo', Foo(5)]
copy: ['foo', Foo(5)]
deepcopy: ['foo', Foo(1)]
Felix already provided an excellent answer, but I thought I'd do a speed comparison of the various methods:
10.59 sec (105.9 µs/itn) - copy.deepcopy(old_list) 10.16 sec (101.6 µs/itn) - pure Python Copy() method copying classes with deepcopy 1.488 sec (14.88 µs/itn) - pure Python Copy() method not copying classes (only dicts/lists/tuples) 0.325 sec (3.25 µs/itn) - for item in old_list: new_list.append(item) 0.217 sec (2.17 µs/itn) - [i for i in old_list] (a list comprehension) 0.186 sec (1.86 µs/itn) - copy.copy(old_list) 0.075 sec (0.75 µs/itn) - list(old_list) 0.053 sec (0.53 µs/itn) - new_list = []; new_list.extend(old_list) 0.039 sec (0.39 µs/itn) - old_list[:] (list slicing)
So the fastest is list slicing. But be aware that copy.copy()
, list[:]
and list(list)
, unlike copy.deepcopy()
and the python version don't copy any lists, dictionaries and class instances in the list, so if the originals change, they will change in the copied list too and vice versa.
(Here's the script if anyone's interested or wants to raise any issues:)
from copy import deepcopy
class old_class:
def __init__(self):
self.blah = 'blah'
class new_class(object):
def __init__(self):
self.blah = 'blah'
dignore = {str: None, unicode: None, int: None, type(None): None}
def Copy(obj, use_deepcopy=True):
t = type(obj)
if t in (list, tuple):
if t == tuple:
# Convert to a list if a tuple to
# allow assigning to when copying
is_tuple = True
obj = list(obj)
else:
# Otherwise just do a quick slice copy
obj = obj[:]
is_tuple = False
# Copy each item recursively
for x in xrange(len(obj)):
if type(obj[x]) in dignore:
continue
obj[x] = Copy(obj[x], use_deepcopy)
if is_tuple:
# Convert back into a tuple again
obj = tuple(obj)
elif t == dict:
# Use the fast shallow dict copy() method and copy any
# values which aren't immutable (like lists, dicts etc)
obj = obj.copy()
for k in obj:
if type(obj[k]) in dignore:
continue
obj[k] = Copy(obj[k], use_deepcopy)
elif t in dignore:
# Numeric or string/unicode?
# It's immutable, so ignore it!
pass
elif use_deepcopy:
obj = deepcopy(obj)
return obj
if __name__ == '__main__':
import copy
from time import time
num_times = 100000
L = [None, 'blah', 1, 543.4532,
['foo'], ('bar',), {'blah': 'blah'},
old_class(), new_class()]
t = time()
for i in xrange(num_times):
Copy(L)
print 'Custom Copy:', time()-t
t = time()
for i in xrange(num_times):
Copy(L, use_deepcopy=False)
print 'Custom Copy Only Copying Lists/Tuples/Dicts (no classes):', time()-t
t = time()
for i in xrange(num_times):
copy.copy(L)
print 'copy.copy:', time()-t
t = time()
for i in xrange(num_times):
copy.deepcopy(L)
print 'copy.deepcopy:', time()-t
t = time()
for i in xrange(num_times):
L[:]
print 'list slicing [:]:', time()-t
t = time()
for i in xrange(num_times):
list(L)
print 'list(L):', time()-t
t = time()
for i in xrange(num_times):
[i for i in L]
print 'list expression(L):', time()-t
t = time()
for i in xrange(num_times):
a = []
a.extend(L)
print 'list extend:', time()-t
t = time()
for i in xrange(num_times):
a = []
for y in L:
a.append(y)
print 'list append:', time()-t
t = time()
for i in xrange(num_times):
a = []
a.extend(i for i in L)
print 'generator expression extend:', time()-t
I've been told that Python 3.3+ adds the list.copy()
method, which should be as fast as slicing:
newlist = old_list.copy()
s.copy()
creates a shallow copy of s
(same as s[:]
).
python3.8
, .copy()
is slightly faster than slicing. See below @AaronsHall answer.
alist.copy()
is now a dict
lookup on the list
type, then a relatively cheap no-arg function call that ultimately invokes the same thing as slicing. Slicing still has to build a slice
object, then go through type checks and unpacking to do the same thing.
What are the options to clone or copy a list in Python?
In Python 3, a shallow copy can be made with:
a_copy = a_list.copy()
In Python 2 and 3, you can get a shallow copy with a full slice of the original:
a_copy = a_list[:]
Explanation
There are two semantic ways to copy a list. A shallow copy creates a new list of the same objects, a deep copy creates a new list containing new equivalent objects.
Shallow list copy
A shallow copy only copies the list itself, which is a container of references to the objects in the list. If the objects contained themselves are mutable and one is changed, the change will be reflected in both lists.
There are different ways to do this in Python 2 and 3. The Python 2 ways will also work in Python 3.
Python 2
In Python 2, the idiomatic way of making a shallow copy of a list is with a complete slice of the original:
a_copy = a_list[:]
You can also accomplish the same thing by passing the list through the list constructor,
a_copy = list(a_list)
but using the constructor is less efficient:
>>> timeit
>>> l = range(20)
>>> min(timeit.repeat(lambda: l[:]))
0.30504298210144043
>>> min(timeit.repeat(lambda: list(l)))
0.40698814392089844
Python 3
In Python 3, lists get the list.copy
method:
a_copy = a_list.copy()
In Python 3.5:
>>> import timeit
>>> l = list(range(20))
>>> min(timeit.repeat(lambda: l[:]))
0.38448613602668047
>>> min(timeit.repeat(lambda: list(l)))
0.6309100328944623
>>> min(timeit.repeat(lambda: l.copy()))
0.38122922903858125
Making another pointer does not make a copy
Using new_list = my_list then modifies new_list every time my_list changes. Why is this?
my_list
is just a name that points to the actual list in memory. When you say new_list = my_list
you're not making a copy, you're just adding another name that points at that original list in memory. We can have similar issues when we make copies of lists.
>>> l = [[], [], []]
>>> l_copy = l[:]
>>> l_copy
[[], [], []]
>>> l_copy[0].append('foo')
>>> l_copy
[['foo'], [], []]
>>> l
[['foo'], [], []]
The list is just an array of pointers to the contents, so a shallow copy just copies the pointers, and so you have two different lists, but they have the same contents. To make copies of the contents, you need a deep copy.
Deep copies
To make a deep copy of a list, in Python 2 or 3, use deepcopy
in the copy
module:
import copy
a_deep_copy = copy.deepcopy(a_list)
To demonstrate how this allows us to make new sub-lists:
>>> import copy
>>> l
[['foo'], [], []]
>>> l_deep_copy = copy.deepcopy(l)
>>> l_deep_copy[0].pop()
'foo'
>>> l_deep_copy
[[], [], []]
>>> l
[['foo'], [], []]
And so we see that the deep copied list is an entirely different list from the original. You could roll your own function - but don't. You're likely to create bugs you otherwise wouldn't have by using the standard library's deepcopy function.
Don't use eval
You may see this used as a way to deepcopy, but don't do it:
problematic_deep_copy = eval(repr(a_list))
It's dangerous, particularly if you're evaluating something from a source you don't trust. It's not reliable, if a subelement you're copying doesn't have a representation that can be eval'd to reproduce an equivalent element. It's also less performant.
In 64 bit Python 2.7:
>>> import timeit
>>> import copy
>>> l = range(10)
>>> min(timeit.repeat(lambda: copy.deepcopy(l)))
27.55826997756958
>>> min(timeit.repeat(lambda: eval(repr(l))))
29.04534101486206
on 64 bit Python 3.5:
>>> import timeit
>>> import copy
>>> l = list(range(10))
>>> min(timeit.repeat(lambda: copy.deepcopy(l)))
16.84255409205798
>>> min(timeit.repeat(lambda: eval(repr(l))))
34.813894678023644
list_copy=[]
for item in list: list_copy.append(copy(item))
and it is much faster.
Let's start from the beginning and explore this question.
So let's suppose you have two lists:
list_1 = ['01', '98']
list_2 = [['01', '98']]
And we have to copy both lists, now starting from the first list:
So first let's try by setting the variable copy
to our original list, list_1
:
copy = list_1
Now if you are thinking copy copied the list_1, then you are wrong. The id
function can show us if two variables can point to the same object. Let's try this:
print(id(copy))
print(id(list_1))
The output is:
4329485320
4329485320
Both variables are the exact same argument. Are you surprised?
So as we know, Python doesn't store anything in a variable, Variables are just referencing to the object and object store the value. Here object is a list
but we created two references to that same object by two different variable names. This means that both variables are pointing to the same object, just with different names.
When you do copy = list_1
, it is actually doing:
https://i.stack.imgur.com/vNGqw.jpg
Here in the image list_1 and copy are two variable names, but the object is same for both variable which is list
.
So if you try to modify copied list then it will modify the original list too because the list is only one there, you will modify that list no matter you do from the copied list or from the original list:
copy[0] = "modify"
print(copy)
print(list_1)
Output:
['modify', '98']
['modify', '98']
So it modified the original list:
Now let's move onto a Pythonic method for copying lists.
copy_1 = list_1[:]
This method fixes the first issue we had:
print(id(copy_1))
print(id(list_1))
4338792136
4338791432
So as we can see our both list having different id and it means that both variables are pointing to different objects. So what actually going on here is:
https://i.stack.imgur.com/iitLk.jpg
Now let's try to modify the list and let's see if we still face the previous problem:
copy_1[0] = "modify"
print(list_1)
print(copy_1)
The output is:
['01', '98']
['modify', '98']
As you can see, it only modified the copied list. That means it worked.
Do you think we're done? No. Let's try to copy our nested list.
copy_2 = list_2[:]
list_2
should reference to another object which is copy of list_2
. Let's check:
print(id((list_2)), id(copy_2))
We get the output:
4330403592 4330403528
Now we can assume both lists are pointing different object, so now let's try to modify it and let's see it is giving what we want:
copy_2[0][1] = "modify"
print(list_2, copy_2)
This gives us the output:
[['01', 'modify']] [['01', 'modify']]
This may seem a little bit confusing, because the same method we previously used worked. Let's try to understand this.
When you do:
copy_2 = list_2[:]
You're only copying the outer list, not the inside list. We can use the id
function once again to check this.
print(id(copy_2[0]))
print(id(list_2[0]))
The output is:
4329485832
4329485832
When we do copy_2 = list_2[:]
, this happens:
https://i.stack.imgur.com/3hPti.jpg
It creates the copy of list, but only outer list copy, not the nested list copy. The nested list is same for both variable, so if you try to modify the nested list then it will modify the original list too as the nested list object is same for both lists.
What is the solution? The solution is the deepcopy
function.
from copy import deepcopy
deep = deepcopy(list_2)
Let's check this:
print(id((list_2)), id(deep))
4322146056 4322148040
Both outer lists have different IDs. Let's try this on the inner nested lists.
print(id(deep[0]))
print(id(list_2[0]))
The output is:
4322145992
4322145800
As you can see both IDs are different, meaning we can assume that both nested lists are pointing different object now.
This means when you do deep = deepcopy(list_2)
what actually happens:
https://i.stack.imgur.com/O7yoo.jpg
Both nested lists are pointing different object and they have separate copy of nested list now.
Now let's try to modify the nested list and see if it solved the previous issue or not:
deep[0][1] = "modify"
print(list_2, deep)
It outputs:
[['01', '98']] [['01', 'modify']]
As you can see, it didn't modify the original nested list, it only modified the copied list.
There are many answers already that tell you how to make a proper copy, but none of them say why your original 'copy' failed.
Python doesn't store values in variables; it binds names to objects. Your original assignment took the object referred to by my_list
and bound it to new_list
as well. No matter which name you use there is still only one list, so changes made when referring to it as my_list
will persist when referring to it as new_list
. Each of the other answers to this question give you different ways of creating a new object to bind to new_list
.
Each element of a list acts like a name, in that each element binds non-exclusively to an object. A shallow copy creates a new list whose elements bind to the same objects as before.
new_list = list(my_list) # or my_list[:], but I prefer this syntax
# is simply a shorter way of:
new_list = [element for element in my_list]
To take your list copy one step further, copy each object that your list refers to, and bind those element copies to a new list.
import copy
# each element must have __copy__ defined for this...
new_list = [copy.copy(element) for element in my_list]
This is not yet a deep copy, because each element of a list may refer to other objects, just like the list is bound to its elements. To recursively copy every element in the list, and then each other object referred to by each element, and so on: perform a deep copy.
import copy
# each element must have __deepcopy__ defined for this...
new_list = copy.deepcopy(my_list)
See the documentation for more information about corner cases in copying.
Use thing[:]
>>> a = [1,2]
>>> b = a[:]
>>> a += [3]
>>> a
[1, 2, 3]
>>> b
[1, 2]
>>>
Python 3.6 Timings
Here are the timing results using Python 3.6.8. Keep in mind these times are relative to one another, not absolute.
I stuck to only doing shallow copies, and also added some new methods that weren't possible in Python 2, such as list.copy()
(the Python 3 slice equivalent) and two forms of list unpacking (*new_list, = list
and new_list = [*list]
):
METHOD TIME TAKEN
b = [*a] 2.75180600000021
b = a * 1 3.50215399999990
b = a[:] 3.78278899999986 # Python 2 winner (see above)
b = a.copy() 4.20556500000020 # Python 3 "slice equivalent" (see above)
b = []; b.extend(a) 4.68069800000012
b = a[0:len(a)] 6.84498999999959
*b, = a 7.54031799999984
b = list(a) 7.75815899999997
b = [i for i in a] 18.4886440000000
b = copy.copy(a) 18.8254879999999
b = []
for item in a:
b.append(item) 35.4729199999997
We can see the Python 2 winner still does well, but doesn't edge out Python 3 list.copy()
by much, especially considering the superior readability of the latter.
The dark horse is the unpacking and repacking method (b = [*a]
), which is ~25% faster than raw slicing, and more than twice as fast as the other unpacking method (*b, = a
).
b = a * 1
also does surprisingly well.
Note that these methods do not output equivalent results for any input other than lists. They all work for sliceable objects, a few work for any iterable, but only copy.copy()
works for more general Python objects.
Here is the testing code for interested parties (Template from here):
import timeit
COUNT = 50000000
print("Array duplicating. Tests run", COUNT, "times")
setup = 'a = [0,1,2,3,4,5,6,7,8,9]; import copy'
print("b = list(a)\t\t", timeit.timeit(stmt='b = list(a)', setup=setup, number=COUNT))
print("b = copy.copy(a)\t", timeit.timeit(stmt='b = copy.copy(a)', setup=setup, number=COUNT))
print("b = a.copy()\t\t", timeit.timeit(stmt='b = a.copy()', setup=setup, number=COUNT))
print("b = a[:]\t\t", timeit.timeit(stmt='b = a[:]', setup=setup, number=COUNT))
print("b = a[0:len(a)]\t\t", timeit.timeit(stmt='b = a[0:len(a)]', setup=setup, number=COUNT))
print("*b, = a\t\t\t", timeit.timeit(stmt='*b, = a', setup=setup, number=COUNT))
print("b = []; b.extend(a)\t", timeit.timeit(stmt='b = []; b.extend(a)', setup=setup, number=COUNT))
print("b = []; for item in a: b.append(item)\t", timeit.timeit(stmt='b = []\nfor item in a: b.append(item)', setup=setup, number=COUNT))
print("b = [i for i in a]\t", timeit.timeit(stmt='b = [i for i in a]', setup=setup, number=COUNT))
print("b = [*a]\t\t", timeit.timeit(stmt='b = [*a]', setup=setup, number=COUNT))
print("b = a * 1\t\t", timeit.timeit(stmt='b = a * 1', setup=setup, number=COUNT))
b=[*a]
- the one obvious way to do it;).
a * 1
method really not possible in 2.x?
Python's idiom for doing this is newList = oldList[:]
All of the other contributors gave great answers, which work when you have a single dimension (leveled) list, however of the methods mentioned so far, only copy.deepcopy()
works to clone/copy a list and not have it point to the nested list
objects when you are working with multidimensional, nested lists (list of lists). While Felix Kling refers to it in his answer, there is a little bit more to the issue and possibly a workaround using built-ins that might prove a faster alternative to deepcopy
.
While new_list = old_list[:]
, copy.copy(old_list)'
and for Py3k old_list.copy()
work for single-leveled lists, they revert to pointing at the list
objects nested within the old_list
and the new_list
, and changes to one of the list
objects are perpetuated in the other.
Edit: New information brought to light
As was pointed out by both Aaron Hall and PM 2Ring using eval() is not only a bad idea, it is also much slower than copy.deepcopy(). This means that for multidimensional lists, the only option is copy.deepcopy(). With that being said, it really isn't an option as the performance goes way south when you try to use it on a moderately sized multidimensional array. I tried to timeit using a 42x42 array, not unheard of or even that large for bioinformatics applications, and I gave up on waiting for a response and just started typing my edit to this post. It would seem that the only real option then is to initialize multiple lists and work on them independently. If anyone has any other suggestions, for how to handle multidimensional list copying, it would be appreciated.
As others have stated, there are significant performance issues using the copy
module and copy.deepcopy
for multidimensional lists.
repr()
is sufficient to re-create the object. Also, eval()
is a tool of last resort; see Eval really is dangerous by SO veteran Ned Batchelder for details. So when you advocate the use eval()
you really should mention that it can be dangerous.
eval()
function in Python in general is a risk. It isn't so much whether or not you make use of the function in code but that it is a security hole in Python in and of itself. My example isn't using it with a function that receives input from input()
, sys.agrv
, or even a text file. It is more along the lines of initializing a blank multidimensional list once, and then just having a way of copying it in a loop instead of reinitializing at each iteration of the loop.
new_list = eval(repr(old_list))
, so besides it being a bad idea, it probably is also way too slow to work.
It surprises me that this hasn't been mentioned yet, so for the sake of completeness...
You can perform list unpacking with the "splat operator": *
, which will also copy elements of your list.
old_list = [1, 2, 3]
new_list = [*old_list]
new_list.append(4)
old_list == [1, 2, 3]
new_list == [1, 2, 3, 4]
The obvious downside to this method is that it is only available in Python 3.5+.
Timing wise though, this appears to perform better than other common methods.
x = [random.random() for _ in range(1000)]
%timeit a = list(x)
%timeit a = x.copy()
%timeit a = x[:]
%timeit a = [*x]
#: 2.47 µs ± 38.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
#: 2.47 µs ± 54.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
#: 2.39 µs ± 58.2 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
#: 2.22 µs ± 43.2 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
old_list
and new_list
are two different lists, editing one will not change the other (unless you’re directly mutating the elements themselves (such as list of list), none-of these methods are deep copies).
new_list = my_list[:]
new_list = my_list
Try to understand this. Let's say that my_list is in the heap memory at location X, i.e., my_list is pointing to the X. Now by assigning new_list = my_list
you're letting new_list point to the X. This is known as a shallow copy.
Now if you assign new_list = my_list[:]
, you're simply copying each object of my_list to new_list. This is known as a deep copy.
The other ways you can do this are:
new_list = list(old_list)
import copy new_list = copy.deepcopy(old_list)
my_list[:]
is a shallow copy. The only way to deepcopy a list is using copy.deepcopy()
A very simple approach independent of python version was missing in already-given answers which you can use most of the time (at least I do):
new_list = my_list * 1 # Solution 1 when you are not using nested lists
However, if my_list contains other containers (for example, nested lists) you must use deepcopy as others suggested in the answers above from the copy library. For example:
import copy
new_list = copy.deepcopy(my_list) # Solution 2 when you are using nested lists
.Bonus: If you don't want to copy elements use (AKA shallow copy):
new_list = my_list[:]
Let's understand difference between solution #1 and solution #2
>>> a = range(5)
>>> b = a*1
>>> a,b
([0, 1, 2, 3, 4], [0, 1, 2, 3, 4])
>>> a[2] = 55
>>> a,b
([0, 1, 55, 3, 4], [0, 1, 2, 3, 4])
As you can see, solution #1 worked perfectly when we were not using the nested lists. Let's check what will happen when we apply solution #1 to nested lists.
>>> from copy import deepcopy
>>> a = [range(i,i+4) for i in range(3)]
>>> a
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
>>> b = a*1
>>> c = deepcopy(a)
>>> for i in (a, b, c): print i
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
>>> a[2].append('99')
>>> for i in (a, b, c): print i
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5, 99]]
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5, 99]] # Solution #1 didn't work in nested list
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]] # Solution #2 - DeepCopy worked in nested list
I wanted to post something a bit different than some of the other answers. Even though this is most likely not the most understandable, or fastest option, it provides a bit of an inside view of how deep copy works, as well as being another alternative option for deep copying. It doesn't really matter if my function has bugs, since the point of this is to show a way to copy objects like the question answers, but also to use this as a point to explain how deepcopy works at its core.
At the core of any deep copy function is way to make a shallow copy. How? Simple. Any deep copy function only duplicates the containers of immutable objects. When you deepcopy a nested list, you are only duplicating the outer lists, not the mutable objects inside of the lists. You are only duplicating the containers. The same works for classes, too. When you deepcopy a class, you deepcopy all of its mutable attributes. So, how? How come you only have to copy the containers, like lists, dicts, tuples, iters, classes, and class instances?
It's simple. A mutable object can't really be duplicated. It can never be changed, so it is only a single value. That means you never have to duplicate strings, numbers, bools, or any of those. But how would you duplicate the containers? Simple. You make just initialize a new container with all of the values. Deepcopy relies on recursion. It duplicates all the containers, even ones with containers inside of them, until no containers are left. A container is an immutable object.
Once you know that, completely duplicating an object without any references is pretty easy. Here's a function for deepcopying basic data-types (wouldn't work for custom classes but you could always add that)
def deepcopy(x):
immutables = (str, int, bool, float)
mutables = (list, dict, tuple)
if isinstance(x, immutables):
return x
elif isinstance(x, mutables):
if isinstance(x, tuple):
return tuple(deepcopy(list(x)))
elif isinstance(x, list):
return [deepcopy(y) for y in x]
elif isinstance(x, dict):
values = [deepcopy(y) for y in list(x.values())]
keys = list(x.keys())
return dict(zip(keys, values))
Python's own built-in deepcopy is based around that example. The only difference is it supports other types, and also supports user-classes by duplicating the attributes into a new duplicate class, and also blocks infinite-recursion with a reference to an object it's already seen using a memo list or dictionary. And that's really it for making deep copies. At its core, making a deep copy is just making shallow copies. I hope this answer adds something to the question.
EXAMPLES
Say you have this list: [1, 2, 3]
. The immutable numbers cannot be duplicated, but the other layer can. You can duplicate it using a list comprehension: [x for x in [1, 2, 3]]
Now, imagine you have this list: [[1, 2], [3, 4], [5, 6]]
. This time, you want to make a function, which uses recursion to deep copy all layers of the list. Instead of the previous list comprehension:
[x for x in _list]
It uses a new one for lists:
[deepcopy_list(x) for x in _list]
And deepcopy_list looks like this:
def deepcopy_list(x):
if isinstance(x, (str, bool, float, int)):
return x
else:
return [deepcopy_list(y) for y in x]
Then now you have a function which can deepcopy any list of strs, bools, floast, ints and even lists to infinitely many layers using recursion. And there you have it, deepcopying.
TLDR: Deepcopy uses recursion to duplicate objects, and merely returns the same immutable objects as before, as immutable objects cannot be duplicated. However, it deepcopies the most inner layers of mutable objects until it reaches the outermost mutable layer of an object.
Note that there are some cases where if you have defined your own custom class and you want to keep the attributes then you should use copy.copy()
or copy.deepcopy()
rather than the alternatives, for example in Python 3:
import copy
class MyList(list):
pass
lst = MyList([1,2,3])
lst.name = 'custom list'
d = {
'original': lst,
'slicecopy' : lst[:],
'lstcopy' : lst.copy(),
'copycopy': copy.copy(lst),
'deepcopy': copy.deepcopy(lst)
}
for k,v in d.items():
print('lst: {}'.format(k), end=', ')
try:
name = v.name
except AttributeError:
name = 'NA'
print('name: {}'.format(name))
Outputs:
lst: original, name: custom list
lst: slicecopy, name: NA
lst: lstcopy, name: NA
lst: copycopy, name: custom list
lst: deepcopy, name: custom list
Remember that in Python when you do:
list1 = ['apples','bananas','pineapples']
list2 = list1
List2 isn't storing the actual list, but a reference to list1. So when you do anything to list1, list2 changes as well. use the copy module (not default, download on pip) to make an original copy of the list(copy.copy()
for simple lists, copy.deepcopy()
for nested ones). This makes a copy that doesn't change with the first list.
A slight practical perspective to look into memory through id and gc.
>>> b = a = ['hell', 'word']
>>> c = ['hell', 'word']
>>> id(a), id(b), id(c)
(4424020872, 4424020872, 4423979272)
| |
-----------
>>> id(a[0]), id(b[0]), id(c[0])
(4424018328, 4424018328, 4424018328) # all referring to same 'hell'
| | |
-----------------------
>>> id(a[0][0]), id(b[0][0]), id(c[0][0])
(4422785208, 4422785208, 4422785208) # all referring to same 'h'
| | |
-----------------------
>>> a[0] += 'o'
>>> a,b,c
(['hello', 'word'], ['hello', 'word'], ['hell', 'word']) # b changed too
>>> id(a[0]), id(b[0]), id(c[0])
(4424018384, 4424018384, 4424018328) # augmented assignment changed a[0],b[0]
| |
-----------
>>> b = a = ['hell', 'word']
>>> id(a[0]), id(b[0]), id(c[0])
(4424018328, 4424018328, 4424018328) # the same hell
| | |
-----------------------
>>> import gc
>>> gc.get_referrers(a[0])
[['hell', 'word'], ['hell', 'word']] # one copy belong to a,b, the another for c
>>> gc.get_referrers(('hell'))
[['hell', 'word'], ['hell', 'word'], ('hell', None)] # ('hello', None)
There is another way of copying a list that was not listed until now: adding an empty list: l2 = l + []
.
I tested it with Python 3.8:
l = [1,2,3]
l2 = l + []
print(l,l2)
l[0] = 'a'
print(l,l2)
It is not the best answer, but it works.
b = [*a]
for short lists.
The deepcopy option is the only method that works for me:
from copy import deepcopy
a = [ [ list(range(1, 3)) for i in range(3) ] ]
b = deepcopy(a)
b[0][1]=[3]
print('Deep:')
print(a)
print(b)
print('-----------------------------')
a = [ [ list(range(1, 3)) for i in range(3) ] ]
b = a*1
b[0][1]=[3]
print('*1:')
print(a)
print(b)
print('-----------------------------')
a = [ [ list(range(1, 3)) for i in range(3) ] ]
b = a[:]
b[0][1]=[3]
print('Vector copy:')
print(a)
print(b)
print('-----------------------------')
a = [ [ list(range(1, 3)) for i in range(3) ] ]
b = list(a)
b[0][1]=[3]
print('List copy:')
print(a)
print(b)
print('-----------------------------')
a = [ [ list(range(1, 3)) for i in range(3) ] ]
b = a.copy()
b[0][1]=[3]
print('.copy():')
print(a)
print(b)
print('-----------------------------')
a = [ [ list(range(1, 3)) for i in range(3) ] ]
b = a
b[0][1]=[3]
print('Shallow:')
print(a)
print(b)
print('-----------------------------')
leads to output of:
Deep:
[[[1, 2], [1, 2], [1, 2]]]
[[[1, 2], [3], [1, 2]]]
-----------------------------
*1:
[[[1, 2], [3], [1, 2]]]
[[[1, 2], [3], [1, 2]]]
-----------------------------
Vector copy:
[[[1, 2], [3], [1, 2]]]
[[[1, 2], [3], [1, 2]]]
-----------------------------
List copy:
[[[1, 2], [3], [1, 2]]]
[[[1, 2], [3], [1, 2]]]
-----------------------------
.copy():
[[[1, 2], [3], [1, 2]]]
[[[1, 2], [3], [1, 2]]]
-----------------------------
Shallow:
[[[1, 2], [3], [1, 2]]]
[[[1, 2], [3], [1, 2]]]
-----------------------------
This is because, the line new_list = my_list
assigns a new reference to the variable my_list
which is new_list
This is similar to the C
code given below,
int my_list[] = [1,2,3,4];
int *new_list;
new_list = my_list;
You should use the copy module to create a new list by
import copy
new_list = copy.deepcopy(my_list)
The method to use depends on the contents of the list being copied. If the list contains nested dicts
than deepcopy is the only method that works, otherwise most of the methods listed in the answers (slice, loop [for], copy, extend, combine, or unpack) will work and execute in similar time (except for loop and deepcopy, which preformed the worst).
Script
from random import randint
from time import time
import copy
item_count = 100000
def copy_type(l1: list, l2: list):
if l1 == l2:
return 'shallow'
return 'deep'
def run_time(start, end):
run = end - start
return int(run * 1000000)
def list_combine(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = [] + l1
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'combine', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_extend(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = []
l2.extend(l1)
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'extend', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_unpack(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = [*l1]
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'unpack', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_deepcopy(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = copy.deepcopy(l1)
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'deepcopy', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_copy(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = list.copy(l1)
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'copy', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_slice(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = l1[:]
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'slice', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_loop(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = []
for i in range(len(l1)):
l2.append(l1[i])
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'loop', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
def list_list(data):
l1 = [data for i in range(item_count)]
start = time()
l2 = list(l1)
end = time()
if type(data) == dict:
l2[0]['test'].append(1)
elif type(data) == list:
l2.append(1)
return {'method': 'list()', 'copy_type': copy_type(l1, l2),
'time_µs': run_time(start, end)}
if __name__ == '__main__':
list_type = [{'list[dict]': {'test': [1, 1]}},
{'list[list]': [1, 1]}]
store = []
for data in list_type:
key = list(data.keys())[0]
store.append({key: [list_unpack(data[key]), list_extend(data[key]),
list_combine(data[key]), list_deepcopy(data[key]),
list_copy(data[key]), list_slice(data[key]),
list_loop(data[key])]})
print(store)
Results
[{"list[dict]": [
{"method": "unpack", "copy_type": "shallow", "time_µs": 56149},
{"method": "extend", "copy_type": "shallow", "time_µs": 52991},
{"method": "combine", "copy_type": "shallow", "time_µs": 53726},
{"method": "deepcopy", "copy_type": "deep", "time_µs": 2702616},
{"method": "copy", "copy_type": "shallow", "time_µs": 52204},
{"method": "slice", "copy_type": "shallow", "time_µs": 52223},
{"method": "loop", "copy_type": "shallow", "time_µs": 836928}]},
{"list[list]": [
{"method": "unpack", "copy_type": "deep", "time_µs": 52313},
{"method": "extend", "copy_type": "deep", "time_µs": 52550},
{"method": "combine", "copy_type": "deep", "time_µs": 53203},
{"method": "deepcopy", "copy_type": "deep", "time_µs": 2608560},
{"method": "copy", "copy_type": "deep", "time_µs": 53210},
{"method": "slice", "copy_type": "deep", "time_µs": 52937},
{"method": "loop", "copy_type": "deep", "time_µs": 834774}
]}]
Success story sharing
[:]
it is a simple and compact syntax and it does make sense to use it every time you need to copy a list and can avoid adeepcopy