1. Varaibles Are Labels, Not Boxes.
Python varaibles are like reference variables in Java, so it's better to think of them as labels attached to objects.
Example 8-1. Variables a and b hold references to the same list, not copies of the list.
>>> a = [1, 2, 3] >>> b = a >>> a.append(4) >>> b [1, 2, 3, 4] # a 和 b 这两个引用指向的是同一个 list 对象， b 并不是 a 的复本。
With reference variables, it makes much more sense to day that varaible is assigned to an object, and not the other way around. After all, the object is created before the assignment. Example 8-2 proves that the righthand side of an assignment happens first.
Example 8-2. Variables are assigned to objects only after the objects are created.
>>> class Gizmo: ... def __init__(self): ... print('Gizmo id: %d' % id(self)) ... >>> x = Gizmo() Gizmo id: 4357027432 >>> y = Gizmo() * 10 Gizmo id: 4357027544 # Here is proof that a second Gizmo was actually instantiated before the multiplication was attempted. Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for *: 'Gizmo' and 'int' >>> >>> dir() # Variable y was never created, because the exception happened while the righthand side of the assignment was being evaluated. ['Gizmo', '__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'x'] # To understand an assignment in Python, always read the righthand side first: that's where the object is created or retrieved. After that, the variable on the left is bound to the object, like a label stuck to it.
Because varaibles are mere labels, nothing prevents an object from having several labels assigned to it. When that happens, you have aliasing.
2. Identity, Equality, and Aliases
Example 8-3. charles and lewis refer to the same object.
>>> charles = {'name':'Charles L. Dodgson', 'born':1832}
>>> lewis = charles # lewis is an alias for charles
>>> lewis is charles
True
>>> id(charles), id(lewis)
(4355163984, 4355163984)
>>> lewis['balance'] = 950 # Adding an item to lewis is the same as adding an item to charles
>>> charles
{'name': 'Charles L. Dodgson', 'born': 1832, 'balance': 950}
>>> alex = {'name': 'Charles L. Dodgson', 'born': 1832, 'balance': 950} # alex refers to an object that is a replica of the object assigned to charles
>>> alex == charles
True
>>> alex is not charles # They are distinct objects.
True
In the example above, lewis and charles are aliases: two variables bound to the same object. On the other hand, alex is not an alias for charles: these variables are bound to distinct objects. The objects to alex and charles have the same value -- that's what == compares -- but they have different identities.
# Every object has an identity, a type and a value. An object's identity never changes once it has been created; you may think of it as the object's address in memory. The is operator compares the identity of two objects; the id() function returns an integer representing its identity.
3. Choosing Between == and is
The == operator compares the values of objects (the data they hold), while is compares their identities.
If you are comparing a variable to a singleton, then it makes sense to use is. By far, the most common case is checking whether a variable is bound to None.
The is operator is faster than ==, because it cannot be overloaded, so Python does not have to find and invoke special methods to evaluate it, and computing is as simple as comparing two integer IDs. In contrast, a == b is syntactic sugar for a.__eq__(b) .
4. Copies Are Shallow by Default
A copy is an equal object with a different ID.
The easiesg way to copy a list (or most built-in mutable collections) is to use the built-in constructor for the type itself. For example:
>>> l1 = [3, [55, 44], (7, 8, 9)] >>> l2 = list(l1) # list(l1) creates a copy of l1. >>> l2 [3, [55, 44], (7, 8, 9)] >>> l2 == l1 True >>> l2 is l1 False # For lists and other mutable sequences, the shortcut l2 = li1[:] also make a copy. # Using the constructor or [:] produces a shallow copy: the outermost container is duplicated, but the copy is filled with references to the same items held by the original container.
5. Deep and Shallow Copies of Arbitrary Objects
deep copy: duplicates that do not share references of embedded objects. The copy module provides the deepcopy and copy functions that return deep and shallow copies of arbitrary objects.
Example 8-8. Bus picks up and drops off passengers and Effects of using copy versus deepcopy.
class Bus: def __init__(self, passengers=None): if passengers is None: self.passengers = [] else: self.passengers = list(passengers) def pick(self, name): self.passengers.append(name) def drop(self, name): self.passengers.remove(name) """ >>> import copy >>> bus1 = Bus(['Alice', 'Bill', 'Claire', 'David']) >>> bus2 = copy.copy(bus1) >>> bus3 = copy.deepcopy(bus1) >>> id(bus1), id(bus2), id(bus3) # Using copy and deepcopy, we create three distinct Bus instances. (4350768184, 4350768688, 4350768912) >>> bus1.drop('Bill') >>> bus2.passengers ['Alice', 'Claire', 'David'] # After bus1 drops 'Bill', he is also missing from bus2 >>> id(bus1.passengers), id(bus2.passengers), id(bus3.passengers) (4350754056, 4350754056, 4350751752) # Inspection of the passengers attributes shows that bus1 and bus2 share the same list object, because bus2 is a shallow copy of bus1. >>> bus3.passengers ['Alice', 'Bill', 'Claire', 'David'] # bus3 is a deep copy of bus1, so its passengers attribute refers to another list. """
6. Function Parameters as References
The only mode of parameter passing in Python is call by sharing（共享传参）. Call by sharing means that each formal parameter of the function gets a copy of each reference in the arguments. In other words, the parameters inside the function become aliases of the actual arguments.
The result of this scheme is that a function may change any mutable object passed as a parameters, but it cannot change the identity of those objects.
Example 8-11. A function may change any mutable object it receives.
"""A function may change any mutable object it receives""" >>> def f(a, b): ... a += b ... return a ... >>> x = 1 >>> y = 2 >>> f(x, y) 3 >>> x, y # The number x is unchanged. (1, 2) >>> >>> a = [1, 2] >>> b = [3, 4] >>> f(a, b) [1, 2, 3, 4] >>> a, b # The list a is changed. ([1, 2, 3, 4], [3, 4]) >>> >>> >>> t = (10, 20) >>> u = (30, 40) >>> f(t, u) (10, 20, 30, 40) >>> t, u # The tuple t is unchanged. ((10, 20), (30, 40)) >>>
Python 函数传参参考：https://www.cnblogs.com/zhoug2020/p/9110663.html
7. Mutable Types as Parameter Defaults: Bad idea
You should avoid mutable objects as default values for parameters.
EXample 8-12. A simple class to illustrate the danger of a mutable default.
class HauntedBus: def __init__(self, passengers=[]): # When the passengers argument is not given, this parameter is bound to the default list object, which is initially empty. self.passengers = passengers # This assignment makes self.passenger an alias for passengers, which is itself an alias for the default list, when no passengers argument is given. def pick(self, name): self.passengers.append(name) # When the methods .remove() and .append() are used with self.passengers, we are actually mutating the default list, which is an attribute of the function object. def drop(self, name): self.passengers.remove(name) # Example 8-13 shows the eerie behavior of the HauntedBus """ >>> bus1 = HauntedBus(['Alice', 'Bill']) >>> bus1.passengers ['Alice', 'Bill'] >>> bus1.pick('Charlie') >>> bus1.drop('Alice') >>> bus1.passengers # bus1 正常运行 ['Bill', 'Charlie'] >>> >>> bus2 = HauntedBus() # bus2 starts empty, so the default empty list is assigned to self.passengers. >>> bus2.pick('Carrie') >>> bus2.passengers ['Carrie'] >>> >>> bus3 = HauntedBus() # bus3 also starts empty, again the default list is assigned. >>> bus3.passengers # The default is no longer empty! ['Carrie'] >>> bus3.pick('Dave') >>> bus2.passengers # Now Dave, picked by bus3, appears in bus2. ['Carrie', 'Dave'] >>> >>> bus2.passengers is bus3.passengers # bus2.passengers and bus3.passengers refer to the same list. True >>> bus1.passengers # But bus1.passengers is a distinct list. ['Bill', 'Charlie'] """ # The problem is that Bus instances that don't get an initial passenger list end up sharing the same passenger list among themselves. """ Strange things happen only when a HauntedBus starts empty, because then self.passengers becomes am alias for the default value of the passengers parameter. The problem is that each default value is evaluated when the function is defined -- i.e., usually when the module is loaded -- and the default values become attributes of the function object. So if a default value is a mutable object, and you change it, the change will affect every future call of the function. """ # Inspect the HauntedBus.__init__ object and see the students haunting its __defaults__ attribute: """ >>> dir(HauntedBus.__init__) ['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__'] >>> HauntedBus.__init__.__defaults__ (['Carrie', 'Dave'],) >>> """ # Finally, we can verify that bus2.passengers is an alias bound to the first element of the HauntedBus.__init__.__defaults__ attributes: """ >>> HauntedBus.__init__.__defaults__[0] is bus2.passengers True """ # The issue with mutable defaults explains why None is often used as the default value for parameters that may receive # mutable values.
8. Defensive Programming with Mutable Paramters
When you are coding a function that reveives a mutable parameter, you should carefully consider whether the caller expects the argument passed to be changed.
Example 8-15. A simple class to show the perils of mutating received arguments.
class TwilightBus: def __init__(self, passengers=None): if passengers is None: self.passengers = [] else: self.passengers = passengers # This assignment makes self.passengers an alias for passengers, which is itself an alias for the actual argument passed to __init__ (self.passengers , passengers 和 传入的实参互为 alias) def pick(self, name): self.passengers.append(name) def drop(self, name): self.passengers.remove(name) # When the methods .remove() and .append() are used with self.passengers, we are actually mutating the original list received as argument to the constructor. # 运行结果：（这种 赋值方式的问题） """ >>> basketball_team = ['Sue', 'Tina', 'Maya', 'Diana', 'Pat'] >>> bus = TwilightBus(basketball_team) >>> bus.drop('Pat') >>> basketball_team # 问题： The dropped passenger vanished from the basketball_team ['Sue', 'Tina', 'Maya', 'Diana'] """ # The TwilightBus of this Example violates the "Principle of least astonishment", a best practice of interface design. # It surely is astonishing that when the bus drops a student, her name is removed from the basketball team roster. """ The problem here is that the bus is aliasing the list that is passed to the constructor. Instead, it should keep its own passenger list. The fix is simple: in __init__ , when the passengers parameters is provided, self.passengers should be initialized with a copy of it, as we did correctly in Example 8-8: def __init__(self, passengers=None): if passengers is None: self.passengers = [] else: self.passengers = list(passengers) # Make a copy of the passengers list, or convert it to a list if it's not one. Now our internal handling of the passenger list will not affect the argument used to initialize the bus. As a bonus, this solution is more flexible: now the argument passed to the passengers parameter may be a tuple or any other iterable, like a set or even database result, because the list constructor accepts any iterable. """ # Unless a method is explicitly intended to mutate an object received as argument, you should think twice before aliasing # the argument object by simply assigning it to an instance variable in your class. If in doubt, make a copy. Your clients will often be happier.
9. del and Garbage Collection
Example 8-16. Watching the end of an object when no more references point to it.
""" The del statement deletes names, not objects. An object may be garbage collected as result of a del command, but only if the variable deleted holds the last reference to teh object, or if the object becomes unreachable. Rebinding a variable may also cause the number of references to an object to reach zero, causing its destruction. In CPython, the primary algorithm for garbage collection is reference counting. Each object keeps count of how many references point to it. As soon as that refcount reaches zero, the object is immediately destroyed: CPython calls the __del__ method on the object (if defined) and then frees the memory allocated to the object. """ >>> import weakref >>> s1 = {1, 2, 3} >>> s2 = s1 # s1 and s2 are aliases referring to the same set, {1, 2, 3}. >>> def bye(): # This function must not be a bound method of the object about to be destroyed or otherwise hold a reference to it. ... print('Gone with the wind...') ... >>> ender = weakref.finalize(s1, bye) # Register the bye callback on the object referred by s1. >>> ender.alive # The .alive attribute is True before the finalize object is called. True >>> del s1 >>> ender.alive # del does not delete an object, just a reference to it. True >>> s2 = 'spam' # Rebinding the last reference, s2, makes {1, 2, 3} unreachable. It is destroyed, the bye callback is invoked, and ender.alive becomes False. Gone with the wind... >>> ender.alive False >>> # This example makes explicit that del does not delete objects, but objects may be deleted as a consequence of being unreachable after del is used.
10. Weak References
Example 8-17. A weak reference is a callable that returns the referenced object or None if the referent is no more.
""" Weak references to an object do not increase its reference count. The object that is the target of a reference is called the referent. Therefore, we say that a weak reference does not prevent the referent from being garbage collected. Weak references are useful in caching applications because you don't want the cached objects to be kept alive just because they are referenced by the cache. """ # The Python console automatically binds the _ variable to the result of expressions that are not None. (Python 控制台会把不为None的表达式赋值给 _ 变量) >>> import weakref >>> a_set = {0, 1} >>> wref = weakref.ref(a_set) # The wref weak reference object is created. >>> wref <weakref at 0x103b2ef48; to 'set' at 0x103b2d2e8> >>> wref() # Invoking wref() returns the referenced object, {0,1}. Because this is a console session, the result {0,1} is bound to the _ variable. {0, 1} >>> _ # _ 的值为 {0,1} {0, 1} >>> a_set = {2,3,4} # a_set no longer refers to the {0,1} set, so its reference count is decreased. But the _ variable still refers to it. >>> wref() # Calling wref() still returns {0,1} {0, 1} >>> wref() is None # When this expression is evaluated, {0,1} lives, therefore wref() is not None. But _ is then bound to the resulting value, False. Now there are no more strong references to {0,1}. False >>> _ False >>> wref() is None # Because the {0,1} object is now gone, this last call to wref() returns None. True # Consider using WeakKeyDictionary, WeakValueDictionary, WeakSet, and finalize(which use weak references internally) instead of creating # and handling your own weakref.ref instances by hand. # In practice, most of the time Python programs use the weakref collections.
10.2 The WeakValueDictionary Skit
Example 8-18. Cheese has a kind attribute and a standard representation.
""" The class WeakValueDictionary implements a mutable mapping where the values are weak references to object. When a referred object is garbage collected elsewhere in the program, the corresponding key is automatically removed from WeakValueDictionary. This is commonly used for caching. """ class Cheese: def __init__(self, kind): self.kind = kind def __repr__(self): return 'Cheese(%r)' % self.kind # 运行结果： Each cheese is loaded from a catalog to stock implemented as a WeakValueDictionary. """ >>> import weakref >>> >>> stock = weakref.WeakValueDictionary() # stock is a WeakKeyDictionary >>> catalog = [Cheese('Red Leicester'), Cheese('Tilsit'),Cheese('Brie'), Cheese('Parmesan')] >>> for cheese in catalog: ... stock[cheese.kind] = cheese # The stock maps the name of the cheese to a weak reference to the cheese instance in the catalog. ... >>> sorted(stock.keys()) ['Brie', 'Parmesan', 'Red Leicester', 'Tilsit'] >>> del catalog >>> sorted(stock.keys()) # Why not all ??? ['Parmesan'] >>> del cheese >>> sorted(stock.keys()) [] """ # A temporary variable may cause an object to last longer than expected by holding a reference to it. This is usually not # a problem with local variables: they are destroyed when the function returns. But in this example, the for loop variable # cheese is a global variable and will never go away unless explicitly deleted.
A counterpart to the WeakValueDictionary is the WeakKeyDictionary in which the keys are weak references.
The weakref module also provides a WeakSet, simply described in the docs as "Set class that keeps weak references to ths elements. An element will be discarded when no strong reference to it exists any more". If you need to build a class that is aware of every one of its instances, a good solution is to create a class attribute with a WeakSet to hold the references to the instances. Otherwise, if a regular set was used, the instances wouldf never be garbage collected, because the class itself would have strong references to them, and classes live as long as the Python process unless you deliberately delete them.
10.3 Limitations of Weak References
Not every Python object may be the target, or referent, of a weak reference. Basic list and dict instances may not be referents, but a plain subclass of either can solve this problem easily:
import weakref class MyList(list): """list subclass whose instance may be weakly referenced""" a_list = MyList(range(10)) # a_list can be the target of a weak reference. wref_to_a_list = weakref.ref(a_list)
A set instance can be a referent, and that's why a set was used in Example 8-17. User-defined types also pose no problem, which the silly Cheese class was needed in Example 8-19. But int and tuple instances cannot be targets of weak references, even if subclasses of those types are created.
11. Tricks Python Plays with Immutables
For a tuple t, t[:] does not make a copy, but returns a reference to the same object. You also get a reference to the same tuple if you write tuple(t).
The same behavior can be observed with instances of str, bytes, and frozenset. A frozenset is not a sequence, so fs[:] does not work if fs is a frozenset. But fs.copy() has the same effect: it returns a reference to the same object, and not a coypy at all.
12. Chapter Summary
Every Python object has an identity, a type, and a value. Only the value of an object changes over time.(Actually the type of an object may be changed by merely assigning a different class to its __class__ attribute, but that is pure evil)
If two variables refer to immutable objects that have equal values (a == b is True), in practice it rarely matters if they refer to copies or are aliases referring to the same object because the value of an immutable object does not change, with one exception. The exeption is immutable collections such as tuples and frozensets: if an immutable collection holds references to mutable items, then its value may actually change when the value of a mutable item changes. In practice, this scenaria is not so common. What never changes in an immutable collection are the identities of the objects within.
The fact that variables hold references has many practical consequences in Python programming:
- Simple assignment does not create copies.
- Augmented assignment with += or *= creates new objects if the lefthand variable is bound to an immutable object, but may many modify a mutable object in place.
- Assigning a new value to an existing variable does not change the object previously bound to it. This is called a rebinding: the variable is now bound to a different object. If that variable was the last reference to the previous object, that object will be garbage collected.
- Function parameters are passed as aliases, which means the function may change any mutable object received as an argument. There is no way to prevent this, except making local copies or using immutable objects(e.g., passing a tuple instead of a list)
- Using mutable objects as default values for function parameters is dangerous because if the parameters are changed in place, then the default is changed, affecting every future call that relies on the default.
In CPython, objects are discarded as soon as the number of references to them reaches zero. They may also be discarded if they form groups with cyclic references but no outside references.
In some situations, it may be useful to hold a reference to an object that will not-- by itself -- keep an object alive. One example is a class that wants to keep track of all its current instances. This cab be done with weak references, a low-level mechanism underlying the more useful collections WeakValueDictionary, WeakKeyDictionary, WeakSet, and the finalize function from the weakref module.
13. Parameter Passing: Call by Sharing
A popular way of explaining how parameter passing works in Python is the phrase: "Parameters are passed by value, but the values are references." This is not wrong, but causes confusion because th most common parameter passing modes in older languages are call by value(the function gets a copy of the argument) and call by reference(the function gets a pointer to the argument). In Python, the function gets a copy of the arguments, but the arguments are always references. So the value of the referenced objects may be changed, if they are mutable, but their identity cannot. Also, because the function gets a copy of the reference in an argument, rebinding it has no effect outside of the function. I adopted the term "call by sharing".