Python类的内建方法-即所谓的魔术方法
Posted Augustone-最后一个质数
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Python类的内建方法-即所谓的魔术方法相关的知识,希望对你有一定的参考价值。
在Python中以两个下划线开头的方法,__init__、__str__、__doc__、__new__等,被称为"魔术方法"(Magic methods)。魔术方法在类或对象的某些事件出发后会自动执行,如果希望根据自己的程序定制自己特殊功能的类,那么就需要对这些方法进行重写。Python 将所有以 __包起来(即前后皆有__)的类方法保留为魔术方法。
注:如果只前面有双下划线__,则为私有方法或私有变量
一,
A Guide to Python‘s Magic Methods
Rafe Kettler
Copyright © 2012 Rafe Kettler
Version 1.17
A PDF version of this guide can be obtained from my site or Github. The magic methods guide has a git repository at http://www.github.com/RafeKettler/magicmethods. Any issues can be reported there, along with comments, (or even contributions!).
- Introduction
- Construction and Initialization
- Making Operators Work on Custom Classes
- Representing your Classes
- Controlling Attribute Access
- Making Custom Sequences
- Reflection
- Abstract Base Classes
- Callable Objects
- Context Managers
- Building Descriptor Objects
- Copying
- Pickling your Objects
- Conclusion
- Appendix 1: How to Call Magic Methods
- Appendix 2: Changes in Python 3
Introduction
This guide is the culmination of a few months‘ worth of blog posts. The subject is magic methods.
What are magic methods? They‘re everything in object-oriented Python. They‘re special methods that you can define to add "magic" to your classes. They‘re always surrounded by double underscores (e.g. __init__
or __lt__
). They‘re also not as well documented as they need to be. All of the magic methods for Python appear in the same section in the Python docs, but they‘re scattered about and only loosely organized. There‘s hardly an example to be found in that section (and that may very well be by design, since they‘re all detailed in the language reference, along with boring syntax descriptions, etc.).
So, to fix what I perceived as a flaw in Python‘s documentation, I set out to provide some more plain-English, example-driven documentation for Python‘s magic methods. I started out with weekly blog posts, and now that I‘ve finished with those, I‘ve put together this guide.
I hope you enjoy it. Use it as a tutorial, a refresher, or a reference; it‘s just intended to be a user-friendly guide to Python‘s magic methods.
Construction and Initialization
Everyone knows the most basic magic method, __init__
. It‘s the way that we can define the initialization behavior of an object. However, when I call x = SomeClass()
, __init__
is not the first thing to get called. Actually, it‘s a method called __new__
, which actually creates the instance, then passes any arguments at creation on to the initializer. At the other end of the object‘s lifespan, there‘s __del__
. Let‘s take a closer look at these 3 magic methods:
__new__(cls, [...)
__new__
is the first method to get called in an object‘s instantiation. It takes the class, then any other arguments that it will pass along to__init__
.__new__
is used fairly rarely, but it does have its purposes, particularly when subclassing an immutable type like a tuple or a string. I don‘t want to go in to too much detail on__new__
because it‘s not too useful, but it is covered in great detail in the Python docs.__init__(self, [...)
- The initializer for the class. It gets passed whatever the primary constructor was called with (so, for example, if we called
x = SomeClass(10, ‘foo‘)
,__init__
would get passed10
and‘foo‘
as arguments.__init__
is almost universally used in Python class definitions. __del__(self)
- If
__new__
and__init__
formed the constructor of the object,__del__
is the destructor. It doesn‘t implement behavior for the statementdel x
(so that code would not translate tox.__del__()
). Rather, it defines behavior for when an object is garbage collected. It can be quite useful for objects that might require extra cleanup upon deletion, like sockets or file objects. Be careful, however, as there is no guarantee that__del__
will be executed if the object is still alive when the interpreter exits, so__del__
can‘t serve as a replacement for good coding practices (like always closing a connection when you‘re done with it. In fact,__del__
should almost never be used because of the precarious circumstances under which it is called; use it with caution!
Putting it all together, here‘s an example of __init__
and __del__
in action:
from os.path import join
class FileObject:
‘‘‘Wrapper for file objects to make sure the file gets closed on deletion.‘‘‘
def __init__(self, filepath=‘~‘, filename=‘sample.txt‘):
# open a file filename in filepath in read and write mode
self.file = open(join(filepath, filename), ‘r+‘)
def __del__(self):
self.file.close()
del self.file
Making Operators Work on Custom Classes
One of the biggest advantages of using Python‘s magic methods is that they provide a simple way to make objects behave like built-in types. That means you can avoid ugly, counter-intuitive, and nonstandard ways of performing basic operators. In some languages, it‘s common to do something like this:
if instance.equals(other_instance):
# do something
You could certainly do this in Python, too, but this adds confusion and is unnecessarily verbose. Different libraries might use different names for the same operations, making the client do way more work than necessary. With the power of magic methods, however, we can define one method (__eq__
, in this case), and say what we mean instead:
if instance == other_instance:
#do something
That‘s part of the power of magic methods. The vast majority of them allow us to define meaning for operators so that we can use them on our own classes just like they were built in types.
Comparison magic methods
Python has a whole slew of magic methods designed to implement intuitive comparisons between objects using operators, not awkward method calls. They also provide a way to override the default Python behavior for comparisons of objects (by reference). Here‘s the list of those methods and what they do:
__cmp__(self, other)
__cmp__
is the most basic of the comparison magic methods. It actually implements behavior for all of the comparison operators (<, ==, !=, etc.), but it might not do it the way you want (for example, if whether one instance was equal to another were determined by one criterion and and whether an instance is greater than another were determined by something else).__cmp__
should return a negative integer ifself < other
, zero ifself == other
, and positive ifself > other
. It‘s usually best to define each comparison you need rather than define them all at once, but__cmp__
can be a good way to save repetition and improve clarity when you need all comparisons implemented with similar criteria.__eq__(self, other)
- Defines behavior for the equality operator,
==
. __ne__(self, other)
- Defines behavior for the inequality operator,
!=
. __lt__(self, other)
- Defines behavior for the less-than operator,
<
. __gt__(self, other)
- Defines behavior for the greater-than operator,
>
. __le__(self, other)
- Defines behavior for the less-than-or-equal-to operator,
<=
. __ge__(self, other)
- Defines behavior for the greater-than-or-equal-to operator,
>=
.
For an example, consider a class to model a word. We might want to compare words lexicographically (by the alphabet), which is the default comparison behavior for strings, but we also might want to do it based on some other criterion, like length or number of syllables. In this example, we‘ll compare by length. Here‘s an implementation:
class Word(str):
‘‘‘Class for words, defining comparison based on word length.‘‘‘
def __new__(cls, word):
# Note that we have to use __new__. This is because str is an immutable
# type, so we have to initialize it early (at creation)
if ‘ ‘ in word:
print "Value contains spaces. Truncating to first space."
word = word[:word.index(‘ ‘)] # Word is now all chars before first space
return str.__new__(cls, word)
def __gt__(self, other):
return len(self) > len(other)
def __lt__(self, other):
return len(self) < len(other)
def __ge__(self, other):
return len(self) >= len(other)
def __le__(self, other):
return len(self) <= len(other)
Now, we can create two Word
s (by using Word(‘foo‘)
and Word(‘bar‘)
) and compare them based on length. Note, however, that we didn‘t define __eq__
and __ne__
. This is because this would lead to some weird behavior (notably that Word(‘foo‘) == Word(‘bar‘)
would evaluate to true). It wouldn‘t make sense to test for equality based on length, so we fall back on str
‘s implementation of equality.
Now would be a good time to note that you don‘t have to define every comparison magic method to get rich comparisons. The standard library has kindly provided us with a class decorator in the module functools
that will define all rich comparison methods if you only define __eq__
and one other (e.g. __gt__
, __lt__
, etc.) This feature is only available in Python 2.7, but when you get a chance it saves a great deal of time and effort. You can use it by placing @total_ordering
above your class definition.
Numeric magic methods
Just like you can create ways for instances of your class to be compared with comparison operators, you can define behavior for numeric operators. Buckle your seat belts, folks...there‘s a lot of these. For organization‘s sake, I‘ve split the numeric magic methods into 5 categories: unary operators, normal arithmetic operators, reflected arithmetic operators (more on this later), augmented assignment, and type conversions.
Unary operators and functions
Unary operators and functions only have one operand, e.g. negation, absolute value, etc.
__pos__(self)
- Implements behavior for unary positive (e.g.
+some_object
) __neg__(self)
- Implements behavior for negation (e.g.
-some_object
) __abs__(self)
- Implements behavior for the built in
abs()
function. __invert__(self)
- Implements behavior for inversion using the
~
operator. For an explanation on what this does, see the Wikipedia article on bitwise operations. __round__(self, n)
- Implements behavior for the built in
round()
function.n
is the number of decimal places to round to. __floor__(self)
- Implements behavior for
math.floor()
, i.e., rounding down to the nearest integer. __ceil__(self)
- Implements behavior for
math.ceil()
, i.e., rounding up to the nearest integer. __trunc__(self)
- Implements behavior for
math.trunc()
, i.e., truncating to an integral.
Normal arithmetic operators
Now, we cover the typical binary operators (and a function or two): +, -, * and the like. These are, for the most part, pretty self-explanatory.
__add__(self, other)
- Implements addition.
__sub__(self, other)
- Implements subtraction.
__mul__(self, other)
- Implements multiplication.
__floordiv__(self, other)
- Implements integer division using the
//
operator. __div__(self, other)
- Implements division using the
/
operator. __truediv__(self, other)
- Implements true division. Note that this only works when
from __future__ import division
is in effect. __mod__(self, other)
- Implements modulo using the
%
operator. __divmod__(self, other)
- Implements behavior for long division using the
divmod()
built in function. __pow__
- Implements behavior for exponents using the
**
operator. __lshift__(self, other)
- Implements left bitwise shift using the
<<
operator. __rshift__(self, other)
- Implements right bitwise shift using the
>>
operator. __and__(self, other)
- Implements bitwise and using the
&
operator. __or__(self, other)
- Implements bitwise or using the
|
operator. __xor__(self, other)
- Implements bitwise xor using the
^
operator.
Reflected arithmetic operators
You know how I said I would get to reflected arithmetic in a bit? Some of you might think it‘s some big, scary, foreign concept. It‘s actually quite simple. Here‘s an example:
some_object + other
That was "normal" addition. The reflected equivalent is the same thing, except with the operands switched around:
other + some_object
So, all of these magic methods do the same thing as their normal equivalents, except the perform the operation with other as the first operand and self as the second, rather than the other way around. In most cases, the result of a reflected operation is the same as its normal equivalent, so you may just end up defining __radd__
as calling __add__
and so on. Note that the object on the left hand side of the operator (other
in the example) must not define (or return NotImplemented
) for its definition of the non-reflected version of an operation. For instance, in the example, some_object.__radd__
will only be called if other
does not define __add__
.
__radd__(self, other)
- Implements reflected addition.
__rsub__(self, other)
- Implements reflected subtraction.
__rmul__(self, other)
- Implements reflected multiplication.
__rfloordiv__(self, other)
- Implements reflected integer division using the
//
operator. __rdiv__(self, other)
- Implements reflected division using the
/
operator. __rtruediv__(self, other)
- Implements reflected true division. Note that this only works when
from __future__ import division
is in effect. __rmod__(self, other)
- Implements reflected modulo using the
%
operator. __rdivmod__(self, other)
- Implements behavior for long division using the
divmod()
built in function, whendivmod(other, self)
is called. __rpow__
- Implements behavior for reflected exponents using the
**
operator. __rlshift__(self, other)
- Implements reflected left bitwise shift using the
<<
operator. __rrshift__(self, other)
- Implements reflected right bitwise shift using the
>>
operator. __rand__(self, other)
- Implements reflected bitwise and using the
&
operator. __ror__(self, other)
- Implements reflected bitwise or using the
|
operator. __rxor__(self, other)
- Implements reflected bitwise xor using the
^
operator.
Augmented assignment
Python also has a wide variety of magic methods to allow custom behavior to be defined for augmented assignment. You‘re probably already familiar with augmented assignment, it combines "normal" operators with assignment. If you still don‘t know what I‘m talking about, here‘s an example:
x = 5
x += 1 # in other words x = x + 1
Each of these methods should return the value that the variable on the left hand side should be assigned to (for instance, for a += b
, __iadd__
might return a + b
, which would be assigned to a
). Here‘s the list:
__iadd__(self, other)
- Implements addition with assignment.
__isub__(self, other)
- Implements subtraction with assignment.
__imul__(self, other)
- Implements multiplication with assignment.
__ifloordiv__(self, other)
- Implements integer division with assignment using the
//=
operator. __idiv__(self, other)
- Implements division with assignment using the
/=
operator. __itruediv__(self, other)
- Implements true division with assignment. Note that this only works when
from __future__ import division
is in effect. __imod__(self, other)
- Implements modulo with assignment using the
%=
operator. __ipow__
- Implements behavior for exponents with assignment using the
**=
operator. __ilshift__(self, other)
- Implements left bitwise shift with assignment using the
<<=
operator. __irshift__(self, other)
- Implements right bitwise shift with assignment using the
>>=
operator. __iand__(self, other)
- Implements bitwise and with assignment using the
&=
operator. __ior__(self, other)
- Implements bitwise or with assignment using the
|=
operator. __ixor__(self, other)
- Implements bitwise xor with assignment using the
^=
operator.
Type conversion magic methods
Python also has an array of magic methods designed to implement behavior for built in type conversion functions like float()
. Here they are:
__int__(self)
- Implements type conversion to int.
__long__(self)
- Implements type conversion to long.
__float__(self)
- Implements type conversion to float.
__complex__(self)
- Implements type conversion to complex.
__oct__(self)
- Implements type conversion to octal.
__hex__(self)
- Implements type conversion to hexadecimal.
__index__(self)
- Implements type conversion to an int when the object is used in a slice expression. If you define a custom numeric type that might be used in slicing, you should define
__index__
. __trunc__(self)
- Called when
math.trunc(self)
is called.__trunc__
should return the value of `self truncated to an integral type (usually a long). __coerce__(self, other)
- Method to implement mixed mode arithmetic.
__coerce__
should returnNone
if type conversion is impossible. Otherwise, it should return a pair (2-tuple) ofself
andother
, manipulated to have the same type.
Representing your Classes
It‘s often useful to have a string representation of a class. In Python, there are a few methods that you can implement in your class definition to customize how built in functions that return representations of your class behave.
__str__(self)
- Defines behavior for when
str()
is called on an instance of your class. __repr__(self)
- Defines behavior for when
repr()
is called on an instance of your class. The major difference betweenstr()
andrepr()
is intended audience.repr()
is intended to produce output that is mostly machine-readable (in many cases, it could be valid Python code even), whereasstr()
is intended to be human-readable. __unicode__(self)
- Defines behavior for when
unicode()
is called on an instance of your class.unicode()
is likestr()
, but it returns a unicode string. Be wary: if a client callsstr()
on an instance of your class and you‘ve only defined__unicode__()
, it won‘t work. You should always try to define__str__()
as well in case someone doesn‘t have the luxury of using unicode. __format__(self, formatstr)
- Defines behavior for when an instance of your class is used in new-style string formatting. For instance,
"Hello, {0:abc}!".format(a)
would lead to the calla.__format__("abc")
. This can be useful for defining your own numerical or string types that you might like to give special formatting options. __hash__(self)
- Defines behavior for when
hash()
is called on an instance of your class. It has to return an integer, and its result is used for quick key comparison in dictionaries. Note that this usually entails implementing__eq__
as well. Live by the following rule:a == b
implieshash(a) == hash(b)
. __nonzero__(self)
- Defines behavior for when
bool()
is called on an instance of your class. Should returnTrue
orFalse
, depending on whether you would want to consider the instance to beTrue
orFalse
. __dir__(self)
- Defines behavior for when
dir()
is called on an instance of your class. This method should return a list of attributes for the user. Typically, implementing__dir__
is unnecessary, but it can be vitally important for interactive use of your classes if you redefine__getattr__
or__getattribute__
(which you will see in the next section) or are otherwise dynamically generating attributes. __sizeof__(self)
- Defines behavior for when
sys.getsizeof()
is called on an instance of your class. This should return the size of your object, in bytes. This is generally more useful for Python classes implemented in C extensions, but it helps to be aware of it.
We‘re pretty much done with the boring (and example-free) part of the magic methods guide. Now that we‘ve covered some of the more basic magic methods, it‘s time to move to more advanced material.
Controlling Attribute Access
Many people coming to Python from other languages complain that it lacks true encapsulation for classes; that is, there‘s no way to define private attributes with public getter and setters. This couldn‘t be farther than the truth: it just happens that Python accomplishes a great deal of encapsulation through "magic", instead of explicit modifiers for methods or fields. Take a look:
__getattr__(self, name)
- You can define behavior for when a user attempts to access an attribute that doesn‘t exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes (you can still choose to compute and return that attribute, if you wish), or deftly handing an
AttributeError
. It only gets called when a nonexistent attribute is accessed, however, so it isn‘t a true encapsulation solution. __setattr__(self, name, value)
- Unlike
__getattr__
,__setattr__
is an encapsulation solution. It allows you to define behavior for assignment to an attribute regardless of whether or not that attribute exists, meaning you can define custom rules for any changes in the values of attributes. However, you have to be careful with how you use__setattr__
, as the example at the end of the list will show. __delattr__(self, name)
- This is the exact same as
__setattr__
, but for deleting attributes instead of setting them. The same precautions need to be taken as with__setattr__
as well in order to prevent infinite recursion (callingdel self.name
in the implementation of__delattr__
would cause infinite recursion). __getattribute__(self, name)
- After all this,
__getattribute__
fits in pretty well with its companions__setattr__
and__delattr__
. However, I don‘t recommend you use it.__getattribute__
can only be used with new-style classes (all classes are new-style in the newest versions of Python, and in older versions you can make a class new-style by subclassingobject
. It allows you to define rules for whenever an attribute‘s value is accessed. It suffers from some similar infinite recursion problems as its partners-in-crime (this time you call the base class‘s__getattribute__
method to prevent this). It also mainly obviates the need for__getattr__
, which, when__getattribute__
is implemented, only gets called if it is called explicitly or anAttributeError
is raised. This method can be used (after all, it‘s your choice), but I don‘t recommend it because it has a small use case (it‘s far more rare that we need special behavior to retrieve a value than to assign to it) and because it can be really difficult to implement bug-free.
You can easily cause a problem in your definitions of any of the methods controlling attribute access. Consider this example:
def __setattr__(self, name, value):
self.name = value
# since every time an attribute is assigned, __setattr__() is called, this
# is recursion.
# so this really means self.__setattr__(‘name‘, value). Since the method
# keeps calling itself, the recursion goes on forever causing a crash
def __setattr__(self, name, value):
self.__dict__[name] = value # assigning to the dict of names in the class
# define custom behavior here
Again, Python‘s magic methods are incredibly powerful, and with great power comes great responsibility. It‘s important to know the proper way to use magic methods so you don‘t break any code.
So, what have we learned about custom attribute access in Python? It‘s not to be used lightly. In fact, it tends to be excessively powerful and counter-intuitive. But the reason why it exists is to scratch a certain itch: Python doesn‘t seek to make bad things impossible, but just to make them difficult. Freedom is paramount, so you can really do whatever you want. Here‘s an example of some of the special attribute access methods in action (note that we use super
because not all classes have an attribute __dict__
):
class AccessCounter(object):
‘‘‘A class that contains a value and implements an access counter.
The counter increments each time the value is changed.‘‘‘
def __init__(self, val):
super(AccessCounter, self).__setattr__(‘counter‘, 0)
super(AccessCounter, self).__setattr__(‘value‘, val)
def __setattr__(self, name, value):
if name == ‘value‘:
super(AccessCounter, self).__setattr__(‘counter‘, self.counter + 1)
# Make this unconditional.
# If you want to prevent other attributes to be set, raise AttributeError(name)
super(AccessCounter, self).__setattr__(name, value)
def __delattr__(self, name):
if name == ‘value‘:
super(AccessCounter, self).__setattr__(‘counter‘, self.counter + 1)
super(AccessCounter, self).__delattr__(name)
Making Custom Sequences
There‘s a number of ways to get your Python classes to act like built in sequences (dict
, tuple
, list
, str
, etc.). These are by far my favorite magic methods in Python because of the absurd degree of control they give you and the way that they magically make a whole array of global functions work beautifully on instances of your class. But before we get down to the good stuff, a quick word on requirements.
Requirements
Now that we‘re talking about creating your own sequences in Python, it‘s time to talk about protocols. Protocols are somewhat similar to interfaces in other languages in that they give you a set of methods you must define. However, in Python protocols are totally informal and require no explicit declarations to implement. Rather, they‘re more like guidelines.
Why are we talking about protocols now? Because implementing custom container types in Python involves using some of these protocols. First, there‘s the protocol for defining immutable containers: to make an immutable container, you need only define __len__
and __getitem__
(more on these later). The mutable container protocol requires everything that immutable containers require plus __setitem__
and __delitem__
. Lastly, if you want your object to be iterable, you‘ll have to define __iter__
, which returns an iterator. That iterator must conform to an iterator protocol, which requires iterators to have methods called __iter__
(returning itself) and next
.
The magic behind containers
Without any more wait, here are the magic methods that containers use:
__len__(self)
- Returns the length of the container. Part of the protocol for both immutable and mutable containers.
__getitem__(self, key)
- Defines behavior for when an item is accessed, using the notation
self[key]
. This is also part of both the mutable and immutable container protocols. It should also raise appropriate exceptions:TypeError
if the type of the key is wrong andKeyError
if there is no corresponding value for the key. __setitem__(self, key, value)
- Defines behavior for when an item is assigned to, using the notation
self[nkey] = value
. This is part of the mutable container protocol. Again, you should raiseKeyError
andTypeError
where appropriate. __delitem__(self, key)
- Defines behavior for when an item is deleted (e.g.
del self[key]
). This is only part of the mutable container protocol. You must raise the appropriate exceptions when an invalid key is used. __iter__(self)
- Should return an iterator for the container. Iterators are returned in a number of contexts, most notably by the
iter()
built in function and when a container is looped over using the formfor x in container:
. Iterators are their own objects, and they also must define an__iter__
method that returnsself
. __reversed__(self)
- Called to implement behavior for the
reversed()
built in function. Should return a reversed version of the sequence. Implement this only if the sequence class is ordered, like list or tuple. __contains__(self, item)
__contains__
defines behavior for membership tests usingin
andnot in
. Why isn‘t this part of a sequence protocol, you ask? Because when__contains__
isn‘t defined, Python just iterates over the sequence and returnsTrue
if it comes across the item it‘s looking for.__missing__(self, key)
__missing__
is used in subclasses ofdict
. It defines behavior for whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had a dictionaryd
and saidd["george"]
when"george"
is not a key in the dict,d.__missing__("george")
would be called).
An example
For our example, let‘s look at a list that implements some functional constructs that you might be used to from other languages (Haskell, for example).
class FunctionalList:
‘‘‘A class wrapping a list with some extra functional magic, like head,
tail, init, last, drop, and take.‘‘‘
def __init__(self, values=None):
if values is None:
self.values = []
else:
self.values = values
def __len__(self):
return len(self.values)
def __getitem__(self, key):
# if key is of invalid type or value, the list values will raise the error
return self.values[key]
def __setitem__(self, key, value):
self.values[key] = value
def __delitem__(self, key):
del self.values[key]
def __iter__(self):
return iter(self.values)
def __reversed__(self):
return reversed(self.values)
def append(self, value):
self.values.append(value)
def head(self):
# get the first element
return self.values[0]
def tail(self):
# get all elements after the first
return self.values[1:]
def init(self):
# get elements up to the last
return self.values[:-1]
def last(self):
# get last element
return self.values[-1]
def drop(self, n):
# get all elements except first n
return self.values[n:]
def take(self, n):
# get first n elements
return self.values[:n]
There you have it, a (marginally) useful example of how to implement your own sequence. Of course, there are more useful applications of custom sequences, but quite a few of them are already implemented in the standard library (batteries included, right?), like Counter
, OrderedDict
, and NamedTuple
.
Reflection
You can also control how reflection using the built in functions isinstance()
and issubclass()
behaves by defining magic methods. The magic methods are:
__instancecheck__(self, instance)
- Checks if an instance is an instance of the class you defined (e.g.
isinstance(instance, class)
. __subclasscheck__(self, subclass)
- Checks if a class subclasses the class you defined (e.g.
issubclass(subclass, class)
).
The use case for these magic methods might seem small, and that may very well be true. I won‘t spend too much more time on reflection magic methods because they aren‘t very important, but they reflect something important about object-oriented programming in Python and Python in general: there is almost always an easy way to do something, even if it‘s rarely necessary. These magic methods might not seem useful, but if you ever need them you‘ll be glad that they‘re there (and that you read this guide!).
Callable Objects
As you may already know, in Python, functions are first-class objects. This means that they can be passed to functions and methods just as if they were objects of any other kind. This is an incredibly powerful feature.
A special magic method in Python allows instances of your classes to behave as if they were functions, so that you can "call" them, pass them to functions that take functions as arguments, and so on. This is another powerful convenience feature that makes programming in Python that much sweeter.
__call__(self, [args...])
- Allows an instance of a class to be called as a function. Essentially, this means that
x()
is the same asx.__call__()
. Note that__call__
takes a variable number of arguments; this means that you define__call__
as you would any other function, taking however many arguments you‘d like it to.
__call__
can be particularly useful in classes with instances that need to often change state. "Calling" the instance can be an intuitive and elegant way to change the object‘s state. An example might be a class representing an entity‘s position on a plane:
class Entity:
‘‘‘Class to represent an entity. Callable to update the entity‘s position.‘‘‘
def __init__(self, size, x, y):
self.x, self.y = x, y
self.size = size
def __call__(self, x, y):
‘‘‘Change the position of the entity.‘‘‘
self.x, self.y = x, y
# snip...
Context Managers
In Python 2.5, a new keyword was introduced in Python along with a new method for code reuse: the with
statement. The concept of context managers was hardly new in Python (it was implemented before as a part of the library), but not until PEP 343 was accepted did it achieve status as a first-class language construct. You may have seen with
statements before:
with open(‘foo.txt‘) as bar:
# perform some action with bar
Context managers allow setup and cleanup actions to be taken for objects when their creation is wrapped with a with
statement. The behavior of the context manager is determined by two magic methods:
__enter__(self)
- Defines what the context manager should do at the beginning of the block created by the
with
statement. Note that the return value of__enter__
is bound to the target of thewith
statement, or the name after theas
. __exit__(self, exception_type, exception_value, traceback)
- Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully,
exception_type
,exception_value
, andtraceback
will beNone
. Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure__exit__
returnsTrue
after all is said and done. If you don‘t want the exception to be handled by the context manager, just let it happen.
__enter__
and __exit__
can be useful for specific classes that have well-defined and common behavior for setup and cleanup. You can also use these methods to create generic context managers that wrap other objects. Here‘s an example:
class Closer:
‘‘‘A context manager to automatically close an object with a close method
in a with statement.‘‘‘
def __init__(self, obj):
self.obj = obj
def __enter__(self):
return self.obj # bound to target
def __exit__(self, exception_type, exception_val, trace):
try:
self.obj.close()
except AttributeError: # obj isn‘t closable
print ‘Not closable.‘
return True # exception handled successfully
Here‘s an example of Closer
in action, using an FTP connection to demonstrate it (a closable socket):
>>> from magicmethods import Closer
>>> from ftplib import FTP
>>> with Closer(FTP(‘ftp.somesite.com‘)) as conn:
... conn.dir()
...
# output omitted for brevity
>>> conn.dir()
# long AttributeError message, can‘t use a connection that‘s closed
>>> with Closer(int(5)) as i:
... i += 1
...
Not closable.
>>> i
6
See how our wrapper gracefully handled both proper and improper uses? That‘s the power of context managers and magic methods. Note that the Python standard library includes a module contextlib that contains a context manager, contextlib.closing()
, that does approximately the same thing (without any handling of the case where an object does not have a close()
method).
Abstract Base Classes
See http://docs.python.org/2/library/abc.html.
Building Descriptor Objects
Descriptors are classes which, when accessed through either getting, setting, or deleting, can also alter other objects. Descriptors aren‘t meant to stand alone; rather, they‘re meant to be held by an owner class. Descriptors can be useful when building object-oriented databases or classes that have attributes whose values are dependent on each other. Descriptors are particularly useful when representing attributes in several different units of measurement or representing computed attributes (like distance from the origin in a class to represent a point on a grid).
To be a descriptor, a class must have at least one of __get__
, __set__
, and __delete__
implemented. Let‘s take a look at those magic methods:
__get__(self, instance, owner)
- Define behavior for when the descriptor‘s value is retrieved.
instance
is the instance of the owner object.owner
is the owner class itself. __set__(self, instance, value)
- Define behavior for when the descriptor‘s value is changed.
instance
is the instance of the owner class andvalue
is the value to set the descriptor to. __delete__(self, instance)
- Define behavior for when the descriptor‘s value is deleted.
instance
is the instance of the owner object.
Now, an example of a useful application of descriptors: unit conversions.
class Meter(object):
‘‘‘Descriptor for a meter.‘‘‘
def __init__(self, value=0.0):
self.value = float(value)
def __get__(self, instance, owner):
return self.value
def __set__(self, instance, value):
self.value = float(value)
class Foot(object):
‘‘‘Descriptor for a foot.‘‘‘
def __get__(self, instance, owner):
return instance.meter * 3.2808
def __set__(self, instance, value):
instance.meter = float(value) / 3.2808
class Distance(object):
‘‘‘Class to represent distance holding two descriptors for feet and
meters.‘‘‘
meter = Meter()
foot = Foot()
Copying
Sometimes, particularly when dealing with mutable objects, you want to be able to copy an object and make changes without affecting what you copied from. This is where Python‘s copy
comes into play. However (fortunately), Python modules are not sentient, so we don‘t have to worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy things.
__copy__(self)
- Defines behavior for
copy.copy()
for instances of your class.copy.copy()
returns a shallow copy of your object -- this means that, while the instance itself is a new instance, all of its data is referenced -- i.e., the object itself is copied, but its data is still referenced (and hence changes to data in a shallow copy may cause changes in the original). __deepcopy__(self, memodict={})
- Defines behavior for
copy.deepcopy()
for instances of your class.copy.deepcopy()
returns a deep copy of your object -- the object and its data are both copied.memodict
is a cache of previously copied objects -- this optimizes copying and prevents infinite recursion when copying recursive data structures. When you want to deep copy an individual attribute, callcopy.deepcopy()
on that attribute withmemodict
as the first argument.
What are some use cases for these magic methods? As always, in any case where you need more fine-grained control than what the default behavior gives you. For instance, if you are attempting to copy an object that stores a cache as a dictionary (which might be large), it might not make sense to copy the cache as well -- if the cache can be shared in memory between instances, then it should be.
Pickling Your Objects
If you spend time with other Pythonistas, chances are you‘ve at least heard of pickling. Pickling is a serialization process for Python data structures, and can be incredibly useful when you need to store an object and retrieve it later (usually for caching). It‘s also a major source of worries and confusion.
Pickling is so important that it doesn‘t just have its own module (pickle
), but its own protocol and the magic methods to go with it. But first, a brief word on how to pickle existing types(feel free to skip it if you already know).
Pickling: A Quick Soak in the Brine
Let‘s dive into pickling. Say you have a dictionary that you want to store and retrieve later. You couldwrite it‘s contents to a file, carefully making sure that you write correct syntax, then retrieve it using either exec()
or processing the file input. But this is precarious at best: if you store important data in plain text, it could be corrupted or changed in any number of ways to make your program crash or worse run malicious code on your computer. Instead, we‘re going to pickle it:
import pickle
data = {‘foo‘: [1, 2, 3],
‘bar‘: (‘Hello‘, ‘world!‘),
‘baz‘: True}
jar = open(‘data.pkl‘, ‘wb‘)
pickle.dump(data, jar) # write the pickled data to the file jar
jar.close()
Now, a few hours later, we want it back. All we have to do is unpickle it:
import pickle
pkl_file = open(‘data.pkl‘, ‘rb‘) # connect to the pickled data
data = pickle.load(pkl_file) # load it into a variable
print data
pkl_file.close()
What happens? Exactly what you expect. It‘s just like we had data
all along.
Now, for a word of caution: pickling is not perfect. Pickle files are easily corrupted on accident and on purpose. Pickling may be more secure than using flat text files, but it still can be used to run malicious code. It‘s also incompatible across different versions of Python, so don‘t expect to distribute pickled objects and expect people to be able to open them. However, it can also be a powerful tool for caching and other common serialization tasks.
Pickling your own Objects
Pickling isn‘t just for built-in types. It‘s for any class that follows the pickle protocol. The pickle protocol has four optional methods for Python objects to customize how they act (it‘s a bit different for C extensions, but that‘s not in our scope):
__getinitargs__(self)
- If you‘d like for
__init__
to be called when your class is unpickled, you can define__getinitargs__
, which should return a tuple of the arguments that you‘d like to be passed to__init__
. Note that this method will only work for old-style classes. __getnewargs__(self)
- For new-style classes, you can influence what arguments get passed to
__new__
upon unpickling. This method should also return a tuple of arguments that will then be passed to__new__
. __getstate__(self)
- Instead of the object‘s
__dict__
attribute being stored, you can return a custom state to be stored when the object is pickled. That state will be used by__setstate__
when the object is unpickled. __setstate__(self, state)
- When the object is unpickled, if
__setstate__
is defined the object‘s state will be passed to it instead of directly applied to the object‘s__dict__
. This goes hand in hand with__getstate__
: when both are defined, you can represent the object‘s pickled state however you want with whatever you want. __reduce__(self)
- When defining extension types (i.e., types implemented using Python‘s C API), you have to tell Python how to pickle them if you want them to pickle them.
__reduce__()
is called when an object defining it is pickled. It can either return a string representing a global name that Python will look up and pickle, or a tuple. The tuple contains between 2 and 5 elements: a callable object that is called to recreate the object, a tuple of arguments for that callable object, state to be passed to__setstate__
(optional), an iterator yielding list items to be pickled (optional), and an iterator yielding dictionary items to be pickled (optional). __reduce_ex__(self)
__reduce_ex__
exists for compatibility. If it is defined,__reduce_ex__
will be called over__reduce__
on pickling.__reduce__
can be defined as well for older versions of the pickling API that did not support__reduce_ex__
.
An Example
Our example is a Slate
, which remembers what its values have been and when those values were written to it. However, this particular slate goes blank each time it is pickled: the current value will not be saved.
import time
class Slate:
‘‘‘Class to store a string and a changelog, and forget its value when
pickled.‘‘‘
def __init__(self, value):
self.value = value
self.last_change = time.asctime()
self.history = {}
def change(self, new_value):
# Change the value. Commit last value to history
self.history[self.last_change] = self.value
self.value = new_value
self.last_change = time.asctime()
def print_changes(self):
print ‘Changelog for Slate object:‘
for k, v in self.history.items():
print ‘%s\t %s‘ % (k, v)
def __getstate__(self):
# Deliberately do not return self.value or self.last_change.
# We want to have a "blank slate" when we unpickle.
return self.history
def __setstate__(self, state):
# Make self.history = state and last_change and value undefined
self.history = state
self.value, self.last_change = None, None
Conclusion
The goal of this guide is to bring something to anyone that reads it, regardless of their experience with Python or object-oriented programming. If you‘re just getting started with Python, you‘ve gained valuable knowledge of the basics of writing feature-rich, elegant, and easy-to-use classes. If you‘re an intermediate Python programmer, you‘ve probably picked up some slick new concepts and strategies and some good ways to reduce the amount of code written by you and clients. If you‘re an expert Pythonista, you‘ve been refreshed on some of the stuff you might have forgotten about and maybe picked up a few new tricks along the way. Whatever your experience level, I hope that this trip through Python‘s special methods has been truly magical. (I couldn‘t resist the final pun!)
Appendix 1: How to Call Magic Methods
Some of the magic methods in Python directly map to built-in functions; in this case, how to invoke them is fairly obvious. However, in other cases, the invocation is far less obvious. This appendix is devoted to exposing non-obvious syntax that leads to magic methods getting called.
Magic Method | When it gets invoked (example) | Explanation |
---|---|---|
__new__(cls [,...]) |
instance = MyClass(arg1, arg2) |
__new__ is called on instance creation |
__init__(self [,...]) |
instance = MyClass(arg1, arg2) |
__init__ is called on instance creation |
__cmp__(self, other) |
self == other , self > other , etc. |
Called for any comparison |
__pos__(self) |
+self |
Unary plus sign |
__neg__(self) |
-self |
Unary minus sign |
__invert__(self) |
~self |
Bitwise inversion |
__index__(self) |
x[self] |
Conversion when object is used as index |
__nonzero__(self) |
bool(self) |
Boolean value of the object |
__getattr__(self, name) |
self.name # name doesn‘t exist |
Accessing nonexistent attribute |
__setattr__(self, name, val) |
self.name = val |
Assigning to an attribute |
__delattr__(self, name) |
del self.name |
Deleting an attribute |
__getattribute__(self, name) |
self.name |
Accessing any attribute |
__getitem__(self, key) |
self[key] |
Accessing an item using an index |
__setitem__(self, key, val) |
self[key] = val |
Assigning to an item using an index |
__delitem__(self, key) |
del self[key] |
Deleting an item using an index |
__iter__(self) |
for x in self |
Iteration |
__contains__(self, value) |
value in self , value not in self |
Membership tests using in |
__call__(self [,...]) |
self(args) |
"Calling" an instance |
__enter__(self) |
with self as x: |
with statement context managers |
__exit__(self, exc, val, trace) |
with self as x: |
with statement context managers |
__getstate__(self) |
pickle.dump(pkl_file, self) |
Pickling |
__setstate__(self) |
data = pickle.load(pkl_file) |
Pickling |
Hopefully, this table should have cleared up any questions you might have had about what syntax invokes which magic method.
Appendix 2: Changes in Python 3
Here, we document a few major places where Python 3 differs from 2.x in terms of its object model:
- Since the distinction between string and unicode has been done away with in Python 3,
__unicode__
is gone and__bytes__
(which behaves similarly to__str__
and__unicode__
in 2.7) exists for a new built-in for constructing byte arrays. - Since division defaults to true division in Python 3,
__div__
is gone in Python 3 __coerce__
is gone due to redundancy with other magic methods and confusing behavior__cmp__
is gone due to redundancy with other magic methods__nonzero__
has been renamed to__bool__
二,
Introduction
The so-called magic methods have nothing to do with wizardry. You have already seen them in previous chapters of our tutorial. They are special methods with fixed names. They are the methods with this clumsy syntax, i.e. the double underscores at the beginning and the end. They are also hard to talk about. How do you pronounce or say a method name like __init__? "Underscore underscore init underscore underscore" sounds horrible and is nearly a tongue twister. "Double underscore init double underscore" is a lot better, but the ideal way is "dunder init dunder"1 That‘s why magic methods methods are sometimes called dunder methods!
So what‘s magic about the __init__ method? The answer is, you don‘t have to invoke it directly.
The invocation is realized behind the scenes. When you create an instance x of a class A with
the statement "x = A()", Python will do the necessary calls to __new__ and __init__.
Nearly at the end of this chapter of our tutorial we will introduce the __call__ method.
It is overlooked by many beginners and even advanced programmers of
Python. It is a functionality which many programming languages do not
have, so programmers are generally not looking for. The __call__ method
enables Python programmers to write classes where the instances behave
like functions. Both functions and the instances of such classes are
called callables.
We have encountered the concept of operator overloading many times in the course of this tutorial.
We had used the plus sign to add numerical values, to concatenate strings or to combine lists:
>>> 4 + 5 9 >>> 3.8 + 9 12.8 >>> "Peter" + " " + "Pan" ‘Peter Pan‘ >>> [3,6,8] + [7,11,13] [3, 6, 8, 7, 11, 13] >>>
It‘s even possible to overload the "+" operator as well as all the other operators for the purposes of your own class. To do this, you need to understand the underlying mechanism. There is a special (or a "magic") method for every operator sign. The magic method for the "+" sign is the __add__ method. For "-" it is "__sub__" and so on. We have a complete listing of all the magic methods a little bit further down.
The mechanism works like this: If we have an expression "x + y" and x is an instance of
class K, then Python will check the class definition of K. If K has a method __add__ it
will be called with x.__add__(y), otherwise we will get an error message.
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: ‘K‘ and ‘K‘
Overview of Magic Methods
Binary Operators
Operator | Method |
---|---|
+ | object.__add__(self, other) |
- | object.__sub__(self, other) |
* | object.__mul__(self, other) |
// | object.__floordiv__(self, other) |
/ | object.__truediv__(self, other) |
% | object.__mod__(self, other) |
** | object.__pow__(self, other[, modulo]) |
<< | object.__lshift__(self, other) |
>> | object.__rshift__(self, other) |
& | object.__and__(self, other) |
^ | object.__xor__(self, other) |
| | object.__or__(self, other) |
Extended Assignments
Operator | Method |
---|---|
+= | object.__iadd__(self, other) |
-= | object.__isub__(self, other) |
*= | object.__imul__(self, other) |
/= | object.__idiv__(self, other) |
//= | object.__ifloordiv__(self, other) |
%= | object.__imod__(self, other) |
**= | object.__ipow__(self, other[, modulo]) |
<<= | object.__ilshift__(self, other) |
>>= | object.__irshift__(self, other) |
&= | object.__iand__(self, other) |
^= | object.__ixor__(self, other) |
|= | object.__ior__(self, other) |
Unary Operators
Operator | Method |
---|---|
- | object.__neg__(self) |
+ | object.__pos__(self) |
abs() | object.__abs__(self) |
~ | object.__invert__(self) |
complex() | object.__complex__(self) |
int() | object.__int__(self) |
long() | object.__long__(self) |
float() | object.__float__(self) |
oct() | object.__oct__(self) |
hex() | object.__hex__(self |
Comparison Operators
Operator | Method |
---|---|
< | object.__lt__(self, other) |
<= | object.__le__(self, other) |
== | object.__eq__(self, other) |
!= | object.__ne__(self, other) |
>= | object.__ge__(self, other) |
> | object.__gt__(self, other) |
Example class: Length
We will demonstrate in the following Length class, how you can overload the "+"
operator for your own class. To do this, we have to overload the __add__ method.
Our class contains the __str__ and __repr__ methods as well. The instances of the
class Length contain length or distance information. The attributes of an instance
are self.value and self.unit.
This class allows us to calculate expressions with mixed units like this one:
2.56 m + 3 yd + 7.8 in + 7.03 cm
The class can be used like this:
>>> from unit_conversions import Length >>> L = Length >>> print(L(2.56,"m") + L(3,"yd") + L(7.8,"in") + L(7.03,"cm")) 5.57162 >>>
The listing of the class:
class Length: __metric = {"mm" : 0.001, "cm" : 0.01, "m" : 1, "km" : 1000, "in" : 0.0254, "ft" : 0.3048, "yd" : 0.9144, "mi" : 1609.344 } def __init__(self, value, unit = "m" ): self.value = value self.unit = unit def Converse2Metres(self): return self.value * Length.__metric[self.unit] def __add__(self, other): l = self.Converse2Metres() + other.Converse2Metres() return Length(l / Length.__metric[self.unit], self.unit ) def __str__(self): return str(self.Converse2Metres()) def __repr__(self): return "Length(" + str(self.value) + ", ‘" + self.unit + "‘)" if __name__ == "__main__": x = Length(4) print(x) y = eval(repr(x)) z = Length(4.5, "yd") + Length(1) print(repr(z)) print(z)
If we start this program, we get the following output:
4 Length(5.593613298337708, ‘yd‘) 5.1148
We use the method__iadd__ to implement the extended assignment:
def __iadd__(self, other): l = self.Converse2Metres() + other.Converse2Metres() self.value = l / Length.__metric[self.unit] return self
Now we are capable to write the following assignments:
x += Length(1) x += Length(4, "yd")
We have added 1 metre in the example above by writing "x += Length(1))". Most certainly,
you will agree with us that it would be more convenient to simply write "x += 1" instead.
We also want to treat expressions like "Length(5,"yd") + 4.8" similarly. So, if somebody
uses a type int or float, our class takes it automatically for "metre" and
converts it into a Length object. It‘s easy to adapt our __add__ and "__iadd__" method
for this task. All we have to do is to check the type of the parameter "other":
def __add__(self, other): if type(other) == int or type(other) == float: l = self.Converse2Metres() + other else: l = self.Converse2Metres() + other.Converse2Metres() return Length(l / Length.__metric[self.unit], self.unit ) def __iadd__(self, other): if type(other) == int or type(other) == float: l = self.Converse2Metres() + other else: l = self.Converse2Metres() + other.Converse2Metres() self.value = l / Length.__metric[self.unit] return self
It‘s a safe bet that if somebody works for a while with adding integers and floats
from the right sight that he or she wants to the same from the left side!
So let‘s try it out:
>>> from unit_conversions import Length >>> x = Length(3, "yd") + 5 >>> x = 5 + Length(3, "yd") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: ‘int‘ and ‘Length‘ >>>
Of course, the left side has to be of type "Length", because otherwise Python
tries to apply the __add__ method from int, which can‘t cope with Length objects as
second arguments!
Python provides a solution for this problem as well. It‘s the __radd__ method.
It works like this: Python tries to evaluate the expression "5 + Length(3, ‘yd‘)".
First it calls int.__add__(5,Length(3, ‘yd‘)), which will raise an exception.
After this it will try to invoke Length.__radd__(Length(3, "yd"), 5).
It‘s easy to recognize that the implementation of __radd__ is analogue to __add__:
def __radd__(self, other): if type(other) == int or type(other) == float: l = self.Converse2Metres() + otherLength.__radd__(Length(3, "yd"), 5) else: l = self.Converse2Metres() + other.Converse2Metres() return Length(l / Length.__metric[self.unit], self.unit )
It‘s advisable to make use of the __add__ method in the __radd__ method:
def __radd__(self, other): return Length.__add__(self,other)
The following diagram illustrates the relationship between __add__ and __radd__:
The __call__ method
The __call__ method can be used to turn the instances of the class into callables. Functions are callable objects. A callable object is an object which can be used and behaves like a function but might not be a function. By using the __call__ method it is possible to define classes in a way that the instances will be callable objects. The __call__ method is called, if the instance is called "like a function", i.e. using brackets. The following example defines a class with which we can create abitrary polynomial functions:
class Polynomial: def __init__(self, *coefficients): self.coefficients = coefficients[::-1] def __call__(self, x): res = 0 for index, coeff in enumerate(self.coefficients): res += coeff * x** index return res # a constant function p1 = Polynomial(42) # a straight Line p2 = Polynomial(0.75, 2) # a third degree Polynomial p3 = Polynomial(1, -0.5, 0.75, 2) for i in range(1, 10): print(i, p1(i), p2(i), p3(i))
These are the results of the previous function:
1 42 2.75 3.25 2 42 3.5 9.5 3 42 4.25 26.75 4 42 5.0 61.0 5 42 5.75 118.25 6 42 6.5 204.5 7 42 7.25 325.75 8 42 8.0 488.0 9 42 8.75 697.25
You will find further interesting examples of the __call__ function in our tutorial in the chapters
Decorators and Memoization with Decorators. You may also consult our chapter on Polynomials
Standard Classes as Base Classes
It‘s possible to use standard classes - like int, float, dict or lists - as base classes
as well.
We extend the list class by adding a push method:
class Plist(list): def __init__(self, l): list.__init__(self, l) def push(self, item): self.append(item) if __name__ == "__main__": x = Plist([3,4]) x.push(47) print(x)
This means that all the previously introduced binary and extended assignment operators exist in the "reversed" version as well:
__radd__ __rsub__ __rmul__ ... and so on
Exercises
-
Write a class with the name Ccy, similar to the previously defined Length class.
Ccy should contain values in various currencies, e.g. "EUR", "GBP" or "USD". An instance should contain the amount and the currency unit.
The class, you are going to design as an excercise, might be best described with the following example session:
>>> from currencies import Ccy >>> v1 = Ccy(23.43, "EUR") >>> v2 = Ccy(19.97, "USD") >>> print(v1 + v2) 32.89 EUR >>> print(v2 + v1) 31.07 USD >>> print(v1 + 3) # an int or a float is considered to be a EUR value 27.43 EUR >>> print(3 + v1) 27.43 EUR >>>
Solutions to our Exercises
- First exercise:
""" The class "Ccy" can be used to define money values in various currencies. A Ccy instance has the string attributes ‘unit‘ (e.g. ‘CHF‘, ‘CAD‘ od ‘EUR‘ and the ‘value‘ as a float. A currency object consists of a value and the corresponding unit. """ class Ccy: currencies = {‘CHF‘: 1.0821202355817312, ‘CAD‘: 1.488609845538393, ‘GBP‘: 0.8916546282920325, ‘JPY‘: 114.38826536281809, ‘EUR‘: 1.0, ‘USD‘: 1.11123458162018} def __init__(self, value, unit="EUR"): self.value = value self.unit = unit def __str__(self): return "{0:5.2f}".format(self.value) + " " + self.unit def changeTo(self, new_unit): """ An Ccy object is transformed from the unit "self.unit" to "new_unit" """ self.value = (self.value / Ccy.currencies[self.unit] * Ccy.currencies[new_unit]) self.unit = new_unit def __add__(self, other): """ Defines the ‘+‘ operator. If other is a CCy object the currency values are added and the result will be the unit of self. If other is an int or a float, other will be treated as a Euro value. """ if type(other) == int or type(other) == float: x = (other * Ccy.currencies[self.unit]) else: x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit]) return Ccy(x + self.value, self.unit) def __iadd__(self, other): """ Similar to __add__ """ if type(other) == int or type(other) == float: x = (other * Ccy.currencies[self.unit]) else: x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit]) self.value += x return self def __radd__(self, other): res = self + other if self.unit != "EUR": res.changeTo("EUR") return res # __sub__, __isub__ and __rsub__ can be defined analogue x = Ccy(10,"USD") y = Ccy(11) z = Ccy(12.34, "JPY") z = 7.8 + x + y + 255 + z print(z) lst = [Ccy(10,"USD"), Ccy(11), Ccy(12.34, "JPY"), Ccy(12.34, "CAD")] z = sum(lst) print(z)
The program returns:282.91 EUR 28.40 EUR
Another interesting aspect of this currency converter class in Python can be shown, if we add multiplication. You will easily understand that it makes no sense to allow expressions like "12.4 € * 3.4 $" (or in praefix notation: "€ 12.4 * $ 3.4"), but it makes perfect sense to evaluate "3 * 4.54 €". You can find the new currency converter class with the newly added methods for __mul__, __imul__ and __rmul__ in the following listing:""" The class "Ccy" can be used to define money values in various currencies. A Ccy instance has the string attributes ‘unit‘ (e.g. ‘CHF‘, ‘CAD‘ od ‘EUR‘ and the ‘value‘ as a float. A currency object consists of a value and the corresponding unit. """ class Ccy: currencies = {‘CHF‘: 1.0821202355817312, ‘CAD‘: 1.488609845538393, ‘GBP‘: 0.8916546282920325, ‘JPY‘: 114.38826536281809, ‘EUR‘: 1.0, ‘USD‘: 1.11123458162018} def __init__(self, value, unit="EUR"): self.value = value self.unit = unit def __str__(self): return "{0:5.2f}".format(self.value) + " " + self.unit def __repr__(self): return ‘Ccy(‘ + str(self.value) + ‘, "‘ + self.unit + ‘")‘ def changeTo(self, new_unit): """ An Ccy object is transformed from the unit "self.unit" to "new_unit" """ self.value = (self.value / Ccy.currencies[self.unit] * Ccy.currencies[new_unit]) self.unit = new_unit def __add__(self, other): """ Defines the ‘+‘ operator. If other is a CCy object the currency values are added and the result will be the unit of self. If other is an int or a float, other will be treated as a Euro value. """ if type(other) == int or type(other) == float: x = (other * Ccy.currencies[self.unit]) else: x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit]) return Ccy(x + self.value, self.unit) def __iadd__(self, other): """ Similar to __add__ """ if type(other) == int or type(other) == float: x = (other * Ccy.currencies[self.unit]) else: x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit]) self.value += x return self def __radd__(self, other): res = self + other if self.unit != "EUR": res.changeTo("EUR") return res # __sub__, __isub__ and __rsub__ can be defined analogue def __mul__(self, other): """ Multiplication is only defined as a scalar multiplication, i.e. a money value can be multiplied by an int or a float. It is not possible to multiply to money values """ if type(other)==int or type(other)==float: return Ccy(self.value * other, self.unit) else: raise TypeError("unsupported operand type(s) for *: ‘Ccy‘ and " + type(other).__name__) def __rmul__(self, other): return self.__mul__(other) def __imul__(self, other): if type(other)==int or type(other)==float: self.value *= other return self else: raise TypeError("unsupported operand type(s) for *: ‘Ccy‘ and " + type(other).__name__)
Assuming that you have saved the class under the name currency_converter, you can use it in the following way in the command shell:>>> from currency_converter import Ccy >>> x = Ccy(10.00, "EUR") >>> y = Ccy(10.00, "GBP") >>> x + y Ccy(21.215104685942173, "EUR") >>> print(x + y) 21.22 EUR >>> print(2*x + y*0.9) 30.09 EUR >>>
We can further improve our currency converter class by using a function get_currencies, which downloads the latest exchange rates from finance.yahoo.com. This function returns an exchange rates dictionary in our previously used format. The function is in a module called exchange_rates.py This is the code of the function exchange_rates.py:from urllib.request import urlopen from bs4 import BeautifulSoup def get_currency_rates(base="USD"): """ The file at location url is read in and the exchange rates are extracted """ url = "https://finance.yahoo.com/webservice/v1/symbols/allcurrencies/quote" data = urlopen(url).read() data = data.decode(‘utf-8‘) soup = BeautifulSoup(data, ‘html.parser‘) data = soup.get_text() flag = False currencies = {} for line in data.splitlines(): if flag: value = float(line) flag = False currencies[currency] = value if line.startswith("USD/"): flag = True currency = line[4:7] currencies["USD"] = 1 # we must add it, because it‘s not included in file if base != "USD": base_currency_rate = currencies[base] for currency in currencies: currencies[currency] /= base_currency_rate return currencies
We can import this function from our module. (You have to save it somewhare in your Python path or the directory where your program runs):from exchange_rates import get_currency_rates class Ccy: currencies = get_currencies() # continue with the code from the previous version
We save this version as currency_converter_web.>>> from currency_converter_web import Ccy >>> x = Ccy(1000, "JPY") >>> y = Ccy(10, "CHF") >>> z = Ccy(15, "CAD") >>> print(2*x + 4.11*y + z) 7722.98 JPY >>>
三,
Python - Magic Methods
Magic methods in Python are the special methods which add "magic" to your class. Magic methods are not meant to be invoked directly by you, but the invocation happens internally from the class on a certain action. For example, when you add two numbers using the + operator, internally, the __add__()
method will be called.
Built-in classes in Python define many magic methods. Use the dir()
function to see the number of magic methods inherited by a class. For example, the following lists all the attributes and methods defined in the int
class.
>>> dir(int)
[‘__abs__‘, ‘__add__‘, ‘__and__‘, ‘__bool__‘, ‘__ceil__‘,
‘__class__‘, ‘__delattr__‘, ‘__dir__‘, ‘__divmod__‘, ‘__doc__‘,
‘__eq__‘, ‘__float__‘, ‘__floor__‘, ‘__floordiv__‘, ‘__format__‘,
‘__ge__‘, ‘__getattribute__‘, ‘__getnewargs__‘, ‘__gt__‘, ‘__hash__‘,
‘__index__‘, ‘__init__‘, ‘__init_subclass__‘, ‘__int__‘, ‘__invert__‘,
‘__le__‘, ‘__lshift__‘, ‘__lt__‘, ‘__mod__‘, ‘__mul__‘, ‘__ne__‘,
‘__neg__‘, ‘__new__‘, ‘__or__‘, ‘__pos__‘, ‘__pow__‘, ‘__radd__‘,
‘__rand__‘, ‘__rdivmod__‘, ‘__reduce__‘, ‘__reduce_ex__‘, ‘__repr__‘,
‘__rfloordiv__‘, ‘__rlshift__‘, ‘__rmod__‘, ‘__rmul__‘, ‘__ror__‘,
‘__round__‘, ‘__rpow__‘, ‘__rrshift__‘, ‘__rshift__‘, ‘__rsub__‘,
‘__rtruediv__‘, ‘__rxor__‘, ‘__setattr__‘, ‘__sizeof__‘, ‘__str__‘,
‘__sub__‘, ‘__subclasshook__‘, ‘__truediv__‘, ‘__trunc__‘, ‘__xor__‘,
‘bit_length‘, ‘conjugate‘, ‘denominator‘, ‘from_bytes‘, ‘imag‘,
‘numerator‘, ‘real‘, ‘to_bytes‘]
As you can see above, the int class includes various magic methods surrounded by double underscores.
For example, the __add__
method is a magic method which gets called when we add two numbers using the + operator.
Consider the following example.
>>> num=10
>>> num + 5
15
>>> num.__add__(5)
15
As you can see, when you do num+10
, the + operator calls the __add__(10)
method.
You can also call num.__add__(5)
directly which
will give the same result.
However, as mentioned before, magic methods are not meant to
be called directly, but internally, through some other methods or
actions.
Magic methods are most frequently used to define overloaded behaviours of predefined operators in Python. For instance, arithmetic operators by default operate upon numeric operands. This means that numeric objects must be used along with operators like +, -, *, /, etc. The + operator is also defined as a concatenation operator in string, list and tuple classes. We can say that the + operator is overloaded.
In order to make the overloaded behaviour available in your
own custom class, the corresponding magic method should be overridden.
For example, in order to use the + operator with objects of a
user-defined class, it should include the __add__()
method.
Let‘s see how to implement and use some of the important magic methods.
__new__() method
Languages such as Java and C# use the new operator to create a new instance of a class. In Python the __new__()
magic method is implicitly called before the __init__()
method. The __new__()
method returns a new object, which is then initialized by __init__()
.
class employee:
def __new__(cls):
print ("__new__ magic method is called")
inst = object.__new__(cls)
return inst
def __init__(self):
print ("__init__ magic method is called")
self.name=‘Satya‘
The above example will produce the following output when you create an instance of the Employee
class.
>>> e1=employee()
__new__ magic method is called
__init__ magic method is called
Thus, the __new__()
method is called before the __init__()
method.
__str__() method
Another useful magic method is __str__()
. It is overridden to return a printable string representation of any user defined class.
We have seen str()
built-in function which returns a string from the object parameter. For example, str(12)
returns ‘12‘. When invoked, it calls the __str__()
method in the int class.
>>> num=12
>>> str(num)
‘12‘
>>> #This is equivalent to
>>> int.__str__(num)
‘12‘
Let us now override the __str__()
method in the employee class to return a string representation of its object.
class employee:
def __init__(self):
self.name=‘Swati‘
self.salary=10000
def __str__(self):
return ‘name=‘+self.name+‘ salary=$‘+str(self.salary)
See how the str()
function internally calls the __str__()
method defined in the employee class. This is why it is called a magic method!
>>> e1=employee()
>>> print(e1)
name=Swati salary=$10000
__add__() method
In following example, a class named distance is defined with two instance attributes - ft and inch. The addition of these two distance objects is desired to be performed using the overloading + operator.
To achieve this, the magic method __add__()
is overridden, which performs the addition of the ft and inch attributes of the two objects.
The __str__()
method returns the object‘s string representation.
class distance:
def __init__(self, x=None,y=None):
self.ft=x
self.inch=y
def __add__(self,x):
temp=distance()
temp.ft=self.ft+x.ft
temp.inch=self.inch+x.inch
if temp.inch>=12:
temp.ft+=1
temp.inch-=12
return temp
def __str__(self):
return ‘ft:‘+str(self.ft)+‘ in: ‘+str(self.inch)
Run the above Python script to verify the overloaded operation of the + operator.
>>> d1=distance(3,10)
>>> d2=distance(4,4)
>>> print("d1= {} d2={}".format(d1, d2))
d1= ft:3 in: 10 d2=ft:4 in: 4
>>>d3=d1+d2
>>>print(d3)
ft:8 in: 2
__ge__() method
The following method is added in the distance class to overload the >=
operator.
class distance:
def __init__(self, x=None,y=None):
self.ft=x
self.inch=y
def __ge__(self, x):
val1=self.ft*12+self.inch
val2=x.ft*12+x.inch
if val1>=val2:
return True
else:
return False
This method gets invoked when the >=
operator is used and returns True or False. Accordingly, the appropriate message can be displayed
>>>d1=distance(2,1)
>>>d2=distance(4,10)
>>>d1>=d2
False
Important Magic Methods
The following tables list important magic methods in Python 3.
Initialization and Construction | Description |
---|---|
__new__(cls, other) | To get called in an object‘s instantiation. |
__init__(self, other) | To get called by the __new__ method. |
__del__(self) | Destructor method. |
Unary operators and functions | Description |
---|---|
__pos__(self) | To get called for unary positive e.g. +someobject. |
__neg__(self) | To get called for unary negative e.g. -someobject. |
__abs__(self) | To get called by built-in abs() function. |
__invert__(self) | To get called for inversion using the ~ operator. |
__round__(self,n) | To get called by built-in round() function. |
__floor__(self) | To get called by built-in math.floor() function. |
__ceil__(self) | To get called by built-in math.ceil() function. |
__trunc__(self) | To get called by built-in math.trunc() function. |
Augmented Assignment | Description |
---|---|
__iadd__(self, other) | To get called on addition with assignment e.g. a +=b. |
__isub__(self, other) | To get called on subtraction with assignment e.g. a -=b. |
__imul__(self, other) | To get called on multiplication with assignment e.g. a *=b. |
__ifloordiv__(self, other) | To get called on integer division with assignment e.g. a //=b. |
__idiv__(self, other) | To get called on division with assignment e.g. a /=b. |
__itruediv__(self, other) | To get called on true division with assignment |
__imod__(self, other) | To get called on modulo with assignment e.g. a%=b. |
__ipow__(self, other) | To get called on exponentswith assignment e.g. a **=b. |
__ilshift__(self, other) | To get called on left bitwise shift with assignment e.g. a<<=b. |
__irshift__(self, other) | To get called on right bitwise shift with assignment e.g. a >>=b. |
__iand__(self, other) | To get called on bitwise AND with assignment e.g. a&=b. |
__ior__(self, other) | To get called on bitwise OR with assignment e.g. a|=b. |
__ixor__(self, other) | To get called on bitwise XOR with assignment e.g. a ^=b. |
Type Conversion Magic Methods | Description |
---|---|
__int__(self) | To get called by built-int int() method to convert a type to an int. |
__float__(self) | To get called by built-int float() method to convert a type to float. |
__complex__(self) | To get called by built-int complex() method to convert a type to complex. |
__oct__(self) | To get called by built-int oct() method to convert a type to octal. |
__hex__(self) | To get called by built-int hex() method to convert a type to hexadecimal. |
__index__(self) | To get called on type conversion to an int when the object is used in a slice expression. |
__trunc__(self) | To get called from math.trunc() method. |
String Magic Methods | Description |
---|---|
__str__(self) | To get called by built-int str() method to return a string representation of a type. |
__repr__(self) | To get called by built-int repr() method to return a machine readable representation of a type. |
__unicode__(self) | To get called by built-int unicode() method to return an unicode string of a type. |
__format__(self, formatstr) | To get called by built-int string.format() method to return a new style of string. |
__hash__(self) | To get called by built-int hash() method to return an integer. |
__nonzero__(self) | To get called by built-int bool() method to return True or False. |
__dir__(self) | To get called by built-int dir() method to return a list of attributes of a class. |
__sizeof__(self) | To get called by built-int sys.getsizeof() method to return the size of an object. |
Attribute Magic Methods | Description |
---|---|
__getattr__(self, name) | Is called when the accessing attribute of a class that does not exist. |
__setattr__(self, name, value) | Is called when assigning a value to the attribute of a class. |
__delattr__(self, name) | Is called when deleting an attribute of a class. |
Operator Magic Methods | Description |
---|---|
__add__(self, other) | To get called on add operation using + operator |
__sub__(self, other) | To get called on subtraction operation using - operator. |
__mul__(self, other) | To get called on multiplication operation using * operator. |
__floordiv__(self, other) | To get called on floor division operation using // operator. |
__div__(self, other) | To get called on division operation using / operator. |
__mod__(self, other) | To get called on modulo operation using % operator. |
__pow__(self, other[, modulo]) | To get called on calculating the power using ** operator. |
__lt__(self, other) | To get called on comparison using < operator. |
__le__(self, other) | To get called on comparison using <= operator. |
__eq__(self, other) | To get called on comparison using == operator. |
__ne__(self, other) | To get called on comparison using != operator. |
__ge__(self, other) | To get called on comparison using >= operator. |
Thus, you can use the appropriate magic methods to add various functionalities in your custom class.
四,
魔法方法
|
含义
|
基本的魔法方法
|
|
__new__(cls[, ...]) | 1. __new__ 是在一个对象实例化的时候所调用的第一个方法 2. 它的第一个参数是这个类,其他的参数是用来直接传递给 __init__ 方法 3. __new__ 决定是否要使用该 __init__ 方法,因为 __new__ 可以调用其他类的构造方法或者直接返回别的实例对象来作为本类的实例,如果 __new__ 没有返回实例对象,则 __init__ 不会被调用 4. __new__ 主要是用于继承一个不可变的类型比如一个 tuple 或者 string |
__init__(self[, ...]) | 构造器,当一个实例被创建的时候调用的初始化方法 |
__del__(self) | 析构器,当一个实例被销毁的时候调用的方法 |
__call__(self[, args...]) | 允许一个类的实例像函数一样被调用:x(a, b) 调用 x.__call__(a, b) |
__len__(self) | 定义当被 len() 调用时的行为 |
__repr__(self) | 定义当被 repr() 调用或者直接执行对象时的行为 |
__str__(self) | 定义当被 str() 调用或者打印对象时的行为 |
__bytes__(self) | 定义当被 bytes() 调用时的行为 |
__hash__(self) | 定义当被 hash() 调用时的行为 |
__bool__(self) | 定义当被 bool() 调用时的行为,应该返回 True 或 False |
__format__(self, format_spec) | 定义当被 format() 调用时的行为 |
有关属性
|
|
__getattr__(self, name) | 定义当用户试图获取一个不存在的属性时的行为 |
__getattribute__(self, name) | 定义当该类的属性被访问时的行为 |
__setattr__(self, name, value) | 定义当一个属性被设置时的行为 |
__delattr__(self, name) | 定义当一个属性被删除时的行为 |
__dir__(self) | 定义当 dir() 被调用时的行为 |
__get__(self, instance, owner) | 定义当描述符的值被取得时的行为 |
__set__(self, instance, value) | 定义当描述符的值被改变时的行为 |
__delete__(self, instance) | 定义当描述符的值被删除时的行为 |
比较操作符
|
|
__lt__(self, other) | 定义小于号的行为:x < y 调用 x.__lt__(y) |
__le__(self, other) | 定义小于等于号的行为:x <= y 调用 x.__le__(y) |
__eq__(self, other) | 定义等于号的行为:x == y 调用 x.__eq__(y) |
__ne__(self, other) | 定义不等号的行为:x != y 调用 x.__ne__(y) |
__gt__(self, other) | 定义大于号的行为:x > y 调用 x.__gt__(y) |
__ge__(self, other) | 定义大于等于号的行为:x >= y 调用 x.__ge__(y) |
算数运算符
|
|
__add__(self, other) | 定义加法的行为:+ |
__sub__(self, other) | 定义减法的行为:- |
__mul__(self, other) | 定义乘法的行为:* |
__truediv__(self, other) | 定义真除法的行为:/ |
__floordiv__(self, other) | 定义整数除法的行为:// |
__mod__(self, other) | 定义取模算法的行为:% |
__divmod__(self, other) | 定义当被 divmod() 调用时的行为 |
__pow__(self, other[, modulo]) | 定义当被 power() 调用或 ** 运算时的行为 |
__lshift__(self, other) | 定义按位左移位的行为:<< |
__rshift__(self, other) | 定义按位右移位的行为:>> |
__and__(self, other) | 定义按位与操作的行为:& |
__xor__(self, other) | 定义按位异或操作的行为:^ |
__or__(self, other) | 定义按位或操作的行为:| |
反运算
|
|
__radd__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rsub__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rmul__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rtruediv__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rfloordiv__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rmod__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rdivmod__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rpow__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rlshift__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rrshift__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rand__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__rxor__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
__ror__(self, other) | (与上方相同,当左操作数不支持相应的操作时被调用) |
增量赋值运算
|
|
__iadd__(self, other) | 定义赋值加法的行为:+= |
__isub__(self, other) | 定义赋值减法的行为:-= |
__imul__(self, other) | 定义赋值乘法的行为:*= |
__itruediv__(self, other) | 定义赋值真除法的行为:/= |
__ifloordiv__(self, other) | 定义赋值整数除法的行为://= |
__imod__(self, other) | 定义赋值取模算法的行为:%= |
__ipow__(self, other[, modulo]) | 定义赋值幂运算的行为:**= |
__ilshift__(self, other) | 定义赋值按位左移位的行为:<<= |
__irshift__(self, other) | 定义赋值按位右移位的行为:>>= |
__iand__(self, other) | 定义赋值按位与操作的行为:&= |
__ixor__(self, other) | 定义赋值按位异或操作的行为:^= |
__ior__(self, other) | 定义赋值按位或操作的行为:|= |
一元操作符
|
|
__pos__(self) | 定义正号的行为:+x |
__neg__(self) | 定义负号的行为:-x |
__abs__(self) | 定义当被 abs() 调用时的行为 |
__invert__(self) | 定义按位求反的行为:~x |
类型转换
|
|
__complex__(self) | 定义当被 complex() 调用时的行为(需要返回恰当的值) |
__int__(self) | 定义当被 int() 调用时的行为(需要返回恰当的值) |
__float__(self) | 定义当被 float() 调用时的行为(需要返回恰当的值) |
__round__(self[, n]) | 定义当被 round() 调用时的行为(需要返回恰当的值) |
__index__(self) | 1. 当对象是被应用在切片表达式中时,实现整形强制转换 2. 如果你定义了一个可能在切片时用到的定制的数值型,你应该定义 __index__ 3. 如果 __index__ 被定义,则 __int__ 也需要被定义,且返回相同的值 |
上下文管理(with 语句)
|
|
__enter__(self) | 1. 定义当使用 with 语句时的初始化行为 2. __enter__ 的返回值被 with 语句的目标或者 as 后的名字绑定 |
__exit__(self, exc_type, exc_value, traceback) | 1. 定义当一个代码块被执行或者终止后上下文管理器应该做什么 2. 一般被用来处理异常,清除工作或者做一些代码块执行完毕之后的日常工作 |
容器类型
|
|
__len__(self) | 定义当被 len() 调用时的行为(返回容器中元素的个数) |
__getitem__(self, key) | 定义获取容器中指定元素的行为,相当于 self[key] |
__setitem__(self, key, value) | 定义设置容器中指定元素的行为,相当于 self[key] = value |
__delitem__(self, key) | 定义删除容器中指定元素的行为,相当于 del self[key] |
__iter__(self) | 定义当迭代容器中的元素的行为 |
__reversed__(self) | 定义当被 reversed() 调用时的行为 |
__contains__(self, item) | 定义当使用成员测试运算符(in 或 not in)时的行为 |
以上是关于Python类的内建方法-即所谓的魔术方法的主要内容,如果未能解决你的问题,请参考以下文章