Reference: https://www.toptal.com/python/python-design-patterns
Let’s say it again: Python is a high-level programming language with dynamic typing and dynamic binding. I would describe it as a powerful, high-level dynamic language. Many developers are in love with Python because of its clear syntax, well structured modules and packages, and for its enormous flexibility and range of modern features.
In Python, nothing obliges you to write classes and instantiate objects from them. If you don’t need complex structures in your project, you can just write functions. Even better, you can write a flat script for executing some simple and quick task without structuring the code at all.
At the same time Python is a 100 percent object-oriented language. How’s that? Well, simply put, everything in Python is an object. Functions are objects, first class objects (whatever that means). This fact about functions being objects is important, so please remember it.
So, you can write simple scripts in Python, or just open the Python terminal and execute statements right there (that’s so useful!). But at the same time, you can create complex frameworks, applications, libraries and so on. You can do so much in Python. There are of course a number of limitations, but that’s not the topic of this article.
However, because Python is so powerful and flexible, we need some rules (or patterns) when programming in it. So, let see what patterns are, and how they relate to Python. We will also proceed to implement a few essential Python design patterns.
Why Is Python Good For Patterns?
Any programming language is good for patterns. In fact, patterns should be considered in the context of any given programming language. Both the patterns, language syntax and nature impose limitations on our programming. The limitations that come from the language syntax and language nature (dynamic, functional, object oriented, and the like) can differ, as can the reasons behind their existence. The limitations coming from patterns are there for a reason, they are purposeful. That’s the basic goal of patterns; to tell us how to do something and how not to do it. We’ll speak about patterns, and especially Python design patterns, later.
Python’s philosophy is built on top of the idea of well thought out best practices. Python is a dynamic language (did I already said that?) and as such, already implements, or makes it easy to implement, a number of popular design patterns with a few lines of code. Some design patterns are built into Python, so we use them even without knowing. Other patterns are not needed due of the nature of the language.
For example, Factory is a structural Python design pattern aimed at creating new objects, hiding the instantiation logic from the user. But creation of objects in Python is dynamic by design, so additions like Factory are not necessary. Of course, you are free to implement it if you want to. There might be cases where it would be really useful, but they’re an exception, not the norm.
What is so good about Python’s philosophy? Let’s start with this (explore it in the Python terminal):
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
These might not be patterns in the traditional sense, but these are rules that define the “Pythonic” approach to programming in the most elegant and useful fashion.
We have also PEP-8 code guidelines that help structure our code. It’s a must for me, with some appropriate exceptions, of course. By the way, these exceptions are encouraged by PEP-8 itself:
“But most importantly: know when to be inconsistent – sometimes the style guide just doesn’t apply. When in doubt, use your best judgment. Look at other examples and decide what looks best. And don’t hesitate to ask!”
Combine PEP-8 with The Zen of Python (also a PEP - PEP-20), and you’ll have a perfect foundation to create readable and maintainable code. Add Design Patterns and you are ready to create every kind of software system with consistency and evolvability.
Python Design Patterns
What Is A Design Pattern?
Everything starts with the Gang of Four (GOF). Do a quick online search if you are not familiar with the GOF.
Design patterns are a common way of solving well known problems. Two main principles are in the bases of the design patterns defined by the GOF:
- Program to an interface not an implementation.
- Favor object composition over inheritance.
Let’s take a closer look at these two principles from the perspective of Python programmers.
Program to an interface not an implementation
Think about Duck Typing. In Python we don’t like to define interfaces and program classes according these interfaces, do we? But, listen to me! This doesn’t mean we don’t think about interfaces, in fact with Duck Typing we do that all the time.
Let’s say some words about the infamous Duck Typing approach to see how it fits in this paradigm: program to an interface.
We don’t bother with the nature of the object, we don’t have to care what the object is; we just want to know if it’s able to do what we need (we are only interested in the interface of the object).
Can the object quack? So, let it quack!
try:
bird.quack()
except AttributeError:
self.lol()
Did we define an interface for our duck? No! Did we program to the interface instead of the implementation? Yes! And, I find this so nice.
As Alex Martelli points out in his well known presentation about Design Patterns in Python, “Teaching the ducks to type takes a while, but saves you a lot of work afterwards!”
Favor object composition over inheritance
Now, that’s what I call a Pythonic principle! I have created fewer classes/subclasses compared to wrapping one class (or more often, several classes) in another class.
Instead of doing this:
class User(DbObject):
pass
We can do something like this:
class User:
_persist_methods = ['get', 'save', 'delete']
def __init__(self, persister):
self._persister = persister
def __getattr__(self, attribute):
if attribute in self._persist_methods:
return getattr(self._persister, attribute)
The advantages are obvious. We can restrict what methods of the wrapped class to expose. We can inject the persister instance in runtime! For example, today it’s a relational database, but tomorrow it could be whatever, with the interface we need (again those pesky ducks).
Composition is elegant and natural to Python.
Behavioral Patterns
Behavioural Patterns involve communication between objects, how objects interact and fulfil a given task. According to GOF principles, there are a total of 11 behavioral patterns in Python: Chain of responsibility, Command, Interpreter, Iterator, Mediator, Memento, Observer, State, Strategy, Template, Visitor.
I find these patterns very useful, but this does not mean the other pattern groups are not.
Iterator
Iterators are built into Python. This is one of the most powerful characteristics of the language. Years ago, I read somewhere that iterators make Python awesome, and I think this is still the case. Learn enough about Python iterators and generators and you’ll know everything you need about this particular Python pattern.
Chain of responsibility
This pattern gives us a way to treat a request using different methods, each one addressing a specific part of the request. You know, one of the best principles for good code is the Single Responsibility principle.
Every piece of code must do one, and only one, thing.
This principle is deeply integrated in this design pattern.
For example, if we want to filter some content we can implement different filters, each one doing one precise and clearly defined type of filtering. These filters could be used to filter offensive words, ads, unsuitable video content, and so on.
class ContentFilter(object):
def __init__(self, filters=None):
self._filters = list()
if filters is not None:
self._filters += filters
def filter(self, content):
for filter in self._filters:
content = filter(content)
return content
filter = ContentFilter([
offensive_filter,
ads_filter,
porno_video_filter])
filtered_content = filter.filter(content)
Command
This is one of the first Python design patterns I implemented as a programmer. That reminds me: Patterns are not invented, they are discovered. They exist, we just need to find and put them to use. I discovered this one for an amazing project we implemented many years ago: a special purpose WYSIWYM XML editor. After using this pattern intensively in the code, I read more about it on some sites.
The command pattern is handy in situations when, for some reason, we need to start by preparing what will be executed and then to execute it when needed. The advantage is that encapsulating actions in such a way enables Python developers to add additional functionalities related to the executed actions, such as undo/redo, or keeping a history of actions and the like.
Let’s see what a simple and frequently used example looks like:
class RenameFileCommand(object):
def __init__(self, from_name, to_name):
self._from = from_name
self._to = to_name
def execute(self):
os.rename(self._from, self._to)
def undo(self):
os.rename(self._to, self._from)
class History(object):
def __init__(self):
self._commands = list()
def execute(self, command):
self._commands.append(command)
command.execute()
def undo(self):
self._commands.pop().undo()
history = History()
history.execute(RenameFileCommand('docs/cv.doc', 'docs/cv-en.doc'))
history.execute(RenameFileCommand('docs/cv1.doc', 'docs/cv-bg.doc'))
history.undo()
history.undo()
Creational Patterns
Let’s start by pointing out that creational patterns are not commonly used in Python. Why? Because of the dynamic nature of the language.
Someone wiser than I once said that Factory is built into Python. It means that the language itself provides us with all the flexibility we need to create objects in a sufficiently elegant fashion; we rarely need to implement anything on top, like Singleton or Factory.
In one Python Design Patterns tutorial, I found a description of the creational design patterns that stated these design “patterns provide a way to create objects while hiding the creation logic, rather than instantiating objects directly using a new operator.”
That pretty much sums up the problem: We don’t have a new operator in Python!
Nevertheless, let’s see how we can implement a few, should we feel we might gain an advantage by using such patterns.
Singleton
The Singleton pattern is used when we want to guarantee that only one instance of a given class exists during runtime. Do we really need this pattern in Python? Based on my experience, it’s easier to simply create one instance intentionally and then use it instead of implementing the Singleton pattern.
But should you want to implement it, here is some good news: In Python, we can alter the instantiation process (along with virtually anything else). Remember the __new__() method I mentioned earlier? Here we go:
class Logger(object):
def __new__(cls, *args, **kwargs):
if not hasattr(cls, '_logger'):
cls._logger = super(Logger, cls
).__new__(cls, *args, **kwargs)
return cls._logger
In this example, Logger is a Singleton.
These are the alternatives to using a Singleton in Python:
- Use a module.
- Create one instance somewhere at the top-level of your application, perhaps in the config file.
- Pass the instance to every object that needs it. That’s a dependency injection and it’s a powerful and easily mastered mechanism.
Dependency Injection
I don’t intend to get into a discussion on whether dependency injection is a design pattern, but I will say that it’s a very good mechanism of implementing loose couplings, and it helps make our application maintainable and extendable. Combine it with Duck Typing and the Force will be with you. Always.
I listed it in the creational pattern section of this post because it deals with the question of when (or even better: where) the object is created. It’s created outside. Better to say that the objects are not created at all where we use them, so the dependency is not created where it is consumed. The consumer code receives the externally created object and uses it. For further reference, please read the most upvoted answer to this Stackoverflow question.
It’s a nice explanation of dependency injection and gives us a good idea of the potential of this particular technique. Basically the answer explains the problem with the following example: Don’t get things to drink from the fridge yourself, state a need instead. Tell your parents that you need something to drink with lunch.
Python offers us all we need to implement that easily. Think about its possible implementation in other languages such as Java and C#, and you’ll quickly realize the beauty of Python.
Let’s think about a simple example of dependency injection:
class Command:
def __init__(self, authenticate=None, authorize=None):
self.authenticate = authenticate or self._not_authenticated
self.authorize = authorize or self._not_autorized
def execute(self, user, action):
self.authenticate(user)
self.authorize(user, action)
return action()
if in_sudo_mode:
command = Command(always_authenticated, always_authorized)
else:
command = Command(config.authenticate, config.authorize)
command.execute(current_user, delete_user_action)
We inject the authenticator and authorizer methods in the Command class. All the Command class needs is to execute them successfully without bothering with the implementation details. This way, we may use the Command class with whatever authentication and authorization mechanisms we decide to use in runtime.
We have shown how to inject dependencies through the constructor, but we can easily inject them by setting directly the object properties, unlocking even more potential:
command = Command()
if in_sudo_mode:
command.authenticate = always_authenticated
command.authorize = always_authorized
else:
command.authenticate = config.authenticate
command.authorize = config.authorize
command.execute(current_user, delete_user_action)
There is much more to learn about dependency injection; curious people would search for IoC, for example.
But before you do that, read another Stackoverflow answer, the most upvoted one to this question.
Again, we just demonstrated how implementing this wonderful design pattern in Python is just a matter of using the built-in functionalities of the language.
Let’s not forget what all this means: The dependency injection technique allows for very flexible and easy unit-testing. Imagine an architecture where you can change data storing on-the-fly. Mocking a database becomes a trivial task, doesn’t it? For further information, you can check out Toptal’s Introduction to Mocking in Python.
You may also want to research Prototype, Builder and Factory design patterns.
Structural Patterns
Facade
This may very well be the most famous Python design pattern.
Imagine you have a system with a considerable number of objects. Every object is offering a rich set of API methods. You can do a lot of things with this system, but how about simplifying the interface? Why not add an interface object exposing a well thought-out subset of all API methods? A Facade!
Python Facade design pattern example:
class Car(object):
def __init__(self):
self._tyres = [Tyre('front_left'),
Tyre('front_right'),
Tyre('rear_left'),
Tyre('rear_right'), ]
self._tank = Tank(70)
def tyres_pressure(self):
return [tyre.pressure for tyre in self._tyres]
def fuel_level(self):
return self._tank.level
There is no surprise, no tricks, the Car class is a Facade, and that’s all.
Adapter
If Facades are used for interface simplification, Adapters are all about altering the interface. Like using a cow when the system is expecting a duck.
Let’s say you have a working method for logging information to a given destination. Your method expects the destination to have a write() method (as every file object has, for example).
def log(message, destination):
destination.write('[{}] - {}'.format(datetime.now(), message))
I would say it is a well written method with dependency injection, which allows for great extensibility. Say you want to log to some UDP socket instead to a file,you know how to open this UDP socket but the only problem is that the socket object has no write() method. You need an Adapter!
import socket
class SocketWriter(object):
def __init__(self, ip, port):
self._socket = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM)
self._ip = ip
self._port = port
def write(self, message):
self._socket.send(message, (self._ip, self._port))
def log(message, destination):
destination.write('[{}] - {}'.format(datetime.now(), message))
upd_logger = SocketWriter('1.2.3.4', '9999')
log('Something happened', udp_destination)
But why do I find adapter so important? Well, when it’s effectively combined with dependency injection, it gives us huge flexibility. Why alter our well-tested code to support new interfaces when we can just implement an adapter that will translate the new interface to the well known one?
You should also check out and master bridge and proxy design patterns, due to their similarity to adapter. Think how easy they are to implement in Python, and think about different ways you could use them in your project.
Decorator
Oh how lucky we are! Decorators are really nice, and we already have them integrated into the language. What I like the most in Python is that using it teaches us to use best practices. It’s not that we don’t have to be conscious about best practices (and design patterns, in particular), but with Python I feel like I’m following best practices, regardless. Personally, I find Python best practices are intuitive and second nature, and this is something appreciated by novice and elite developers alike.
The decorator pattern is about introducing additional functionality and in particular, doing it without using inheritance.
So, let’s check out how we decorate a method without using built-in Python functionality. Here is a straightforward example.
def execute(user, action):
self.authenticate(user)
self.authorize(user, action)
return action()
What is not so good here is that the execute function does much more than executing something. We are not following the single responsibility principle to the letter.
It would be good to simply write just following:
def execute(action):
return action()
We can implement any authorization and authentication functionality in another place, in a decorator, like so:
def execute(action, *args, **kwargs):
return action()
def autheticated_only(method):
def decorated(*args, **kwargs):
if check_authenticated(kwargs['user']):
return method(*args, **kwargs)
else:
raise UnauthenticatedError
return decorated
def authorized_only(method):
def decorated(*args, **kwargs):
if check_authorized(kwargs['user'], kwargs['action']):
return method(*args, **kwargs)
else:
raise UnauthorizeddError
return decorated
execute = authenticated_only(execute)
execute = authorized_only(execute)
Now the execute() method is:
- Simple to read
- Does only one thing (at least when looking at the code)
- Is decorated with authentication
- Is decorated with authorization
We write the same using Python’s integrated decorator syntax:
def autheticated_only(method):
def decorated(*args, **kwargs):
if check_authenticated(kwargs['user']):
return method(*args, **kwargs )
else:
raise UnauthenticatedError
return decorated
def authorized_only(method):
def decorated(*args, **kwargs):
if check_authorized(kwargs['user'], kwargs['action']):
return method(*args, **kwargs)
else:
raise UnauthorizedError
return decorated
@authorized_only
@authenticated_only
def execute(action, *args, **kwargs):
return action()
It is important to note that you are not limited to functions as decorators. A decorator may involve entire classes. The only requirement is that they must be callables. But we have no problem with that; we just need to define the __call__(self) method.
You may also want to take a closer look at Python’s functools module. There is much to discover there!
Conclusion
I have shown how natural and easy is to use Python’s design patterns, but I have also shown how programming in Python should be easy going, too.
“Simple is better than complex,” remember that? Maybe you have noticed that none of the design patterns is fully and formally described. No complex full-scale implementations were shown. You need to “feel” and implement them in the way that best fits your style and needs. Python is a great language and it gives you all the power you need to produce flexible and reusable code.
However, it gives you more than that. It gives you the “freedom” to write really bad code. Don’t do it! Don’t Repeat Yourself (DRY) and never write code lines longer than 80 characters. And don’t forget to use design patterns where applicable; it’s one of the best ways to learn from others and gain from their wealth of experience free of charge.