So in case you haven’t noticed … I’m a little (VERY) in love with Python as a programming language. I use it for everything now, from web development, to scripting and everything in between.
As such, I thought I’d put together an article of the many amazing libraries that I think every Python programmer needs to know about. Now there’s no doubt in my mind that there are heaps that I’ve missed, but we have to start somewhere. I’ll try to keep revising this article as I come across more great libraries.
Web Development
- Flask: My web framework of choice right now. Don’t be fooled by the word “micro”, this bad boy can do everything under the sun once you plug in the necessary components. Be sure to also check out Werkzeug and Jinja2 libraries that Flask depends on. Werkzeug is a very extensive set of WSGI tools including a full caching implementation. Jinja2 is an extremely elegant templating language (and my personal favourite in Python).
- Django: A quality full-stack web framework with awesome documentation and everything you could need included. The upcoming addition of database migrations take Django to the next level.
- SQLAlchemy: An incredible database ORM which plugs beautifully into Flask.
- Alembic: Database migrations for SQLAlchemy written by the developer of SQLAlchemy itself.
- WTForms: An excellent form validation library which plugs really nicely into Flask.
- dogpile.cache: Another excellent caching library by the author of SQLAlchemy. Much like Werkzeug, this supports the redis-py, python-memcached and pylibmc libraries. For use with memcache, the preference seems to be pylibmc which is mainly written in C and performs around twice as fast as python-memcached.
- Gunicorn: My first choice for serving WSGI applications. Absolutely rock solid and performs great. Also worth checking out are gevent and uWSGI.
Web Interaction
- Selenium: A library that allows you to automate interaction with websites. I highly recommend the PhantomJS web browser for use with Selenium on servers.
- splinter: Although Selenium is excellent, its API can be quite low level when writing tests. The splinter library sits on top of Selenium and provides a clean and simple API for interacting with websites.
- Beautiful Soup: A really easy to use XML / HTML parsing library.
- lxml: A similar library to Beautiful Soup but with much better performance and arguably a cleaner API.
- Requests: The most elegant way to make requests to websites and retrieve responses. Great for interacting with APIs and similar.
Automation
- Ansible: An absolutely awesome build automation tool which compares favourably to Puppet in many important areas.
- Salt: Another Python-based build automation tool which competes with Puppet and Chef.
- Fabric: An amazing automation tool for deploying code, running tasks and interacting with servers.
- Invoke: By the developers who brought us Fabric comes Invoke which will be used by Fabric 2.0 when it’s released. Invoke itself is a very clean and powerful make file replacement in Python (the equivalent of Rake in Ruby).
- BuildBot: A build automation system similar to Jenkins.
Testing Tools
- Coverage.py: Allows you to examine what lines of your code are missing unit tests. This blew my mind the first time I came across it! :)
- mock: Mock allows you to replace bits of your system to help with simulating necessary conditions in unit tests.
- nose: Enhances the included unittest to make written tests more elegant and easier to run.
- pytest: Another Python unit test enhancer which adds some helpful features and functionality.
- factory_boy: Database fixtures for testing web applications.
- fake-factory: An awesome library for generating fake data for testing purposes.
- behave: A fantastic BDD tool for Python very similar to Cucumber.
- sure: Provides a very human-friendly way to write tests.
- pyshould: Very similar to sure but can also be extended with custom assertions.
Miscellaneous
- Flake8: A great tool which combines PEP8 and PyFlakes to help validate your code. I use this religiously.
- IPython: A significantly improved Python shell.
- pandas: A library providing a range of tools for data analysis.
- Pygments: The reference for syntax highlighting of code.
- python-daemon: A library which makes it much easier to implement a daemon in Python.
- python-dateutil: Some very handy functions which make working with dates a lot easier.
- PyYAML: Allows you to read and write YAML files seamlessly. YAML is my preferred format for storing configurations where possible.
- Sentry: A powerful realtime log aggregation system which collects logs from multiple places and makes them available via a web interface.
- six: A very handy library for making code compatible with both Python 2 and Python 3
- Watchdog: A library that can watch for changes to files and take chosen actions when a change is detected.
- Supervisor: A neat little system which keeps track of running daemons or processes and takes action if any failures occur.
Please let me know if I’ve missed anything obvious guys! :)