pytest: no-boilerplate testing
When I first encountered Holger Krekel's pytest this summer on Jeff Knupp's blog I felt like I had been living under a rock for years. I've been using Python's unittest framework since 2006 and nose to find tests since 2008, but here was another test framework that actually predates nose! pytest is a very mature testing tool for testing Python. My favorite features:
- It can run unittest, doctest, and nose, style tests suites, making it ideal for new and legacy projects.
- It includes an intuitively named
raisescontext manager for testing exceptions.
- You can define fixture arguments to generate baseline data. This is very, very different from Django-style fixtures.
pytest.iniyou can change the behavior of pytest.
- Integrates nicely with
Alright, lets dive into usage.
The first thing that pytest provides is test discovery. Like
nose, starting from the directory where it is run, it will find any
Python module prefixed with
test_ and will attempt to run any defined
unittest or function prefixed with
test_. pytest explores
properly defined Python packages, searching recursively through
directories that include
__init__.py modules. Since an image is
probably easier to read, here's a sample directory structure annotated
with which files are checked for tests:
address/ __init__.py envelope.py geo.py test_envelope.py # checked for tests test_geo.py # checked for tests records/ # pytest WON'T look here because it lacks __init__.py records.csv records.py test_records.py # skipped because records/ lacks __init__.py __init__.py main.py test_main.py # checked for tests
Now that I've explained which files are checked for tests, here is how pytest determines what in each Python module is run as a test.
- pytest just runs doctests and unittests.
- pytest runs any function prefixed with
test_as a test.
- pytest does its best to run tests written for nose.
Yes, pytest behaves similarly to nose in test discovery. Next is another feature that it shares with nose that I really enjoy.
Writing Tests as Functions
Python's unittest framework works, but it's always felt like too much boilerplate. I admit I like to write tests, but working with the unittest framework always dimmed that fun. I suppose this is why the assert keyword is useful, because it changes this:
import unittest class TestMyStuff(unittest.TestCase): def test_the_obvious(self): self.assertEqual(True, True) if __name__ == '__main__': unittest.main()
>>> assert True == True
The former is nine lines of code (seven if you are using pytest to find this test) to do what the assert statement does in one. However, the nine lines of unittest code has a couple major advantages:
- Not automatically run when stumbled on by the Python interpreter.
- Produces a more illuminating response than an uninformative AssertionError.
Fortunately, tools like pytest (and nose) provide the ability to
write tests as functions. This means we can combine the advantages of
def test_the_obvious(): assert True == True
Now we are down to just two lines of code! That could be increased to five if we called pytest the same as we did in the unittest example:
import pytest def test_the_obvious(): assert True == True if __name__ == '__main__': pytest.main()
The next part is wonderful. If an
assert statement fails, then
pytest provides a very informative response. Let's check it out by
running the following code:
import pytest def test_gonna_fail(): assert True == False # Going to fail here on line 4 if __name__ == '__main__': pytest.main()
When I run this code, I get the following response:
==================== FAILURES ===================== ----------------- test_gonna_fail ----------------- def test_gonna_fail(): > assert True == False E assert True == False samples.py:4: AssertionError ======== 1 failed, 0 passed in 0.1 seconds ========
As you can see, pytest identified where the
assert statement failed on
line 4 and displays exactly caused the failure (
True did not equal
False). Very nice indeed.
In my next blog post I describe the following features of writing tests with pytest.
- Fixture Teardown