Python Book
🇺🇦 Stand with Ukraine🎓Training Suite
  • Book overview
  • Notes about this book
  • 1. Introduction to Python
    • What is Python
    • Basic syntax
    • Objects in Python
    • Python overview
    • Installation, IDEs etc.
    • ipython
    • Sources for self-learning
  • 2. Strings and numbers
    • Getting help
    • Introspection
    • Basic types
    • None object
    • Numbers
    • Strings
    • Unicode
    • String Formatting
    • Regular expressions
    • Sources for self-learning
  • 3. Containers
    • Data Structures
    • Lists
    • Tuples
    • Dictionaries
    • Sets
    • Conditions
    • Loops
    • Additional modules
    • Sources for self-learning
  • 4. Functions
    • Functions
    • Scopes of visibility
    • Generators
    • Lambdas
    • Type hints
    • Function internals
    • Sources for self-learning
  • 5. Functional Programming
    • Builtins
    • Iterable
    • Iterator
    • Functional Programming
    • Functools
    • Comprehensions
    • Additional modules
    • Sources for self-learning
  • 6. Code Styling
    • Zen of Python
    • Lint
    • PEP 8
    • Modules
    • Packages
    • Sources for self-learning
  • 7. OOP
    • OOP Basics
    • Code design principles
    • Classes
    • Method Resolution Order
    • Magic attributes and methods
    • Super
    • Sources for self-learning
  • 8. Decorators, Exceptions
    • Decorators
    • Exceptions
    • Sources for self-learning
  • 9. Testing
    • Basic Terminology
    • Testing theory
    • Dev unit testing vs QA automated testing
    • Best Practices
    • Doctest
    • Unittest
    • Test Runners
    • Pytest
    • Nose
    • Continuous Integration
  • 10. System Libs
    • Working with files
    • System libraries
    • Subprocess
    • Additional CLI libraries
Powered by GitBook
On this page
  • Running pytest
  • Filtering tests
  • Reporting
  • Coverage

Was this helpful?

Edit on Git
  1. 9. Testing

Pytest

This is the top modern Python testing framework at this moment. It has a lot of options, it offers a lot of unique (non-Junit-style) testing capabilities that at the same time are much easier to compehend and extend if needed.

Pytest has also a runner.

To install and use the runner:

pip install -U pytest pytest-cov

This will add pytest binary to PATH allowing to run this command.

To see all pytest CLI options:

> pytest --help

This will produce a large list of options (In case additional plugins installer the help will be bigger).

Let's check the main Pytest options per functionality.

Running pytest

  • Auto-discovery of all tests starting with current directory:

pytest

  • Run tests in a module

pytest test_mod.py

  • Run tests in a directory

pytest testing/

Options:

  • -q - quiet mode

  • -v - verbose mode

  • -s - no capture (show all print() stdout/stderr outputs)

  • -x - exit instantly on first error or failed test.

  • --runxfail - report the results of xfail tests as if they were not marked

  • --lf - rerun only the tests that failed at the last run (or all if none failed)

  • --ff - run all tests, but run the last failures first.

  • --nf - run tests from new files first, then the rest of the tests sorted by file mtime

Filtering tests

  • Run tests by keyword expressions

pytest -k "MyClass and not method"

Here the keyword option allows to specify a query-like filter which is based on the checks of keyword presense in filenames, class names and function names. The example above will run TestMyClass.test_something but not TestMyClass.test_method_simple.

  • Run a specific test within a module:

pytest test_mod.py::test_func

pytest test_mod.py::TestClass::test_method

  • Run tests by marker expressions (decorated with the @pytest.mark.critical)

pytest -m critical

pytest -m "critical and security"

Reporting

  • The -r flag can be used to display a “short test summary info” at the end of the test session, making it easy in large test suites to get a clear picture of all failures, skips, xfails, etc. It defaults to fE - faults/fails and errors.

Here is the full list of available characters that can be used:

  • f - failed

  • E - error

  • s - skipped

  • x - xfailed

  • X - xpassed

  • p - passed

  • P - passed with output

Special characters for (de)selection of groups:

  • a - all except pP

  • A - all

  • N - none, this can be used to display nothing (since fE is the default)

Additional reporting options:

  • --tb=style - traceback print mode (auto/long/short/line/native/no).

  • --show-capture={no,stdout,stderr,log,all}

    • Controls how captured stdout/stderr/log is shown on failed tests. Default is 'all'.

  • --full-trace - don't cut any tracebacks (default is to cut).

  • --color=color - color terminal output (yes/no/auto).

  • --code-highlight={yes,no}

    • Whether code should be highlighted (only if --color is also enabled)

  • --pastebin=mode - send failed|all info to bpaste.net pastebin service.

  • --junit-xml=path - create junit-xml style report file at given path.

  • --junit-prefix=str - prepend prefix to classnames in junit-xml output

Coverage

Coverage shows what part of the code is covered by unittest. Also there is a metric called "branch coverage" which shows how much the logic of the application is covered by tests (all if/else branches are covered).

It possible to install the plugin for pytest for coverage:

pip install -U pytest-cov

And then to use it:

pytest --cov --cov-report=xml

to produce Cobertura-compliant XML-based report (that can be used in CI tool for reporting).

PreviousTest RunnersNextNose

Last updated 3 years ago

Was this helpful?