
Good Integration Practises
**************************


Work with virtual environments
==============================

We recommend to use virtualenv environments and use easy_install (or
pip) for installing your application dependencies as well as the
"pytest" package itself.  This way you will get a much more
reproducible environment.  A good tool to help you automate test runs
against multiple dependency configurations or Python interpreters is
tox.


Use tox and Continuous Integration servers
==========================================

If you frequently release code to the public you may want to look into
tox, the virtualenv test automation tool and its pytest support. The
basic idea is to generate a JUnitXML file through the "--
junitxml=PATH" option and have a continuous integration server like
Jenkins pick it up and generate reports.


Create a py.test standalone script
==================================

If you are a maintainer or application developer and want others to
easily run tests you can generate a completely standalone "py.test"
script:

   py.test --genscript=runtests.py

generates a "runtests.py" script which is a fully functional basic
"py.test" script, running unchanged under Python2 and Python3. You can
tell people to download the script and then e.g.  run it like this:

   python runtests.py


Integrating with distutils / "python setup.py test"
===================================================

You can integrate test runs into your distutils or setuptools based
project.  Use the genscript method to generate a standalone py.test
script:

   py.test --genscript=runtests.py

and make this script part of your distribution and then add this to
your "setup.py" file:

   from distutils.core import setup, Command
   # you can also import from setuptools

   class PyTest(Command):
       user_options = []
       def initialize_options(self):
           pass
       def finalize_options(self):
           pass
       def run(self):
           import sys,subprocess
           errno = subprocess.call([sys.executable, 'runtests.py'])
           raise SystemExit(errno)
   setup(
       #...,
       cmdclass = {'test': PyTest},
       #...,
   )

If you now type:

   python setup.py test

this will execute your tests using "runtests.py". As this is a
standalone version of "py.test" no prior installation whatsoever is
required for calling the test command. You can also pass additional
arguments to the subprocess-calls such as your test directory or other
options.


Integration with setuptools/distribute test commands
====================================================

Distribute/Setuptools support test requirements, which means its
really easy to extend its test command to support running a pytest
from test requirements:

   from setuptools.command.test import test as TestCommand
   import sys

   class PyTest(TestCommand):
       def finalize_options(self):
           TestCommand.finalize_options(self)
           self.test_args = []
           self.test_suite = True
       def run_tests(self):
           #import here, cause outside the eggs aren't loaded
           import pytest
           errno = pytest.main(self.test_args)
           sys.exit(errno)

   setup(
       #...,
       tests_require=['pytest'],
       cmdclass = {'test': PyTest},
       )

Now if you run:

   python setup.py test

this will download py.test if needed and then run py.test as you would
expect it to.


Conventions for Python test discovery
=====================================

"py.test" implements the following standard test discovery:

* collection starts from the initial command line arguments which may
  be directories, filenames or test ids.

* recurse into directories, unless they match "norecursedirs"

* "test_*.py" or "*_test.py" files, imported by their package name.

* "Test" prefixed test classes (without an "__init__" method)

* "test_" prefixed test functions or methods are test items

For examples of how to customize your test discovery *Changing
standard (Python) test discovery*.

Within Python modules, py.test also discovers tests using the standard
*unittest.TestCase* subclassing technique.


Choosing a test layout / import rules
=====================================

py.test supports common test layouts:

* inlining test directories into your application package, useful if
  you want to keep (unit) tests and actually tested code close
  together:

     mypkg/
         __init__.py
         appmodule.py
         ...
         test/
             test_app.py
             ...

* putting tests into an extra directory outside your actual
  application code, useful if you have many functional tests or want
  to keep tests separate from actual application code:

     mypkg/
         __init__.py
         appmodule.py
     tests/
         test_app.py
         ...

In both cases you usually need to make sure that "mypkg" is
importable, for example by using the setuptools "python setup.py
develop" method.

You can run your tests by pointing to it:

   py.test tests/test_app.py       # for external test dirs
   py.test mypkg/test/test_app.py  # for inlined test dirs
   py.test mypkg                   # run tests in all below test directories
   py.test                         # run all tests below current dir
   ...

Note: If py.test finds a "a/b/test_module.py" test file while recursing
  into the filesystem it determines the import name as follows:

  * find "basedir" -- this is the first "upward" (towards the root)
    directory not containing an "__init__.py".  If both the "a" and
    "b" directories contain an "__init__.py" the basedir will be the
    parent dir of "a".

  * perform "sys.path.insert(0, basedir)" to make the test module
    importable under the fully qualified import name.

  * "import a.b.test_module" where the path is determined by
    converting path separators "/" into "." characters.  This means
    you must follow the convention of having directory and file names
    map directly to the import names.

  The reason for this somewhat evolved importing technique is that in
  larger projects multiple test modules might import from each other
  and thus deriving a canonical import name helps to avoid surprises
  such as a test modules getting imported twice.
