
Parametrizing fixtures and test functions
*****************************************

pytest supports test parametrization in several well-integrated ways:

* "pytest.fixture()" allows to define *parametrization at the level of
  fixture functions*.

* @pytest.mark.parametrize allows to define parametrization at the
  function or class level, provides multiple argument/fixture sets for
  a particular test function or class.

* pytest_generate_tests enables implementing your own custom dynamic
  parametrization scheme or extensions.


"@pytest.mark.parametrize": parametrizing test functions
========================================================

New in version 2.2.

The builtin "pytest.mark.parametrize" decorator enables
parametrization of arguments for a test function.  Here is a typical
example of a test function that implements checking that a certain
input leads to an expected output:

   # content of test_expectation.py
   import pytest
   @pytest.mark.parametrize(("input", "expected"), [
       ("3+5", 8),
       ("2+4", 6),
       ("6*9", 42),
   ])
   def test_eval(input, expected):
       assert eval(input) == expected

Here, the "@parametrize" decorator defines three different argument
sets for the two "(input, output)" arguments of the "test_eval"
function which will thus run three times:

   $ py.test
   =========================== test session starts ============================
   platform linux2 -- Python 2.7.3 -- pytest-2.3.5
   collected 3 items

   test_expectation.py ..F

   ================================= FAILURES =================================
   ____________________________ test_eval[6*9-42] _____________________________

   input = '6*9', expected = 42

       @pytest.mark.parametrize(("input", "expected"), [
           ("3+5", 8),
           ("2+4", 6),
           ("6*9", 42),
       ])
       def test_eval(input, expected):
   >       assert eval(input) == expected
   E       assert 54 == 42
   E        +  where 54 = eval('6*9')

   test_expectation.py:8: AssertionError
   ==================== 1 failed, 2 passed in 0.01 seconds ====================

As expected only one pair of input/output values fails the simple test
function. And as usual with test function arguments, you can see the
"input" and "output" values in the traceback.

Note that there ways how you can mark a class or a module, see
*Marking test functions with attributes*.


Basic "pytest_generate_tests" example
=====================================

Sometimes you may want to implement your own parametrization scheme or
implement some dynamism for determining the parameters or scope of a
fixture.   For this, you can use the "pytest_generate_tests" hook
which is called when collecting a test function.  Through the passed
in *metafunc* object you can inspect the requesting test context and,
most importantly, you can call "metafunc.parametrize()" to cause
parametrization.

For example, let's say we want to run a test taking string inputs
which we want to set via a new py.test command line option.  Let's
first write a simple test accepting a "stringinput" fixture function
argument:

   # content of test_strings.py

   def test_valid_string(stringinput):
       assert stringinput.isalpha()

Now we add a "conftest.py" file containing the addition of a command
line option and the parametrization of our test function:

   # content of conftest.py

   def pytest_addoption(parser):
       parser.addoption("--stringinput", action="append", default=[],
           help="list of stringinputs to pass to test functions")

   def pytest_generate_tests(metafunc):
       if 'stringinput' in metafunc.fixturenames:
           metafunc.parametrize("stringinput",
                                metafunc.config.option.stringinput)

If we now pass two stringinput values, our test will run twice:

   $ py.test -q --stringinput="hello" --stringinput="world" test_strings.py
   ..

Let's also run with a stringinput that will lead to a failing test:

   $ py.test -q --stringinput="!" test_strings.py
   F
   ================================= FAILURES =================================
   ___________________________ test_valid_string[!] ___________________________

   stringinput = '!'

       def test_valid_string(stringinput):
   >       assert stringinput.isalpha()
   E       assert <built-in method isalpha of str object at 0x2ba729dab300>()
   E        +  where <built-in method isalpha of str object at 0x2ba729dab300> = '!'.isalpha

   test_strings.py:3: AssertionError

As expected our test function fails.

If you don't specify a stringinput it will be skipped because
"metafunc.parametrize()" will be called with an empty parameter
listlist:

   $ py.test -q -rs test_strings.py
   s
   ========================= short test summary info ==========================
   SKIP [1] /home/hpk/p/pytest/.tox/regen/local/lib/python2.7/site-packages/_pytest/python.py:974: got empty parameter set, function test_valid_string at /tmp/doc-exec-240/test_strings.py:1

For further examples, you might want to look at *more parametrization
examples*.


The **metafunc** object
=======================

metafunc objects are passed to the "pytest_generate_tests" hook. They
help to inspect a testfunction and to generate tests according to test
configuration or values specified in the class or module where a test
function is defined:

"metafunc.fixturenames": set of required function arguments for given
function

"metafunc.function": underlying python test function

"metafunc.cls": class object where the test function is defined in or
None.

"metafunc.module": the module object where the test function is
defined in.

"metafunc.config": access to command line opts and general config

"metafunc.funcargnames": alias for "fixturenames", for pre-2.3
compatibility
