Incompatible changes
A number of incompatible changes were made in this release, with the intent of removing features deprecated for a long time or change existing behaviors in order to make them less surprising/more useful.
Reinterpretation mode has now been removed. Only plain and rewrite mode are available, consequently the --assert=reinterp option is no longer available. This also means files imported from plugins or conftest.py will not benefit from improved assertions by default, you should use pytest.register_assert_rewrite() to explicitly turn on assertion rewriting for those files. Thanks @flub for the PR.
The following deprecated commandline options were removed:
Thanks to @RedBeardCode for the PR (#1664).
ImportErrors in plugins now are a fatal error instead of issuing a pytest warning (#1479). Thanks to @The-Compiler for the PR.
Removed support code for Python 3 versions < 3.3 (#1627).
Removed all py.test-X* entry points. The versioned, suffixed entry points were never documented and a leftover from a pre-virtualenv era. These entry points also created broken entry points in wheels, so removing them also removes a source of confusion for users (#1632). Thanks @obestwalter for the PR.
pytest.skip() now raises an error when used to decorate a test function, as opposed to its original intent (to imperatively skip a test inside a test function). Previously this usage would cause the entire module to be skipped (#607). Thanks @omarkohl for the complete PR (#1519).
Exit tests if a collection error occurs. A poll indicated most users will hit CTRL-C anyway as soon as they see collection errors, so pytest might as well make that the default behavior (#1421). A --continue-on-collection-errors option has been added to restore the previous behaviour. Thanks @olegpidsadnyi and @omarkohl for the complete PR (#1628).
Renamed the pytest pdb module (plugin) into debugging to avoid clashes with the builtin pdb module.
Raise a helpful failure message when requesting a parametrized fixture at runtime, e.g. with request.getfixturevalue. Previously these parameters were simply never defined, so a fixture decorated like @pytest.fixture(params=[0, 1, 2]) only ran once (#460). Thanks to @nikratio for the bug report, @RedBeardCode and @tomviner for the PR.
_pytest.monkeypatch.monkeypatch class has been renamed to _pytest.monkeypatch.MonkeyPatch so it doesn't conflict with the monkeypatch fixture.
--exitfirst / -x can now be overridden by a following --maxfail=N and is just a synonym for --maxfail=1.
New Features
Support nose-style __test__ attribute on methods of classes, including unittest-style Classes. If set to False, the test will not be collected.
New doctest_namespace fixture for injecting names into the namespace in which doctests run. Thanks @milliams for the complete PR (#1428).
New --doctest-report option available to change the output format of diffs when running (failing) doctests (implements #1749). Thanks @hartym for the PR.
New name argument to pytest.fixture decorator which allows a custom name for a fixture (to solve the funcarg-shadowing-fixture problem). Thanks @novas0x2a for the complete PR (#1444).
New approx() function for easily comparing floating-point numbers in tests. Thanks @kalekundert for the complete PR (#1441).
Ability to add global properties in the final xunit output file by accessing the internal junitxml plugin (experimental). Thanks @tareqalayan for the complete PR #1454).
New ExceptionInfo.match() method to match a regular expression on the string representation of an exception (#372). Thanks @omarkohl for the complete PR (#1502).
__tracebackhide__ can now also be set to a callable which then can decide whether to filter the traceback based on the ExceptionInfo object passed to it. Thanks @The-Compiler for the complete PR (#1526).
New pytest_make_parametrize_id(config, val) hook which can be used by plugins to provide friendly strings for custom types. Thanks @palaviv for the PR.
capsys and capfd now have a disabled() context-manager method, which can be used to temporarily disable capture within a test. Thanks @nicoddemus for the PR.
New cli flag --fixtures-per-test: shows which fixtures are being used for each selected test item. Features doc strings of fixtures by default. Can also show where fixtures are defined if combined with -v. Thanks @hackebrot for the PR.
Introduce pytest command as recommended entry point. Note that py.test still works and is not scheduled for removal. Closes proposal #1629. Thanks @obestwalter and @davehunt for the complete PR (#1633).
New cli flags:
Thanks @d6e, @kvas-it, @sallner, @ioggstream and @omarkohl for the PRs.
New CLI flag --override-ini/-o: overrides values from the ini file. For example: "-o xfail_strict=True"'. Thanks @blueyed and @fengxx for the PR.
New hooks:
Issue warnings for asserts whose test is a tuple literal. Such asserts will never fail because tuples are always truthy and are usually a mistake (see #1562). Thanks @kvas-it, for the PR.
Allow passing a custom debugger class (e.g. --pdbcls=IPython.core.debugger:Pdb). Thanks to @anntzer for the PR.
Changes
Bug Fixes
Bug Fixes
Bug Fixes
New Features
Changes
Important: py.code has been merged into the pytest repository as pytest._code. This decision was made because py.code had very few uses outside pytest and the fact that it was in a different repository made it difficult to fix bugs on its code in a timely manner. The team hopes with this to be able to better refactor out and improve that code. This change shouldn't affect users, but it is useful to let users aware if they encounter any strange behavior.
Keep in mind that the code for pytest._code is private and experimental, so you definitely should not import it explicitly!
Please note that the original py.code is still available in pylib.
pytest_enter_pdb now optionally receives the pytest config object. Thanks @nicoddemus for the PR.
Removed code and documentation for Python 2.5 or lower versions, including removal of the obsolete _pytest.assertion.oldinterpret module. Thanks @nicoddemus for the PR (#1226).
Comparisons now always show up in full when CI or BUILD_NUMBER is found in the environment, even when -vv isn't used. Thanks @The-Compiler for the PR.
--lf and --ff now support long names: --last-failed and --failed-first respectively. Thanks @MichaelAquilina for the PR.
Added expected exceptions to pytest.raises fail message.
Collection only displays progress ("collecting X items") when in a terminal. This avoids cluttering the output when using --color=yes to obtain colors in CI integrations systems (#1397).
Bug Fixes
new --lf and -ff options to run only the last failing tests or "failing tests first" from the last run. This functionality is provided through porting the formerly external pytest-cache plugin into pytest core. BACKWARD INCOMPAT: if you used pytest-cache's functionality to persist data between test runs be aware that we don't serialize sets anymore. Thanks Ronny Pfannschmidt for most of the merging work.
"-r" option now accepts "a" to include all possible reports, similar to passing "fEsxXw" explicitly (isse960). Thanks Abhijeet Kasurde for the PR.
avoid python3.5 deprecation warnings by introducing version specific inspection helpers, thanks Michael Droettboom.
fix issue562: @nose.tools.istest now fully respected.
fix issue934: when string comparison fails and a diff is too large to display without passing -vv, still show a few lines of the diff. Thanks Florian Bruhin for the report and Bruno Oliveira for the PR.
fix issue736: Fix a bug where fixture params would be discarded when combined with parametrization markers. Thanks to Markus Unterwaditzer for the PR.
fix issue710: introduce ALLOW_UNICODE doctest option: when enabled, the u prefix is stripped from unicode strings in expected doctest output. This allows doctests which use unicode to run in Python 2 and 3 unchanged. Thanks Jason R. Coombs for the report and Bruno Oliveira for the PR.
parametrize now also generates meaningful test IDs for enum, regex and class objects (as opposed to class instances). Thanks to Florian Bruhin for the PR.
Add 'warns' to assert that warnings are thrown (like 'raises'). Thanks to Eric Hunsberger for the PR.
Fix issue683: Do not apply an already applied mark. Thanks ojake for the PR.
Deal with capturing failures better so fewer exceptions get lost to /dev/null. Thanks David Szotten for the PR.
fix issue730: deprecate and warn about the --genscript option. Thanks Ronny Pfannschmidt for the report and Christian Pommranz for the PR.
fix issue751: multiple parametrize with ids bug if it parametrizes class with two or more test methods. Thanks Sergey Chipiga for reporting and Jan Bednarik for PR.
fix issue82: avoid loading conftest files from setup.cfg/pytest.ini/tox.ini files and upwards by default (--confcutdir can still be set to override this). Thanks Bruno Oliveira for the PR.
fix issue768: docstrings found in python modules were not setting up session fixtures. Thanks Jason R. Coombs for reporting and Bruno Oliveira for the PR.
added tmpdir_factory, a session-scoped fixture that can be used to create directories under the base temporary directory. Previously this object was installed as a _tmpdirhandler attribute of the config object, but now it is part of the official API and using config._tmpdirhandler is deprecated. Thanks Bruno Oliveira for the PR.
fix issue808: pytest's internal assertion rewrite hook now implements the optional PEP302 get_data API so tests can access data files next to them. Thanks xmo-odoo for request and example and Bruno Oliveira for the PR.
rootdir and inifile are now displayed during usage errors to help users diagnose problems such as unexpected ini files which add unknown options being picked up by pytest. Thanks to Pavel Savchenko for bringing the problem to attention in #821 and Bruno Oliveira for the PR.
Summary bar now is colored yellow for warning situations such as: all tests either were skipped or xpass/xfailed, or no tests were run at all (this is a partial fix for issue500).
fix issue812: pytest now exits with status code 5 in situations where no tests were run at all, such as the directory given in the command line does not contain any tests or as result of a command line option filters all out all tests (-k for example). Thanks Eric Siegerman (issue812) and Bruno Oliveira for the PR.
Summary bar now is colored yellow for warning situations such as: all tests either were skipped or xpass/xfailed, or no tests were run at all (related to issue500). Thanks Eric Siegerman.
New testpaths ini option: list of directories to search for tests when executing pytest from the root directory. This can be used to speed up test collection when a project has well specified directories for tests, being usually more practical than configuring norecursedirs for all directories that do not contain tests. Thanks to Adrian for idea (#694) and Bruno Oliveira for the PR.
fix issue713: JUnit XML reports for doctest failures. Thanks Punyashloka Biswal.
fix issue970: internal pytest warnings now appear as "pytest-warnings" in the terminal instead of "warnings", so it is clear for users that those warnings are from pytest and not from the builtin "warnings" module. Thanks Bruno Oliveira.
Include setup and teardown in junitxml test durations. Thanks Janne Vanhala.
fix issue735: assertion failures on debug versions of Python 3.4+
new option --import-mode to allow to change test module importing behaviour to append to sys.path instead of prepending. This better allows to run test modules against installed versions of a package even if the package under test has the same import root. In this example:
testing/__init__.py testing/test_pkg_under_test.py pkg_under_test/
the tests will run against the installed version of pkg_under_test when --import-mode=append is used whereas by default they would always pick up the local version. Thanks Holger Krekel.
pytester: add method TmpTestdir.delete_loaded_modules(), and call it from inline_run() to allow temporary modules to be reloaded. Thanks Eduardo Schettino.
internally refactor pluginmanager API and code so that there is a clear distinction between a pytest-agnostic rather simple pluginmanager and the PytestPluginManager which adds a lot of behaviour, among it handling of the local conftest files. In terms of documented methods this is a backward compatible change but it might still break 3rd party plugins which relied on details like especially the pluginmanager.add_shutdown() API. Thanks Holger Krekel.
pluginmanagement: introduce pytest.hookimpl and pytest.hookspec decorators for setting impl/spec specific parameters. This substitutes the previous now deprecated use of pytest.mark which is meant to contain markers for test functions only.
write/refine docs for "writing plugins" which now have their own page and are separate from the "using/installing plugins`` page.
fix issue732: properly unregister plugins from any hook calling sites allowing to have temporary plugins during test execution.
deprecate and warn about __multicall__ argument in hook implementations. Use the hookwrapper mechanism instead already introduced with pytest-2.7.
speed up pytest's own test suite considerably by using inprocess tests by default (testrun can be modified with --runpytest=subprocess to create subprocesses in many places instead). The main APIs to run pytest in a test is "runpytest()" or "runpytest_subprocess" and "runpytest_inprocess" if you need a particular way of running the test. In all cases you get back a RunResult but the inprocess one will also have a "reprec" attribute with the recorded events/reports.
fix monkeypatch.setattr("x.y", raising=False) to actually not raise if "y" is not a pre-existing attribute. Thanks Florian Bruhin.
fix issue741: make running output from testdir.run copy/pasteable Thanks Bruno Oliveira.
add a new --noconftest argument which ignores all conftest.py files.
add file and line attributes to JUnit-XML output.
fix issue890: changed extension of all documentation files from txt to rst. Thanks to Abhijeet for the PR.
fix issue714: add ability to apply indirect=True parameter on particular argnames. Thanks Elizaveta239.
fix issue890: changed extension of all documentation files from txt to rst. Thanks to Abhijeet for the PR.
fix issue957: "# doctest: SKIP" option will now register doctests as SKIPPED rather than PASSED. Thanks Thomas Grainger for the report and Bruno Oliveira for the PR.
issue951: add new record_xml_property fixture, that supports logging additional information on xml output. Thanks David Diaz for the PR.
issue949: paths after normal options (for example -s, -v, etc) are now properly used to discover rootdir and ini files. Thanks Peter Lauri for the report and Bruno Oliveira for the PR.
dropped python2.5 from automated release testing of pytest itself which means it's probably going to break soon (but still works with this release we believe).
simplified and fixed implementation for calling finalizers when parametrized fixtures or function arguments are involved. finalization is now performed lazily at setup time instead of in the "teardown phase". While this might sound odd at first, it helps to ensure that we are correctly handling setup/teardown even in complex code. User-level code should not be affected unless it's implementing the pytest_runtest_teardown hook and expecting certain fixture instances are torn down within (very unlikely and would have been unreliable anyway).
PR90: add --color=yes|no|auto option to force terminal coloring mode ("auto" is default). Thanks Marc Abramowitz.
fix issue319 - correctly show unicode in assertion errors. Many thanks to Floris Bruynooghe for the complete PR. Also means we depend on py>=1.4.19 now.
fix issue396 - correctly sort and finalize class-scoped parametrized tests independently from number of methods on the class.
refix issue323 in a better way -- parametrization should now never cause Runtime Recursion errors because the underlying algorithm for re-ordering tests per-scope/per-fixture is not recursive anymore (it was tail-call recursive before which could lead to problems for more than >966 non-function scoped parameters).
fix issue290 - there is preliminary support now for parametrizing with repeated same values (sometimes useful to test if calling a second time works as with the first time).
close issue240 - document precisely how pytest module importing works, discuss the two common test directory layouts, and how it interacts with PEP420-namespace packages.
fix issue246 fix finalizer order to be LIFO on independent fixtures depending on a parametrized higher-than-function scoped fixture. (was quite some effort so please bear with the complexity of this sentence :) Thanks Ralph Schmitt for the precise failure example.
fix issue244 by implementing special index for parameters to only use indices for paramentrized test ids
fix issue287 by running all finalizers but saving the exception from the first failing finalizer and re-raising it so teardown will still have failed. We reraise the first failing exception because it might be the cause for other finalizers to fail.
fix ordering when mock.patch or other standard decorator-wrappings are used with test methods. This fixues issue346 and should help with random "xdist" collection failures. Thanks to Ronny Pfannschmidt and Donald Stufft for helping to isolate it.
fix issue357 - special case "-k" expressions to allow for filtering with simple strings that are not valid python expressions. Examples: "-k 1.3" matches all tests parametrized with 1.3. "-k None" filters all tests that have "None" in their name and conversely "-k 'not None'". Previously these examples would raise syntax errors.
fix issue384 by removing the trial support code since the unittest compat enhancements allow trial to handle it on its own
don't hide an ImportError when importing a plugin produces one. fixes issue375.
fix issue275 - allow usefixtures and autouse fixtures for running doctest text files.
fix issue380 by making --resultlog only rely on longrepr instead of the "reprcrash" attribute which only exists sometimes.
address issue122: allow @pytest.fixture(params=iterator) by exploding into a list early on.
fix pexpect-3.0 compatibility for pytest's own tests. (fixes issue386)
allow nested parametrize-value markers, thanks James Lan for the PR.
fix unicode handling with new monkeypatch.setattr(import_path, value) API. Thanks Rob Dennis. Fixes issue371.
fix unicode handling with junitxml, fixes issue368.
In assertion rewriting mode on Python 2, fix the detection of coding cookies. See issue #330.
make "--runxfail" turn imperative pytest.xfail calls into no ops (it already did neutralize pytest.mark.xfail markers)
refine pytest / pkg_resources interactions: The AssertionRewritingHook PEP302 compliant loader now registers itself with setuptools/pkg_resources properly so that the pkg_resources.resource_stream method works properly. Fixes issue366. Thanks for the investigations and full PR to Jason R. Coombs.
pytestconfig fixture is now session-scoped as it is the same object during the whole test run. Fixes issue370.
avoid one surprising case of marker malfunction/confusion:
@pytest.mark.some(lambda arg: ...) def test_function():
would not work correctly because pytest assumes @pytest.mark.some gets a function to be decorated already. We now at least detect if this arg is an lambda and thus the example will work. Thanks Alex Gaynor for bringing it up.
xfail a test on pypy that checks wrong encoding/ascii (pypy does not error out). fixes issue385.
internally make varnames() deal with classes's __init__, although it's not needed by pytest itself atm. Also fix caching. Fixes issue376.
fix issue221 - handle importing of namespace-package with no __init__.py properly.
refactor internal FixtureRequest handling to avoid monkeypatching. One of the positive user-facing effects is that the "request" object can now be used in closures.
fixed version comparison in pytest.importskip(modname, minverstring)
fix issue377 by clarifying in the nose-compat docs that pytest does not duplicate the unittest-API into the "plain" namespace.
fix verbose reporting for @mock'd test functions
known incompatibilities:
new features:
experimentally introduce a new pytest.yield_fixture decorator which accepts exactly the same parameters as pytest.fixture but mandates a yield statement instead of a return statement from fixture functions. This allows direct integration with "with-style" context managers in fixture functions and generally avoids registering of finalization callbacks in favour of treating the "after-yield" as teardown code. Thanks Andreas Pelme, Vladimir Keleshev, Floris Bruynooghe, Ronny Pfannschmidt and many others for discussions.
allow boolean expression directly with skipif/xfail if a "reason" is also specified. Rework skipping documentation to recommend "condition as booleans" because it prevents surprises when importing markers between modules. Specifying conditions as strings will remain fully supported.
reporting: color the last line red or green depending if failures/errors occurred or everything passed. thanks Christian Theunert.
make "import pdb ; pdb.set_trace()" work natively wrt capturing (no "-s" needed anymore), making pytest.set_trace() a mere shortcut.
fix issue181: --pdb now also works on collect errors (and on internal errors) . This was implemented by a slight internal refactoring and the introduction of a new hook pytest_exception_interact hook (see next item).
fix issue341: introduce new experimental hook for IDEs/terminals to intercept debugging: pytest_exception_interact(node, call, report).
new monkeypatch.setattr() variant to provide a shorter invocation for patching out classes/functions from modules:
monkeypatch.setattr("requests.get", myfunc)
will replace the "get" function of the "requests" module with myfunc.
fix issue322: tearDownClass is not run if setUpClass failed. Thanks Mathieu Agopian for the initial fix. Also make all of pytest/nose finalizer mimic the same generic behaviour: if a setupX exists and fails, don't run teardownX. This internally introduces a new method "node.addfinalizer()" helper which can only be called during the setup phase of a node.
simplify pytest.mark.parametrize() signature: allow to pass a CSV-separated string to specify argnames. For example: pytest.mark.parametrize("input,expected", [(1,2), (2,3)]) works as well as the previous: pytest.mark.parametrize(("input", "expected"), ...).
add support for setUpModule/tearDownModule detection, thanks Brian Okken.
integrate tab-completion on options through use of "argcomplete". Thanks Anthon van der Neut for the PR.
change option names to be hyphen-separated long options but keep the old spelling backward compatible. py.test -h will only show the hyphenated version, for example "--collect-only" but "--collectonly" will remain valid as well (for backward-compat reasons). Many thanks to Anthon van der Neut for the implementation and to Hynek Schlawack for pushing us.
fix issue 308 - allow to mark/xfail/skip individual parameter sets when parametrizing. Thanks Brianna Laugher.
call new experimental pytest_load_initial_conftests hook to allow 3rd party plugins to do something before a conftest is loaded.
Bug fixes:
fix issue202 - better automatic names for parametrized test functions
fix issue139 - introduce @pytest.fixture which allows direct scoping and parametrization of funcarg factories.
fix issue198 - conftest fixtures were not found on windows32 in some circumstances with nested directory structures due to path manipulation issues
fix issue193 skip test functions with were parametrized with empty parameter sets
fix python3.3 compat, mostly reporting bits that previously depended on dict ordering
introduce re-ordering of tests by resource and parametrization setup which takes precedence to the usual file-ordering
fix issue185 monkeypatching time.time does not cause pytest to fail
fix issue172 duplicate call of pytest.fixture decoratored setup_module functions
fix junitxml=path construction so that if tests change the current working directory and the path is a relative path it is constructed correctly from the original current working dir.
fix "python setup.py test" example to cause a proper "errno" return
fix issue165 - fix broken doc links and mention stackoverflow for FAQ
catch unicode-issues when writing failure representations to terminal to prevent the whole session from crashing
fix xfail/skip confusion: a skip-mark or an imperative pytest.skip will now take precedence before xfail-markers because we can't determine xfail/xpass status in case of a skip. see also: http://stackoverflow.com/questions/11105828/in-py-test-when-i-explicitly-skip-a-test-that-is-marked-as-xfail-how-can-i-get
always report installed 3rd party plugins in the header of a test run
fix issue160: a failing setup of an xfail-marked tests should be reported as xfail (not xpass)
fix issue128: show captured output when capsys/capfd are used
fix issue179: properly show the dependency chain of factories
pluginmanager.register(...) now raises ValueError if the plugin has been already registered or the name is taken
fix issue159: improve http://pytest.org/latest/faq.html especially with respect to the "magic" history, also mention pytest-django, trial and unittest integration.
make request.keywords and node.keywords writable. All descendant collection nodes will see keyword values. Keywords are dictionaries containing markers and other info.
fix issue 178: xml binary escapes are now wrapped in py.xml.raw
fix issue 176: correctly catch the builtin AssertionError even when we replaced AssertionError with a subclass on the python level
factory discovery no longer fails with magic global callables that provide no sane __code__ object (mock.call for example)
fix issue 182: testdir.inprocess_run now considers passed plugins
before calling into a test
fix issue 191: add unittest TestCase runTest method support
fix issue 156: monkeypatch correctly handles class level descriptors
reporting refinements:
tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead
fix issue30 - extended xfail/skipif handling and improved reporting. If you have a syntax error in your skip/xfail expressions you now get nice error reports.
Also you can now access module globals from xfail/skipif expressions so that this for example works now:
import pytest
import mymodule
@pytest.mark.skipif("mymodule.__version__[0] == "1")
def test_function():
pass
This will not run the test function if the module's version string does not start with a "1". Note that specifying a string instead of a boolean expressions allows py.test to report meaningful information when summarizing a test run as to what conditions lead to skipping (or xfail-ing) tests.
fix issue28 - setup_method and pytest_generate_tests work together The setup_method fixture method now gets called also for test function invocations generated from the pytest_generate_tests hook.
fix issue27 - collectonly and keyword-selection (-k) now work together Also, if you do "py.test --collectonly -q" you now get a flat list of test ids that you can use to paste to the py.test commandline in order to execute a particular test.
fix issue25 avoid reported problems with --pdb and python3.2/encodings output
fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP Starting with Python3.2 os.symlink may be supported. By requiring a newer py lib version the py.path.local() implementation acknowledges this.
fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular thanks to Laura Creighton who also reviewed parts of the documentation.
fix slightly wrong output of verbose progress reporting for classes (thanks Amaury)
more precise (avoiding of) deprecation warnings for node.Class|Function accesses
avoid std unittest assertion helper code in tracebacks (thanks Ronny)
New features
fix issue103: introduce py.test.raises as context manager, examples:
with py.test.raises(ZeroDivisionError):
x = 0
1 / x
with py.test.raises(RuntimeError) as excinfo:
call_something()
# you may do extra checks on excinfo.value|type|traceback here
(thanks Ronny Pfannschmidt)
Funcarg factories can now dynamically apply a marker to a test invocation. This is for example useful if a factory provides parameters to a test which are expected-to-fail:
def pytest_funcarg__arg(request):
request.applymarker(py.test.mark.xfail(reason="flaky config"))
...
def test_function(arg):
...
improved error reporting on collection and import errors. This makes use of a more general mechanism, namely that for custom test item/collect nodes node.repr_failure(excinfo) is now uniformly called so that you can override it to return a string error representation of your choice which is going to be reported as a (red) string.
introduce '--junitprefix=STR' option to prepend a prefix to all reports in the junitxml file.
Bug fixes
New features
issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain configurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all configurations as xfail.
issue102: introduce new --maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the '-x' or '--exitfirst' option which is now equivalent to '--maxfail=1'. Both '-x' and '--maxfail' will now also print a line near the end indicating the Interruption.
issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list.
improve and refine letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed)
You can use any combination of 'fsxX' with the '-r' extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also fixes issue99.
make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and don't have global state.
Bug Fixes
deprecate --report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of 'xfsX' characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test.
allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature.
add a new pytest_ignore_collect(path, config) hook to allow projects and plugins to define exclusion behaviour for their directory structure - for example you may define in a conftest.py this method:
def pytest_ignore_collect(path):
return path.check(link=1)
to prevent even a collection try of any tests in symlinked dirs.
new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module.
extend and refine xfail mechanism: @py.test.mark.xfail(run=False) do not run the decorated test @py.test.mark.xfail(reason="...") prints the reason string in xfail summaries specifying --runxfail on command line virtually ignores xfail markers
expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(...) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string
expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking
(issue85) fix junitxml plugin to handle tests with non-ascii output
fix/refine python3 compatibility (thanks Benjamin Peterson)
fixes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see http://bugs.jython.org/issue1491 . See pylib install documentation for how to work around.
fixes for handling of unicode exception values and unprintable objects
(issue87) fix unboundlocal error in assertionold code
(issue86) improve documentation for looponfailing
refine IO capturing: stdin-redirect pseudo-file now has a NOP close() method
ship distribute_setup.py version 0.6.10
added links to the new capturelog and coverage plugins
refined usage and options for "py.cleanup":
py.cleanup # remove "*.pyc" and "*$py.class" (jython) files py.cleanup -e .swp -e .cache # also remove files with these extensions py.cleanup -s # remove "build" and "dist" directory next to setup.py files py.cleanup -d # also remove empty directories py.cleanup -a # synonym for "-s -d -e 'pip-log.txt'" py.cleanup -n # dry run, only show what would be removed
add a new option "py.test --funcargs" which shows available funcargs and their help strings (docstrings on their respective factory function) for a given test path
display a short and concise traceback if a funcarg lookup fails
early-load "conftest.py" files in non-dot first-level sub directories. allows to conveniently keep and access test-related options in a test subdir and still add command line options.
fix issue67: new super-short traceback-printing option: "--tb=line" will print a single line for each failing (python) test indicating its filename, lineno and the failure value
fix issue78: always call python-level teardown functions even if the according setup failed. This includes refinements for calling setup_module/class functions which will now only be called once instead of the previous behaviour where they'd be called multiple times if they raise an exception (including a Skipped exception). Any exception will be re-corded and associated with all tests in the according module/class scope.
fix issue63: assume <40 columns to be a bogus terminal width, default to 80
fix pdb debugging to be in the correct frame on raises-related errors
update apipkg.py to fix an issue where recursive imports might unnecessarily break importing
fix plugin links
XXX lots of things missing here XXX
This is a fairly complete list of v0.9.1, which can serve as a reference for developers.