summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/tools/pytest/doc/en/customize.rst
diff options
context:
space:
mode:
Diffstat (limited to 'testing/web-platform/tests/tools/pytest/doc/en/customize.rst')
-rw-r--r--testing/web-platform/tests/tools/pytest/doc/en/customize.rst228
1 files changed, 228 insertions, 0 deletions
diff --git a/testing/web-platform/tests/tools/pytest/doc/en/customize.rst b/testing/web-platform/tests/tools/pytest/doc/en/customize.rst
new file mode 100644
index 000000000..34e319c24
--- /dev/null
+++ b/testing/web-platform/tests/tools/pytest/doc/en/customize.rst
@@ -0,0 +1,228 @@
+Basic test configuration
+===================================
+
+Command line options and configuration file settings
+-----------------------------------------------------------------
+
+You can get help on command line options and values in INI-style
+configurations files by using the general help option::
+
+ py.test -h # prints options _and_ config file settings
+
+This will display command line and configuration file settings
+which were registered by installed plugins.
+
+.. _rootdir:
+.. _inifiles:
+
+initialization: determining rootdir and inifile
+-----------------------------------------------
+
+.. versionadded:: 2.7
+
+pytest determines a "rootdir" for each test run which depends on
+the command line arguments (specified test files, paths) and on
+the existence of inifiles. The determined rootdir and ini-file are
+printed as part of the pytest header. The rootdir is used for constructing
+"nodeids" during collection and may also be used by plugins to store
+project/testrun-specific information.
+
+Here is the algorithm which finds the rootdir from ``args``:
+
+- determine the common ancestor directory for the specified ``args``.
+
+- look for ``pytest.ini``, ``tox.ini`` and ``setup.cfg`` files in the
+ ancestor directory and upwards. If one is matched, it becomes the
+ ini-file and its directory becomes the rootdir. An existing
+ ``pytest.ini`` file will always be considered a match whereas
+ ``tox.ini`` and ``setup.cfg`` will only match if they contain
+ a ``[pytest]`` section.
+
+- if no ini-file was found, look for ``setup.py`` upwards from
+ the common ancestor directory to determine the ``rootdir``.
+
+- if no ini-file and no ``setup.py`` was found, use the already
+ determined common ancestor as root directory. This allows to
+ work with pytest in structures that are not part of a package
+ and don't have any particular ini-file configuration.
+
+Note that options from multiple ini-files candidates are never merged,
+the first one wins (``pytest.ini`` always wins even if it does not
+contain a ``[pytest]`` section).
+
+The ``config`` object will subsequently carry these attributes:
+
+- ``config.rootdir``: the determined root directory, guaranteed to exist.
+
+- ``config.inifile``: the determined ini-file, may be ``None``.
+
+The rootdir is used a reference directory for constructing test
+addresses ("nodeids") and can be used also by plugins for storing
+per-testrun information.
+
+Example::
+
+ py.test path/to/testdir path/other/
+
+will determine the common ancestor as ``path`` and then
+check for ini-files as follows::
+
+ # first look for pytest.ini files
+ path/pytest.ini
+ path/setup.cfg # must also contain [pytest] section to match
+ path/tox.ini # must also contain [pytest] section to match
+ pytest.ini
+ ... # all the way down to the root
+
+ # now look for setup.py
+ path/setup.py
+ setup.py
+ ... # all the way down to the root
+
+
+.. _`how to change command line options defaults`:
+.. _`adding default options`:
+
+How to change command line options defaults
+------------------------------------------------
+
+It can be tedious to type the same series of command line options
+every time you use ``pytest``. For example, if you always want to see
+detailed info on skipped and xfailed tests, as well as have terser "dot"
+progress output, you can write it into a configuration file:
+
+.. code-block:: ini
+
+ # content of pytest.ini
+ # (or tox.ini or setup.cfg)
+ [pytest]
+ addopts = -rsxX -q
+
+Alternatively, you can set a PYTEST_ADDOPTS environment variable to add command
+line options while the environment is in use::
+
+ export PYTEST_ADDOPTS="-rsxX -q"
+
+From now on, running ``pytest`` will add the specified options.
+
+
+
+Builtin configuration file options
+----------------------------------------------
+
+.. confval:: minversion
+
+ Specifies a minimal pytest version required for running tests.
+
+ minversion = 2.1 # will fail if we run with pytest-2.0
+
+.. confval:: addopts
+
+ Add the specified ``OPTS`` to the set of command line arguments as if they
+ had been specified by the user. Example: if you have this ini file content:
+
+ .. code-block:: ini
+
+ [pytest]
+ addopts = --maxfail=2 -rf # exit after 2 failures, report fail info
+
+ issuing ``py.test test_hello.py`` actually means::
+
+ py.test --maxfail=2 -rf test_hello.py
+
+ Default is to add no options.
+
+.. confval:: norecursedirs
+
+ Set the directory basename patterns to avoid when recursing
+ for test discovery. The individual (fnmatch-style) patterns are
+ applied to the basename of a directory to decide if to recurse into it.
+ Pattern matching characters::
+
+ * matches everything
+ ? matches any single character
+ [seq] matches any character in seq
+ [!seq] matches any char not in seq
+
+ Default patterns are ``'.*', 'CVS', '_darcs', '{arch}', '*.egg'``.
+ Setting a ``norecursedirs`` replaces the default. Here is an example of
+ how to avoid certain directories:
+
+ .. code-block:: ini
+
+ # content of setup.cfg
+ [pytest]
+ norecursedirs = .svn _build tmp*
+
+ This would tell ``pytest`` to not look into typical subversion or
+ sphinx-build directories or into any ``tmp`` prefixed directory.
+
+.. confval:: testpaths
+
+ .. versionadded:: 2.8
+
+ Sets list of directories that should be searched for tests when
+ no specific directories, files or test ids are given in the command line when
+ executing pytest from the :ref:`rootdir <rootdir>` directory.
+ Useful when all project tests are in a known location to speed up
+ test collection and to avoid picking up undesired tests by accident.
+
+ .. code-block:: ini
+
+ # content of pytest.ini
+ [pytest]
+ testpaths = testing doc
+
+ This tells pytest to only look for tests in ``testing`` and ``doc``
+ directories when executing from the root directory.
+
+.. confval:: python_files
+
+ One or more Glob-style file patterns determining which python files
+ are considered as test modules.
+
+.. confval:: python_classes
+
+ One or more name prefixes or glob-style patterns determining which classes
+ are considered for test collection. Here is an example of how to collect
+ tests from classes that end in ``Suite``:
+
+ .. code-block:: ini
+
+ # content of pytest.ini
+ [pytest]
+ python_classes = *Suite
+
+ Note that ``unittest.TestCase`` derived classes are always collected
+ regardless of this option, as ``unittest``'s own collection framework is used
+ to collect those tests.
+
+.. confval:: python_functions
+
+ One or more name prefixes or glob-patterns determining which test functions
+ and methods are considered tests. Here is an example of how
+ to collect test functions and methods that end in ``_test``:
+
+ .. code-block:: ini
+
+ # content of pytest.ini
+ [pytest]
+ python_functions = *_test
+
+ Note that this has no effect on methods that live on a ``unittest
+ .TestCase`` derived class, as ``unittest``'s own collection framework is used
+ to collect those tests.
+
+ See :ref:`change naming conventions` for more detailed examples.
+
+.. confval:: doctest_optionflags
+
+ One or more doctest flag names from the standard ``doctest`` module.
+ :doc:`See how py.test handles doctests <doctest>`.
+
+.. confval:: confcutdir
+
+ Sets a directory where search upwards for ``conftest.py`` files stops.
+ By default, pytest will stop searching for ``conftest.py`` files upwards
+ from ``pytest.ini``/``tox.ini``/``setup.cfg`` of the project if any,
+ or up to the file-system root.