summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/tools/pytest/doc/en/example/nonpython.rst
diff options
context:
space:
mode:
Diffstat (limited to 'testing/web-platform/tests/tools/pytest/doc/en/example/nonpython.rst')
-rw-r--r--testing/web-platform/tests/tools/pytest/doc/en/example/nonpython.rst91
1 files changed, 91 insertions, 0 deletions
diff --git a/testing/web-platform/tests/tools/pytest/doc/en/example/nonpython.rst b/testing/web-platform/tests/tools/pytest/doc/en/example/nonpython.rst
new file mode 100644
index 000000000..6437e3984
--- /dev/null
+++ b/testing/web-platform/tests/tools/pytest/doc/en/example/nonpython.rst
@@ -0,0 +1,91 @@
+
+.. _`non-python tests`:
+
+Working with non-python tests
+====================================================
+
+.. _`yaml plugin`:
+
+A basic example for specifying tests in Yaml files
+--------------------------------------------------------------
+
+.. _`pytest-yamlwsgi`: http://bitbucket.org/aafshar/pytest-yamlwsgi/src/tip/pytest_yamlwsgi.py
+.. _`PyYAML`: http://pypi.python.org/pypi/PyYAML/
+
+Here is an example ``conftest.py`` (extracted from Ali Afshnars special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yml`` files and will execute the yaml-formatted content as custom tests:
+
+.. include:: nonpython/conftest.py
+ :literal:
+
+You can create a simple example file:
+
+.. include:: nonpython/test_simple.yml
+ :literal:
+
+and if you installed `PyYAML`_ or a compatible YAML-parser you can
+now execute the test specification::
+
+ nonpython $ py.test test_simple.yml
+ ======= test session starts ========
+ platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
+ rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
+ collected 2 items
+
+ test_simple.yml F.
+
+ ======= FAILURES ========
+ _______ usecase: hello ________
+ usecase execution failed
+ spec failed: 'some': 'other'
+ no further details known at this point.
+ ======= 1 failed, 1 passed in 0.12 seconds ========
+
+.. regendoc:wipe
+
+You get one dot for the passing ``sub1: sub1`` check and one failure.
+Obviously in the above ``conftest.py`` you'll want to implement a more
+interesting interpretation of the yaml-values. You can easily write
+your own domain specific testing language this way.
+
+.. note::
+
+ ``repr_failure(excinfo)`` is called for representing test failures.
+ If you create custom collection nodes you can return an error
+ representation string of your choice. It
+ will be reported as a (red) string.
+
+``reportinfo()`` is used for representing the test location and is also
+consulted when reporting in ``verbose`` mode::
+
+ nonpython $ py.test -v
+ ======= test session starts ========
+ platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- $PYTHON_PREFIX/bin/python3.4
+ cachedir: .cache
+ rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
+ collecting ... collected 2 items
+
+ test_simple.yml::hello FAILED
+ test_simple.yml::ok PASSED
+
+ ======= FAILURES ========
+ _______ usecase: hello ________
+ usecase execution failed
+ spec failed: 'some': 'other'
+ no further details known at this point.
+ ======= 1 failed, 1 passed in 0.12 seconds ========
+
+.. regendoc:wipe
+
+While developing your custom test collection and execution it's also
+interesting to just look at the collection tree::
+
+ nonpython $ py.test --collect-only
+ ======= test session starts ========
+ platform linux -- Python 3.4.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
+ rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
+ collected 2 items
+ <YamlFile 'test_simple.yml'>
+ <YamlItem 'hello'>
+ <YamlItem 'ok'>
+
+ ======= no tests ran in 0.12 seconds ========