summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/docs/review-checklist.md
diff options
context:
space:
mode:
authorMatt A. Tobin <mattatobin@localhost.localdomain>2018-02-02 04:16:08 -0500
committerMatt A. Tobin <mattatobin@localhost.localdomain>2018-02-02 04:16:08 -0500
commit5f8de423f190bbb79a62f804151bc24824fa32d8 (patch)
tree10027f336435511475e392454359edea8e25895d /testing/web-platform/tests/docs/review-checklist.md
parent49ee0794b5d912db1f95dce6eb52d781dc210db5 (diff)
downloadUXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar.gz
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar.lz
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar.xz
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.zip
Add m-esr52 at 52.6.0
Diffstat (limited to 'testing/web-platform/tests/docs/review-checklist.md')
-rw-r--r--testing/web-platform/tests/docs/review-checklist.md128
1 files changed, 128 insertions, 0 deletions
diff --git a/testing/web-platform/tests/docs/review-checklist.md b/testing/web-platform/tests/docs/review-checklist.md
new file mode 100644
index 000000000..70ffb81bc
--- /dev/null
+++ b/testing/web-platform/tests/docs/review-checklist.md
@@ -0,0 +1,128 @@
+When reviewing a test, make sure the test follows the
+[format][format] and [style][style] guidelines.
+
+In addition, the test should be checked for the following:
+
+## All tests
+<input type="checkbox">
+The test passes when it's supposed to pass
+
+<input type="checkbox">
+The test fails when it's supposed to fail
+
+<input type="checkbox">
+The test is testing what it thinks it's testing
+
+<input type="checkbox">
+The spec backs up the expected behavior in the test.
+
+<input type="checkbox">
+The test is automated as either [reftest][reftest] or a
+[script test][scripttest] unless there's a very good reason why the
+test must be manual.
+
+<input type="checkbox">
+The test does not use external resources.
+
+<input type="checkbox">
+The test does not use proprietary features (vendor-prefixed or otherwise).
+
+
+## Reftests Only
+<input type="checkbox">
+The test has a [self-describing][selftest] statement
+
+<input type="checkbox">
+The self-describing statement is accurate, precise, simple, and
+self-explanatory. Your mother/husband/roommate/brother/bus driver
+should be able to say whether the test passed or failed within a few
+seconds, and not need to spend several minutes thinking or asking
+questions.
+
+<input type="checkbox">
+The reference file is accurate and will render pixel-perfect
+identically to the test on all platforms.
+
+<input type="checkbox">
+The reference file uses a different technique that won't fail in
+the same way as the test.
+
+<input type="checkbox">
+The title is descriptive but not too wordy.
+
+<input type="checkbox">
+The test is as cross-platform as reasonably possible, working
+across different devices, screen resolutions, paper sizes, etc. If
+there are limitations (e.g. the test will only work on 96dpi
+devices, or screens wider than 200 pixels) that these are documented
+in the instructions.
+
+
+## Script Tests Only
+
+<input type="checkbox">
+The test uses the most specific asserts possible (e.g. doesn't use
+`assert_true` for everything).
+
+<input type="checkbox">
+The number of tests in each file and the test names are consistent
+across runs and browsers. It is best to avoid the pattern where there is
+a test that asserts that the feature is supported and bails out without
+running the rest of the tests in the file if it isn't.
+
+<input type="checkbox">
+The test avoids patterns that make it less likely to be stable.
+In particular, tests should avoid setting internal timeouts, since the
+time taken to run it may vary on different devices; events should be used
+instead (if at all possible).
+
+<input type="checkbox">
+The test uses `idlharness.js` if it covers the use case.
+
+<input type="checkbox">
+Tests in a single file are separated by one empty line.
+
+
+## In-depth Checklist
+
+<input type="checkbox">
+A test does not use self-closing start tag ("/" (U+002F)) when using the
+HTML syntax.
+
+<input type="checkbox">
+The test does not use the Unicode byte order mark (BOM U+FEFF). The test
+uses Unix line endings (LF, no CR). The executable bit is not set
+unnecessarily.
+
+<input type="checkbox">
+For indentation, spaces are preferred over tabs.
+
+<input type="checkbox">
+The test does not contain trailing whitespace (whitespace at the end of
+lines).
+
+<input type="checkbox">
+The test does not contain commented-out code.
+
+<input type="checkbox">
+The test does not use `console.*` methods for anything. The
+[script test][scripttest] harness never relies on `console.*` methods in
+any way, and so use of `console.*` methods in tests is usually just the
+equivalent of extra `printf`s in production code; i.e., leftover debugging
+that isn't actually useful to the next person running the test. It also
+introduces useless overhead when running tests in automation.
+
+<input type="checkbox">
+The test is placed in the relevant directory, based on the /TR latest
+version link if available.
+
+<input type="checkbox">
+If the test needs code running on the server side, the server code must
+be written in python, and the python code must be reviewed carefully to
+ensure it isn't doing anything dangerous.
+
+[format]: ./test-format-guidelines.html
+[style]: ./test-style-guidelines.html
+[reftest]: ./reftests.html
+[scripttest]: ./testharness-documentation.html
+[selftest]: ./test-style-guidelines.html#self-describing