Add m-esr52 at 52.6.0

1 files changed, 381 insertions, 0 deletions

diff --git a/testing/web-platform/tests/annotation-model/CONTRIBUTING.md b/testing/web-platform/tests/annotation-model/CONTRIBUTING.md
new file mode 100644
index 000000000..f591d2a97
--- /dev/null
+++ b/testing/web-platform/tests/annotation-model/CONTRIBUTING.md
@@ -0,0 +1,381 @@
+Annotation-model: Guidelines for Contributing Tests
+===================================================
+
+This document describes the method people should use for authoring tests and
+integrating them into the repository.  Anyone is welcome to submit new tests to
+this collection.  If you do, please create the tests following the guidelines
+below.  Then submit them as a pull request so they can be evaluated
+
+Structure
+---------
+
+Tests are organized by major section of the Annotation Model specification.  The
+folders associated with these are:
+
+* annotations
+* bodiesTargets
+* collections
+* specificResources
+  * selectors
+  * states
+
+Within these folders, special files ending with the suffix ".test" provide the source
+for the test as a set of declarative assertions about the required shape of the conforming
+JSON object.  These files are transformed using a test generation tool into ".html" files
+that are then accessed by the Web Platform Test framework.
+
+There are a few other folders that provide supporting materials for the tests:
+
+* common - assertionObjects, conditionObjects, and other supporting materials
+* definitions - JSON Schema definitions that can be referenced
+* scripts - JavaScript that are included by tests
+* tools - supporting scripts and files
+
+NOTE: The files in the definitions folder are expected to be JSON Schema
+definitions - basically commonly used concepts that are referenced by other JSON
+Schema files in the system.  All of these 'definitions' are preloaded by the
+system before any other parts of a test are processed.
+
+Test Cases
+----------
+
+Each test is expressed as a simple (or complex) requirement in a test file.
+For each section of the document, the requirement is represented as a structure
+that describes the nature of the test, and then includes or references minimal
+JSON Schema that test the assertions implied by the requirement.
+
+The structure of a test case is defined using a [JSON-LD
+Context](JSONtest-v1.jsonld).  That context defines the following terms:
+
+|Keyword        | Values          | Meaning
+|---------------|-----------------|---------
+|name           | string          | The name of this test for display purposes
+|description    | string          | A long self-describing paragraph that explains the purpose of the test and the expected input
+|ref            | URI             | An optional reference to the portion of the specification to which the test relates
+|testType       | `automated`, `manual`, `ref` | The type of test - this informs [WPT](https://github.com/w3c/web-platform-tests) how the test should be controlled and presented
+|skipFailures   | list of strings | An optional list of assertionType values that, if present, should have their test skipped if the result would be "unexpected".  Defaults to the empty list.
+|assertions     | list of URI, List @@@ATRISK@@@, or AssertionObject | The ordered collection of tests the input should be run against. See [JSON Schema Usage](#jsonSchema) for the structure of the objects.  URI is relative to the top level folder of the test collection if it has a slash; relative to the current directory if it does not. @@@@ATRISK@@@@ Lists can be nested to define groups of sub-tests.  Assertions / groups can be conditionally skipped.  See [Assertion Lists](#assertionLists) for more details.
+|content        | URI or object   | An object containing content to be checked against the referenced assertions, or a URI from which to retrieve that content
+
+Each test case has a suffix of `.test` and a shape like:
+
+<pre>
+{
+  "@context": "https://www.w3.org/ns/JSONtest-v1.jsonld",
+  "name": "Verify annotation conforms to the model",
+  "description": "Supply an example annotation that conforms to the basic structure.",
+  "ref": "https://www.w3.org/TR/annotation-model/#model",
+  "testType": "manual",
+  "assertions": [
+    "common/has_context.json",
+    "common/has_id.json",
+    {
+      "$schema": "http://json-schema.org/draft-04/schema#",
+      "title": "Verify annotation has target",
+      "assertionType": "must",
+      "expectedResult": "valid",
+      "errorMessage": "The object was missing a required 'target' property",
+      "type": "object",
+      "properties": {
+        "target": {
+          "anyOf": [
+          {
+            "type": "string"
+          },
+          {
+            "type": "array",
+            "anyOf": [
+            {
+              "type": "string"
+            }
+            ]
+          }
+          ],
+            "not": {"type": "object"}
+        }
+      },
+      "required": ["target"]
+    }
+  ]
+}
+</pre>
+
+External references are used when the "assertion" is a common one that needs to
+be checked on many different test cases (e.g., that there is an @context in the
+supplied annotation).
+
+NOTE: The title property of an assertionObject can contain markdown.  This can
+help improve readability of the rendered assertions and debugging output.
+
+NOTE: The content property does not yet have a defined use.  One potential use would
+be to act as a pointer to a URI that can supply annotations from an implementation.
+In that case the URI would take a parameter with the test name as a way of telling
+the end point what test is running so it can deliver the right content.
+
+### <a id="assertionLists">Assertion Lists</a> ###
+
+The `assertion` list is an ordered list of assertions that will be evaluated
+against the submitted content. The list is *required*, and MUST have at least
+one entry. Entries in the list have the following types:
+
+* AssertionObject
+
+An in-line Object as defined in the section [Assertion
+Objects](#assertionObjects).
+* URI
+
+A relative or absolute URI that references a AssertionObject in a .json file.
+If the URI is relative but contains no slashes, then it is considered to be
+in the current directory.  If the URI is relative, contains slashes, but
+**does not start with a slash** then it is considered relative to the top of
+the tree of the current test collection (e.g., `annotation-model`).
+* List @@@ATRISK@@@
+
+A nested Assertion List.  While nested Assertion Lists are optional, if one
+is present it MUST have at least one entry.  Entries are as in this list.
+Assertion Lists can be nested to any depth (but don't do that - it would be
+too hard to maintain).
+
+
+<a id="assertionObjects">Assertion Objects</a>
+-----------------
+
+In this collection of tests, Assertion Objects can be contained inline in the
+`.test` files or contained in external files with the suffix `.json`.  The
+vocabularly and structure is as defined in [JSON Schema
+v4](http://json-schema.org/documentation.html) augmented with some additional
+properties defined in this section.
+
+In general each JSON Schema definition provided in this test suite should be as
+minimal as possible.  This promotes clarity and increases the likelihood that
+it is correct.  While it is ---possible--- to create JSON Schema files that
+enforce many different requirements on a data model, it increases the
+complexity and can also reduce the atomicity of tests / sub-tests (because a
+    test ends up testing more than one thing).  Please try to avoid creating
+complex JSON Schema.  (A notable exception is the situation where multiple
+    properties of a structure are interdependent.)
+
+Tools such as [the JSON Schema Creator](http://jsonschema.net/) may be helpful
+in creating schema snippets that can be integrated into JSONtest Assertion
+Objects.  Remember that the JSON Schema you create should be as permissive as
+possible to address the various shapes that a give property might take (e.g., a
+    'foo' might be a string or an object that contains sub-properties that express
+    the string, or an array of 1 or more objects with those sub-properties).
+
+In addition to the validation keys defined in JSON Schema v4, Schema files in
+this collection are also permitted to use the following keywords:
+
+|Keyword        | Values          | Meaning |
+|---------------|-----------------|---------|
+|onUnexpectedResult   | `failAndContinue`, `failAndSkip`, `failAndAbort`, `passAndContinue`, `passAndSkip`, `passAndAbort` | Action to take when the result is not as expected. Default is `failAndContinue` |
+|assertionType  | `must`, `may`, `should` | Informs the system about the severity of a failure. The default is `must` |
+|assertionFile | URI      | An external file that contains an assertion SCHEMA.  When this value is supplied, and local properties will override the ones loaded from the external file.
+|errorMessage   | string          | A human readable explanation of what it means if the test fails.  |
+|expectedResult | `valid`, `invalid`  | Tells the framework whether validating against this schema is expected to succeed or fail.  The default is `valid` |
+
+
+### Example Assertion Object ###
+
+<pre>
+{
+  "$schema": "http://json-schema.org/draft-04/schema#",
+    "title": "Verify annotation has @context",
+    "type": "object",
+    "expectedResult" : "valid",
+    "properties": {
+      "@context": {
+        "anyOf": [
+        {
+          "type": "string"
+        },
+        {
+          "type": "array",
+          "anyOf": [
+          {
+            "type": "string"
+          }
+          ]
+        }
+        ],
+          "not": {"type": "object"}
+      }
+    },
+    "required": ["@context"]
+}
+</pre>
+
+Note that in the case where a feature is *optional* the JSON Schema MUST be
+crafted such that if the attribute is permitted to be missing from the content
+(so that the result is `true`), but when the attribute is present in the
+content it conforms to any requirements.
+
+
+
+<a id="conditionObjects">Condition Objects</a>
+-----------------
+
+A Condition Object is a sub-class of an Assertion Object.  It allows the
+specification of the evaluation strategy for the assertions referenced by the
+object.  An object is a Condition Object IFF it has a `assertions` property. In
+this case, there MUST NOT be an `assertionFile` property.
+
+
+|Keyword        | Values          | Meaning |
+|---------------|-----------------|---------|
+|compareWith    | `and`, `or` | How should the result of any referenced assertions be compared.  Defaults to `and`.  Note that this implies there is also an assertions property with a nested list of assertions to compare. |
+|assertions     | a list of assertions as in a Test Case above. This is required if there is a compareWith property |
+
+
+An example of a test that would pass if there were an `@context` OR there were an `@id`:
+
+<pre>
+{
+  "@context": "https://www.w3.org/ns/JSONtest-v1.jsonld",
+    "name": "A test that has an 'or' clause",
+    "description": "A complex test that uses or-ing among a list of assertions",
+    "ref": "https://www.w3.org/TR/annotation-model/#model",
+    "testType": "manual",
+    "assertions": [
+    { "$schema": "http://json-schema.org/draft-04/schema#",
+      "title": "must have context or id",
+      "description": "A more complex example that allows one of many options to pass",
+      "assertions": [
+      { "title": "Condition Object",
+        "description": "A pseudo-test that will get a result from the aggregate of its children",
+        "assertionType": "must",
+        "expectedResult": "valid",
+        "errorMessage": "Error: None of the various options were present",
+        "compareWith": "or",
+        "assertions": [
+          "common/has_context.json",
+        "common/has_id.json"
+        ]
+      }
+      ]
+    }
+    ]
+}
+</pre>
+
+
+Command Line Tools
+------------------
+
+### Building the Test Files ###
+
+The actual .html test case files are generated using the script
+tools/make_tests.py.  This script will search the directory heriarchy looking for
+files ending on `.test` and creating `.html` files from them using the template in
+the tools folder.  If you want to regenerate the examples too, supply the
+`--examples` option to the script.
+
+Note that when submitting tests to the repository, the `.html` versions must be
+included.
+
+### Testing the Tests ###
+
+### Driving Tests with Input Files ###
+
+Complex Examples
+----------------
+
+This section is a collection of more complex examples to illustrate the
+expressive power of the [Assertion List](#assertionLists) structure.  These can
+be used as templates for creating actual `.test` files.
+
+### Including and Overriding an Assertion ###
+
+Assertions can be contained in external `.json` files.  It is possible for an
+object in an Assertion List to include the external file and override one or
+more of its properties:
+
+<pre>
+{
+  "@context": "https://www.w3.org/ns/JSONtest-v1.jsonld",
+    "name": "Permit no target property",
+    "description": "Ensure there is no 'target' property when there is a 'randomName' property in the Annotation",
+    "assertions": [
+    {
+      "$schema": "http://json-schema.org/draft-04/schema#",
+      "title": "Verify annotation has randomName",
+      "type": "object",
+      "properties": {
+        "randomName": {
+          "type": "string"
+        }
+      },
+      "required": ["randomName"]
+    },
+    { "assertionFile" : "common/target.json",
+      "title" : "Require target to be missing",
+      "expectedResult" : "invalid",
+      "errorMessage" : "The target MUST not be present when 'randomName' is also present",
+    }
+    ]
+}
+</pre>
+
+### Nested Assertion Collections with Skip ###
+
+Assertion Lists can be nested within Assertion Lists.  This feature, combined
+with the `onUnexpectedResult` property, makes it possible to skip a collection
+of tests when an assertion in the list is not satisfied:
+
+<pre>
+{
+  "@context": "https://www.w3.org/ns/JSONtest-v1.jsonld",
+    "name": "If there is no 'target' property, skip some tests",
+    "description": "When 'target' is not present, other properties related to 'target' are not required",
+    "assertions": [
+      "common/context.json",
+    [
+    { "assertionFile" : "common/target.json",
+      "errorMessage" : "Target was not present so skip the rest of this section",
+      "onUnexpectedResult" : "failAndSkip"
+    },
+    "sometest.json",
+    "sometest2.json",
+    "sometest3.json"
+    ]
+    ]
+} ;
+</pre>
+
+### Assertion that finds a specific @context Value ###
+
+Sometimes you want a property to be flexible, but to have one and only one of a
+specific value.  This is especially true with, for example, @context in JSON-LD.
+One way you might do this is:
+
+<pre>
+{
+  "$schema": "http://json-schema.org/draft-04/schema#",
+    "title": "Verify a specific @context",
+    "type": "object",
+    "expectedResult" : "valid",
+    "properties": {
+      "@context": {
+        "anyOf": [
+        {
+          "type": "string"
+            "enum": [ "http://www.w3.org/ns/anno.jsonld" ]
+        },
+        {
+          "type": "array",
+          "minitems": "1",
+          "uniqueItems": true,
+          "additionalItems": true,
+          "items" : [
+          { "type": "string",
+            "enum": [ "http://www.w3.org/ns/anno.jsonld" ]
+          }
+          ]
+        }
+        ],
+          "not": {"type": "object"}
+      }
+    },
+    "required": ["@context"]
+}
+
+</pre>