summaryrefslogtreecommitdiffstats
path: root/browser/experiments/docs
diff options
context:
space:
mode:
Diffstat (limited to 'browser/experiments/docs')
-rw-r--r--browser/experiments/docs/index.rst13
-rw-r--r--browser/experiments/docs/manifest.rst429
2 files changed, 0 insertions, 442 deletions
diff --git a/browser/experiments/docs/index.rst b/browser/experiments/docs/index.rst
deleted file mode 100644
index 11e5d4faa..000000000
--- a/browser/experiments/docs/index.rst
+++ /dev/null
@@ -1,13 +0,0 @@
-=====================
-Telemetry Experiments
-=====================
-
-Telemetry Experiments is a feature of Firefox that allows the installation
-of add-ons called experiments to a subset of the Firefox population for
-the purposes of experimenting with changes and collecting data on specific
-aspects of application usage.
-
-.. toctree::
- :maxdepth: 1
-
- manifest
diff --git a/browser/experiments/docs/manifest.rst b/browser/experiments/docs/manifest.rst
deleted file mode 100644
index d4fad5243..000000000
--- a/browser/experiments/docs/manifest.rst
+++ /dev/null
@@ -1,429 +0,0 @@
-.. _experiments_manifests:
-
-=====================
-Experiments Manifests
-=====================
-
-*Experiments Manifests* are documents that describe the set of active
-experiments a client may run.
-
-*Experiments Manifests* are fetched periodically by clients. When
-fetched, clients look at the experiments within the manifest and
-determine which experiments are applicable. If an experiment is
-applicable, the client may download and start the experiment.
-
-Manifest Format
-===============
-
-Manifests are JSON documents where the main element is an object.
-
-The *schema* of the object is versioned and defined by the presence
-of a top-level ``version`` property, whose integer value is the
-schema version used by that manifest. Each version is documented
-in the sections below.
-
-Version 1
----------
-
-Version 1 is the original manifest format.
-
-The following properties may exist in the root object:
-
-experiments
- An array of objects describing candidate experiments. The format of
- these objects is documented below.
-
- An array is used to create an explicit priority of experiments.
- Experiments listed at the beginning of the array take priority over
- experiments that follow.
-
-Experiments Objects
-^^^^^^^^^^^^^^^^^^^
-
-Each object in the ``experiments`` array may contain the following
-properties:
-
-id
- (required) String identifier of this experiment. The identifier should
- be treated as opaque by clients. It is used to uniquely identify an
- experiment for all of time.
-
-xpiURL
- (required) String URL of the XPI that implements this experiment.
-
- If the experiment is activated, the client will download and install this
- XPI.
-
-xpiHash
- (required) String hash of the XPI that implements this experiment.
-
- The value is composed of a hash identifier followed by a colon
- followed by the hash value. e.g.
- `sha1:f677428b9172e22e9911039aef03f3736e7f78a7`. `sha1` and `sha256`
- are the two supported hashing mechanisms. The hash value is the hex
- encoding of the binary hash.
-
- When the client downloads the XPI for the experiment, it should compare
- the hash of that XPI against this value. If the hashes don't match,
- the client should not install the XPI.
-
- Clients may also use this hash as a means of determining when an
- experiment's XPI has changed and should be refreshed.
-
-startTime
- Integer seconds since UNIX epoch that this experiment should
- start. Clients should not start an experiment if *now()* is less than
- this value.
-
-maxStartTime
- (optional) Integer seconds since UNIX epoch after which this experiment
- should no longer start.
-
- Some experiments may wish to impose hard deadlines after which no new
- clients should activate the experiment. This property may be used to
- facilitate that.
-
-endTime
- Integer seconds since UNIX epoch after which this experiment
- should no longer run. Clients should cease an experiment when the current
- time is beyond this value.
-
-maxActiveSeconds
- Integer seconds defining the max wall time this experiment should be
- active for.
-
- The client should deactivate the experiment this many seconds after
- initial activation.
-
- This value only involves wall time, not browser activity or session time.
-
-appName
- Array of application names this experiment should run on.
-
- An application name comes from ``nsIXULAppInfo.name``. It is a value
- like ``Firefox``, ``Fennec``, or `B2G`.
-
- The client should compare its application name against the members of
- this array. If a match is found, the experiment is applicable.
-
-minVersion
- (optional) String version number of the minimum application version this
- experiment should run on.
-
- A version number is something like ``27.0.0`` or ``28``.
-
- The client should compare its version number to this value. If the client's
- version is greater or equal to this version (using a version-aware comparison
- function), the experiment is applicable.
-
- If this is not specified, there is no lower bound to versions this
- experiment should run on.
-
-maxVersion
- (optional) String version number of the maximum application version this
- experiment should run on.
-
- This is similar to ``minVersion`` except it sets the upper bound for
- application versions.
-
- If the client's version is less than or equal to this version, the
- experiment is applicable.
-
- If this is not specified, there is no upper bound to versions this
- experiment should run on.
-
-version
- (optional) Array of application versions this experiment should run on.
-
- This is similar to ``minVersion`` and ``maxVersion`` except only a
- whitelisted set of specific versions are allowed.
-
- The client should compare its version to members of this array. If a match
- is found, the experiment is applicable.
-
-minBuildID
- (optional) String minimum Build ID this experiment should run on.
-
- Build IDs are values like ``201402261424``.
-
- The client should perform a string comparison of its Build ID against this
- value. If its value is greater than or equal to this value, the experiment
- is applicable.
-
-maxBuildID
- (optional) String maximum Build ID this experiment should run on.
-
- This is similar to ``minBuildID`` except it sets the upper bound
- for Build IDs.
-
- The client should perform a string comparison of its Build ID against
- this value. If its value is less than or equal to this value, the
- experiment is applicable.
-
-buildIDs
- (optional) Array of Build IDs this experiment should run on.
-
- This is similar to ``minBuildID`` and ``maxBuildID`` except only a
- whitelisted set of Build IDs are considered.
-
- The client should compare its Build ID to members of this array. If a
- match is found, the experiment is applicable.
-
-os
- (optional) Array of operating system identifiers this experiment should
- run on.
-
- Values for this array come from ``nsIXULRuntime.OS``.
-
- The client will compare its operating system identifier to members
- of this array. If a match is found, the experiment is applicable to the
- client.
-
-channel
- (optional) Array of release channel identifiers this experiment should run
- on.
-
- The client will compare its channel to members of this array. If a match
- is found, the experiment is applicable.
-
- If this property is not defined, the client should assume the experiment
- is to run on all channels.
-
-locale
- (optional) Array of locale identifiers this experiment should run on.
-
- A locale identifier is a string like ``en-US`` or ``zh-CN`` and is
- obtained by looking at
- ``nsIXULChromeRegistry.getSelectedLocale("global")``.
-
- The client should compare its locale identifier to members of this array.
- If a match is found, the experiment is applicable.
-
- If this property is not defined, the client should assume the experiment
- is to run on all locales.
-
-sample
- (optional) Decimal number indicating the sampling rate for this experiment.
-
- This will contain a value between ``0.0`` and ``1.0``. The client should
- generate a random decimal between ``0.0`` and ``1.0``. If the randomly
- generated number is less than or equal to the value of this field, the
- experiment is applicable.
-
-disabled
- (optional) Boolean value indicating whether an experiment is disabled.
-
- Normally, experiments are deactivated after a certain time has passed or
- after the experiment itself determines it no longer needs to run (perhaps
- it collected sufficient data already).
-
- This property serves as a backup mechanism to remotely disable an
- experiment before it was scheduled to be disabled. It can be used to
- kill experiments that are found to be doing wrong or bad things or that
- aren't useful.
-
- If this property is not defined or is false, the client should assume
- the experiment is active and a candidate for activation.
-
-frozen
- (optional) Boolean value indicating this experiment is frozen and no
- longer accepting new enrollments.
-
- If a client sees a true value in this field, it should not attempt to
- activate an experiment.
-
-jsfilter
- (optional) JavaScript code that will be evaluated to determine experiment
- applicability.
-
- This property contains the string representation of JavaScript code that
- will be evaluated in a sandboxed environment using JavaScript's
- ``eval()``.
-
- The string is expected to contain the definition of a JavaScript function
- ``filter(context)``. This function receives as its argument an object
- holding application state. See the section below for the definition of
- this object.
-
- The purpose of this property is to allow experiments to define complex
- rules and logic for evaluating experiment applicability in a manner
- that is privacy conscious and doesn't require the transmission of
- excessive data.
-
- The return value of this filter indicates whether the experiment is
- applicable. Functions should return true if the experiment is
- applicable.
-
- If an experiment is not applicable, they should throw an Error whose
- message contains the reason the experiment is not applicable. This
- message may be logged and sent to remote servers, so it should not
- contain private or otherwise sensitive data that wouldn't normally
- be submitted.
-
- If a falsey (or undefined) value is returned, the client should
- assume the experiment is not applicable.
-
- If this property is not defined, the client does not consider a custom
- JavaScript filter function when determining whether an experiment is
- applicable.
-
-JavaScript Filter Context Objects
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The object passed to a ``jsfilter`` ``filter()`` function contains the
-following properties:
-
-healthReportSubmissionEnabled
- This property contains a boolean indicating whether Firefox Health
- Report has its data submission flag enabled (whether Firefox Health
- Report is sending data to remote servers).
-
-healthReportPayload
- This property contains the current Firefox Health Report payload.
-
- The payload format is documented at :ref:`healthreport_dataformat`.
-
-telemetryPayload
- This property contains the current Telemetry payload.
-
-The evaluation sandbox for the JavaScript filters may be destroyed
-immediately after ``filter()`` returns. This function should not assume
-async code will finish.
-
-Experiment Applicability and Client Behavior
-============================================
-
-The point of an experiment manifest is to define which experiments are
-available and where and how to run them. This section explains those
-rules in more detail.
-
-Many of the properties in *Experiment Objects* are related to determining
-whether an experiment should run on a given client. This evaluation is
-performed client side.
-
-1. Multiple conditions in an experiment
----------------------------------------
-
-If multiple conditions are defined for an experiment, the client should
-combine each condition with a logical *AND*: all conditions must be
-satisfied for an experiment to run. If one condition fails, the experiment
-is not applicable.
-
-2. Active experiment disappears from manifest
----------------------------------------------
-
-If a specific experiment disappears from the manifest, the client should
-continue conducting an already-active experiment. Furthermore, the
-client should remember what the expiration events were for an experiment
-and honor them.
-
-The rationale here is that we want to prevent an accidental deletion
-or temporary failure on the server to inadvertantly deactivate
-supposed-to-be-active experiments. We also don't want premature deletion
-of an experiment from the manifest to result in indefinite activation
-periods.
-
-3. Inactive experiment disappears from manifest
------------------------------------------------
-
-If an inactive but scheduled-to-be-active experiment disappears from the
-manifest, the client should not activate the experiment.
-
-If that experiment reappears in the manifest, the client should not
-treat that experiment any differently than any other new experiment. Put
-another way, the fact an inactive experiment disappears and then
-reappears should not be significant.
-
-The rationale here is that server operators should have complete
-control of an inactive experiment up to it's go-live date.
-
-4. Re-evaluating applicability on manifest refresh
---------------------------------------------------
-
-When an experiment manifest is refreshed or updated, the client should
-re-evaluate the applicability of each experiment therein.
-
-The rationale here is that the server may change the parameters of an
-experiment and want clients to pick those up.
-
-5. Activating a previously non-applicable experiment
-----------------------------------------------------
-
-If the conditions of an experiment change or the state of the client
-changes to allow an experiment to transition from previously
-non-applicable to applicable, the experiment should be activated.
-
-For example, if a client is running version 28 and the experiment
-initially requires version 29 or above, the client will not mark the
-experiment as applicable. But if the client upgrades to version 29 or if
-the manifest is updated to require 28 or above, the experiment will
-become applicable.
-
-6. Deactivating a previously active experiment
-----------------------------------------------
-
-If the conditions of an experiment change or the state of the client
-changes and an active experiment is no longer applicable, that
-experiment should be deactivated.
-
-7. Calculation of sampling-based applicability
-----------------------------------------------
-
-For calculating sampling-based applicability, the client will associate
-a random value between ``0.0`` and ``1.0`` for each observed experiment
-ID. This random value will be generated the first time sampling
-applicability is evaluated. This random value will be persisted and used
-in future applicability evaluations for this experiment.
-
-By saving and re-using the value, the client is able to reliably and
-consistently evaluate applicability, even if the sampling threshold
-in the manifest changes.
-
-Clients should retain the randomly-generated sampling value for
-experiments that no longer appear in a manifest for a period of at least
-30 days. The rationale is that if an experiment disappears and reappears
-from a manifest, the client will not have multiple opportunities to
-generate a random value that satisfies the sampling criteria.
-
-8. Incompatible version numbers
--------------------------------
-
-If a client receives a manifest with a version number that it doesn't
-recognize, it should ignore the manifest.
-
-9. Usage of old manifests
--------------------------
-
-If a client experiences an error fetching a manifest (server not
-available) or if the manifest is corrupt, not readable, or compatible,
-the client may use a previously-fetched (cached) manifest.
-
-10. Updating XPIs
------------------
-
-If the URL or hash of an active experiment's XPI changes, the client
-should fetch the new XPI, uninstall the old XPI, and install the new
-XPI.
-
-Examples
-========
-
-Here is an example manifest::
-
- {
- "version": 1,
- "experiments": [
- {
- "id": "da9d7f4f-f3f9-4f81-bacd-6f0626ffa360",
- "xpiURL": "https://experiments.mozilla.org/foo.xpi",
- "xpiHash": "sha1:cb1eb32b89d86d78b7326f416cf404548c5e0099",
- "startTime": 1393000000,
- "endTime": 1394000000,
- "appName": ["Firefox", "Fennec"],
- "minVersion": "28",
- "maxVersion": "30",
- "os": ["windows", "linux", "osx"],
- "jsfilter": "function filter(context) { return context.healthReportEnabled; }"
- }
- ]
- }