summaryrefslogtreecommitdiffstats
path: root/browser/experiments/docs
diff options
context:
space:
mode:
authorMatt A. Tobin <mattatobin@localhost.localdomain>2018-02-02 04:16:08 -0500
committerMatt A. Tobin <mattatobin@localhost.localdomain>2018-02-02 04:16:08 -0500
commit5f8de423f190bbb79a62f804151bc24824fa32d8 (patch)
tree10027f336435511475e392454359edea8e25895d /browser/experiments/docs
parent49ee0794b5d912db1f95dce6eb52d781dc210db5 (diff)
downloadUXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar.gz
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar.lz
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.tar.xz
UXP-5f8de423f190bbb79a62f804151bc24824fa32d8.zip
Add m-esr52 at 52.6.0
Diffstat (limited to 'browser/experiments/docs')
-rw-r--r--browser/experiments/docs/index.rst13
-rw-r--r--browser/experiments/docs/manifest.rst429
2 files changed, 442 insertions, 0 deletions
diff --git a/browser/experiments/docs/index.rst b/browser/experiments/docs/index.rst
new file mode 100644
index 000000000..11e5d4faa
--- /dev/null
+++ b/browser/experiments/docs/index.rst
@@ -0,0 +1,13 @@
+=====================
+Telemetry Experiments
+=====================
+
+Telemetry Experiments is a feature of Firefox that allows the installation
+of add-ons called experiments to a subset of the Firefox population for
+the purposes of experimenting with changes and collecting data on specific
+aspects of application usage.
+
+.. toctree::
+ :maxdepth: 1
+
+ manifest
diff --git a/browser/experiments/docs/manifest.rst b/browser/experiments/docs/manifest.rst
new file mode 100644
index 000000000..d4fad5243
--- /dev/null
+++ b/browser/experiments/docs/manifest.rst
@@ -0,0 +1,429 @@
+.. _experiments_manifests:
+
+=====================
+Experiments Manifests
+=====================
+
+*Experiments Manifests* are documents that describe the set of active
+experiments a client may run.
+
+*Experiments Manifests* are fetched periodically by clients. When
+fetched, clients look at the experiments within the manifest and
+determine which experiments are applicable. If an experiment is
+applicable, the client may download and start the experiment.
+
+Manifest Format
+===============
+
+Manifests are JSON documents where the main element is an object.
+
+The *schema* of the object is versioned and defined by the presence
+of a top-level ``version`` property, whose integer value is the
+schema version used by that manifest. Each version is documented
+in the sections below.
+
+Version 1
+---------
+
+Version 1 is the original manifest format.
+
+The following properties may exist in the root object:
+
+experiments
+ An array of objects describing candidate experiments. The format of
+ these objects is documented below.
+
+ An array is used to create an explicit priority of experiments.
+ Experiments listed at the beginning of the array take priority over
+ experiments that follow.
+
+Experiments Objects
+^^^^^^^^^^^^^^^^^^^
+
+Each object in the ``experiments`` array may contain the following
+properties:
+
+id
+ (required) String identifier of this experiment. The identifier should
+ be treated as opaque by clients. It is used to uniquely identify an
+ experiment for all of time.
+
+xpiURL
+ (required) String URL of the XPI that implements this experiment.
+
+ If the experiment is activated, the client will download and install this
+ XPI.
+
+xpiHash
+ (required) String hash of the XPI that implements this experiment.
+
+ The value is composed of a hash identifier followed by a colon
+ followed by the hash value. e.g.
+ `sha1:f677428b9172e22e9911039aef03f3736e7f78a7`. `sha1` and `sha256`
+ are the two supported hashing mechanisms. The hash value is the hex
+ encoding of the binary hash.
+
+ When the client downloads the XPI for the experiment, it should compare
+ the hash of that XPI against this value. If the hashes don't match,
+ the client should not install the XPI.
+
+ Clients may also use this hash as a means of determining when an
+ experiment's XPI has changed and should be refreshed.
+
+startTime
+ Integer seconds since UNIX epoch that this experiment should
+ start. Clients should not start an experiment if *now()* is less than
+ this value.
+
+maxStartTime
+ (optional) Integer seconds since UNIX epoch after which this experiment
+ should no longer start.
+
+ Some experiments may wish to impose hard deadlines after which no new
+ clients should activate the experiment. This property may be used to
+ facilitate that.
+
+endTime
+ Integer seconds since UNIX epoch after which this experiment
+ should no longer run. Clients should cease an experiment when the current
+ time is beyond this value.
+
+maxActiveSeconds
+ Integer seconds defining the max wall time this experiment should be
+ active for.
+
+ The client should deactivate the experiment this many seconds after
+ initial activation.
+
+ This value only involves wall time, not browser activity or session time.
+
+appName
+ Array of application names this experiment should run on.
+
+ An application name comes from ``nsIXULAppInfo.name``. It is a value
+ like ``Firefox``, ``Fennec``, or `B2G`.
+
+ The client should compare its application name against the members of
+ this array. If a match is found, the experiment is applicable.
+
+minVersion
+ (optional) String version number of the minimum application version this
+ experiment should run on.
+
+ A version number is something like ``27.0.0`` or ``28``.
+
+ The client should compare its version number to this value. If the client's
+ version is greater or equal to this version (using a version-aware comparison
+ function), the experiment is applicable.
+
+ If this is not specified, there is no lower bound to versions this
+ experiment should run on.
+
+maxVersion
+ (optional) String version number of the maximum application version this
+ experiment should run on.
+
+ This is similar to ``minVersion`` except it sets the upper bound for
+ application versions.
+
+ If the client's version is less than or equal to this version, the
+ experiment is applicable.
+
+ If this is not specified, there is no upper bound to versions this
+ experiment should run on.
+
+version
+ (optional) Array of application versions this experiment should run on.
+
+ This is similar to ``minVersion`` and ``maxVersion`` except only a
+ whitelisted set of specific versions are allowed.
+
+ The client should compare its version to members of this array. If a match
+ is found, the experiment is applicable.
+
+minBuildID
+ (optional) String minimum Build ID this experiment should run on.
+
+ Build IDs are values like ``201402261424``.
+
+ The client should perform a string comparison of its Build ID against this
+ value. If its value is greater than or equal to this value, the experiment
+ is applicable.
+
+maxBuildID
+ (optional) String maximum Build ID this experiment should run on.
+
+ This is similar to ``minBuildID`` except it sets the upper bound
+ for Build IDs.
+
+ The client should perform a string comparison of its Build ID against
+ this value. If its value is less than or equal to this value, the
+ experiment is applicable.
+
+buildIDs
+ (optional) Array of Build IDs this experiment should run on.
+
+ This is similar to ``minBuildID`` and ``maxBuildID`` except only a
+ whitelisted set of Build IDs are considered.
+
+ The client should compare its Build ID to members of this array. If a
+ match is found, the experiment is applicable.
+
+os
+ (optional) Array of operating system identifiers this experiment should
+ run on.
+
+ Values for this array come from ``nsIXULRuntime.OS``.
+
+ The client will compare its operating system identifier to members
+ of this array. If a match is found, the experiment is applicable to the
+ client.
+
+channel
+ (optional) Array of release channel identifiers this experiment should run
+ on.
+
+ The client will compare its channel to members of this array. If a match
+ is found, the experiment is applicable.
+
+ If this property is not defined, the client should assume the experiment
+ is to run on all channels.
+
+locale
+ (optional) Array of locale identifiers this experiment should run on.
+
+ A locale identifier is a string like ``en-US`` or ``zh-CN`` and is
+ obtained by looking at
+ ``nsIXULChromeRegistry.getSelectedLocale("global")``.
+
+ The client should compare its locale identifier to members of this array.
+ If a match is found, the experiment is applicable.
+
+ If this property is not defined, the client should assume the experiment
+ is to run on all locales.
+
+sample
+ (optional) Decimal number indicating the sampling rate for this experiment.
+
+ This will contain a value between ``0.0`` and ``1.0``. The client should
+ generate a random decimal between ``0.0`` and ``1.0``. If the randomly
+ generated number is less than or equal to the value of this field, the
+ experiment is applicable.
+
+disabled
+ (optional) Boolean value indicating whether an experiment is disabled.
+
+ Normally, experiments are deactivated after a certain time has passed or
+ after the experiment itself determines it no longer needs to run (perhaps
+ it collected sufficient data already).
+
+ This property serves as a backup mechanism to remotely disable an
+ experiment before it was scheduled to be disabled. It can be used to
+ kill experiments that are found to be doing wrong or bad things or that
+ aren't useful.
+
+ If this property is not defined or is false, the client should assume
+ the experiment is active and a candidate for activation.
+
+frozen
+ (optional) Boolean value indicating this experiment is frozen and no
+ longer accepting new enrollments.
+
+ If a client sees a true value in this field, it should not attempt to
+ activate an experiment.
+
+jsfilter
+ (optional) JavaScript code that will be evaluated to determine experiment
+ applicability.
+
+ This property contains the string representation of JavaScript code that
+ will be evaluated in a sandboxed environment using JavaScript's
+ ``eval()``.
+
+ The string is expected to contain the definition of a JavaScript function
+ ``filter(context)``. This function receives as its argument an object
+ holding application state. See the section below for the definition of
+ this object.
+
+ The purpose of this property is to allow experiments to define complex
+ rules and logic for evaluating experiment applicability in a manner
+ that is privacy conscious and doesn't require the transmission of
+ excessive data.
+
+ The return value of this filter indicates whether the experiment is
+ applicable. Functions should return true if the experiment is
+ applicable.
+
+ If an experiment is not applicable, they should throw an Error whose
+ message contains the reason the experiment is not applicable. This
+ message may be logged and sent to remote servers, so it should not
+ contain private or otherwise sensitive data that wouldn't normally
+ be submitted.
+
+ If a falsey (or undefined) value is returned, the client should
+ assume the experiment is not applicable.
+
+ If this property is not defined, the client does not consider a custom
+ JavaScript filter function when determining whether an experiment is
+ applicable.
+
+JavaScript Filter Context Objects
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The object passed to a ``jsfilter`` ``filter()`` function contains the
+following properties:
+
+healthReportSubmissionEnabled
+ This property contains a boolean indicating whether Firefox Health
+ Report has its data submission flag enabled (whether Firefox Health
+ Report is sending data to remote servers).
+
+healthReportPayload
+ This property contains the current Firefox Health Report payload.
+
+ The payload format is documented at :ref:`healthreport_dataformat`.
+
+telemetryPayload
+ This property contains the current Telemetry payload.
+
+The evaluation sandbox for the JavaScript filters may be destroyed
+immediately after ``filter()`` returns. This function should not assume
+async code will finish.
+
+Experiment Applicability and Client Behavior
+============================================
+
+The point of an experiment manifest is to define which experiments are
+available and where and how to run them. This section explains those
+rules in more detail.
+
+Many of the properties in *Experiment Objects* are related to determining
+whether an experiment should run on a given client. This evaluation is
+performed client side.
+
+1. Multiple conditions in an experiment
+---------------------------------------
+
+If multiple conditions are defined for an experiment, the client should
+combine each condition with a logical *AND*: all conditions must be
+satisfied for an experiment to run. If one condition fails, the experiment
+is not applicable.
+
+2. Active experiment disappears from manifest
+---------------------------------------------
+
+If a specific experiment disappears from the manifest, the client should
+continue conducting an already-active experiment. Furthermore, the
+client should remember what the expiration events were for an experiment
+and honor them.
+
+The rationale here is that we want to prevent an accidental deletion
+or temporary failure on the server to inadvertantly deactivate
+supposed-to-be-active experiments. We also don't want premature deletion
+of an experiment from the manifest to result in indefinite activation
+periods.
+
+3. Inactive experiment disappears from manifest
+-----------------------------------------------
+
+If an inactive but scheduled-to-be-active experiment disappears from the
+manifest, the client should not activate the experiment.
+
+If that experiment reappears in the manifest, the client should not
+treat that experiment any differently than any other new experiment. Put
+another way, the fact an inactive experiment disappears and then
+reappears should not be significant.
+
+The rationale here is that server operators should have complete
+control of an inactive experiment up to it's go-live date.
+
+4. Re-evaluating applicability on manifest refresh
+--------------------------------------------------
+
+When an experiment manifest is refreshed or updated, the client should
+re-evaluate the applicability of each experiment therein.
+
+The rationale here is that the server may change the parameters of an
+experiment and want clients to pick those up.
+
+5. Activating a previously non-applicable experiment
+----------------------------------------------------
+
+If the conditions of an experiment change or the state of the client
+changes to allow an experiment to transition from previously
+non-applicable to applicable, the experiment should be activated.
+
+For example, if a client is running version 28 and the experiment
+initially requires version 29 or above, the client will not mark the
+experiment as applicable. But if the client upgrades to version 29 or if
+the manifest is updated to require 28 or above, the experiment will
+become applicable.
+
+6. Deactivating a previously active experiment
+----------------------------------------------
+
+If the conditions of an experiment change or the state of the client
+changes and an active experiment is no longer applicable, that
+experiment should be deactivated.
+
+7. Calculation of sampling-based applicability
+----------------------------------------------
+
+For calculating sampling-based applicability, the client will associate
+a random value between ``0.0`` and ``1.0`` for each observed experiment
+ID. This random value will be generated the first time sampling
+applicability is evaluated. This random value will be persisted and used
+in future applicability evaluations for this experiment.
+
+By saving and re-using the value, the client is able to reliably and
+consistently evaluate applicability, even if the sampling threshold
+in the manifest changes.
+
+Clients should retain the randomly-generated sampling value for
+experiments that no longer appear in a manifest for a period of at least
+30 days. The rationale is that if an experiment disappears and reappears
+from a manifest, the client will not have multiple opportunities to
+generate a random value that satisfies the sampling criteria.
+
+8. Incompatible version numbers
+-------------------------------
+
+If a client receives a manifest with a version number that it doesn't
+recognize, it should ignore the manifest.
+
+9. Usage of old manifests
+-------------------------
+
+If a client experiences an error fetching a manifest (server not
+available) or if the manifest is corrupt, not readable, or compatible,
+the client may use a previously-fetched (cached) manifest.
+
+10. Updating XPIs
+-----------------
+
+If the URL or hash of an active experiment's XPI changes, the client
+should fetch the new XPI, uninstall the old XPI, and install the new
+XPI.
+
+Examples
+========
+
+Here is an example manifest::
+
+ {
+ "version": 1,
+ "experiments": [
+ {
+ "id": "da9d7f4f-f3f9-4f81-bacd-6f0626ffa360",
+ "xpiURL": "https://experiments.mozilla.org/foo.xpi",
+ "xpiHash": "sha1:cb1eb32b89d86d78b7326f416cf404548c5e0099",
+ "startTime": 1393000000,
+ "endTime": 1394000000,
+ "appName": ["Firefox", "Fennec"],
+ "minVersion": "28",
+ "maxVersion": "30",
+ "os": ["windows", "linux", "osx"],
+ "jsfilter": "function filter(context) { return context.healthReportEnabled; }"
+ }
+ ]
+ }