diff options
Diffstat (limited to 'toolkit/components/telemetry/docs/fhr')
| -rw-r--r-- | toolkit/components/telemetry/docs/fhr/architecture.rst | 226 | ||||
| -rw-r--r-- | toolkit/components/telemetry/docs/fhr/dataformat.rst | 1997 | ||||
| -rw-r--r-- | toolkit/components/telemetry/docs/fhr/identifiers.rst | 83 | ||||
| -rw-r--r-- | toolkit/components/telemetry/docs/fhr/index.rst | 34 |
4 files changed, 2340 insertions, 0 deletions
diff --git a/toolkit/components/telemetry/docs/fhr/architecture.rst b/toolkit/components/telemetry/docs/fhr/architecture.rst new file mode 100644 index 000000000..6a857d329 --- /dev/null +++ b/toolkit/components/telemetry/docs/fhr/architecture.rst @@ -0,0 +1,226 @@ +.. _healthreport_architecture: + +============ +Architecture +============ + +``healthreporter.jsm`` contains the main interface for FHR, the +``HealthReporter`` type. An instance of this is created by the +``data_reporting_service`. + +``providers.jsm`` contains numerous ``Metrics.Provider`` and +``Metrics.Measurement`` used for collecting application metrics. If you +are looking for the FHR probes, this is where they are. + +Storage +======= + +Firefox Health Report stores data in 3 locations: + +* Metrics measurements and provider state is stored in a SQLite database + (via ``Metrics.Storage``). +* Service state (such as the IDs of documents uploaded) is stored in a + JSON file on disk (via OS.File). +* Lesser state and run-time options are stored in preferences. + +Preferences +=========== + +Preferences controlling behavior of Firefox Health Report live in the +``datareporting.healthreport.*`` branch. + +Service and Data Control +------------------------ + +The follow preferences control behavior of the service and data upload. + +service.enabled + Controls whether the entire health report service runs. The overall + service performs data collection, storing, and submission. + + This is the primary kill switch for Firefox Health Report + outside of the build system variable. i.e. if you are using an + official Firefox build and wish to disable FHR, this is what you + should set to false to prevent FHR from not only submitting but + also collecting data. + +uploadEnabled + Whether uploading of data is enabled. This is the preference the + checkbox in the preferences UI reflects. If this is + disabled, FHR still collects data - it just doesn't upload it. + +service.loadDelayMsec + How long (in milliseconds) after initial application start should FHR + wait before initializing. + + FHR may initialize sooner than this if the FHR service is requested. + This will happen if e.g. the user goes to ``about:healthreport``. + +service.loadDelayFirstRunMsec + How long (in milliseconds) FHR should wait to initialize on first + application run. + + FHR waits longer than normal to initialize on first application run + because first-time initialization can use a lot of I/O to initialize + the SQLite database and this I/O should not interfere with the + first-run user experience. + +documentServerURI + The URI of a Bagheera server that FHR should interface with for + submitting documents. + + You typically do not need to change this. + +documentServerNamespace + The namespace on the document server FHR should upload documents to. + + You typically do not need to change this. + +infoURL + The URL of a page containing more info about FHR, it's privacy + policy, etc. + +about.reportUrl + The URL to load in ``about:healthreport``. + +about.reportUrlUnified + The URL to load in ``about:healthreport``. This is used instead of ``reportUrl`` for UnifiedTelemetry when it is not opt-in. + +service.providerCategories + A comma-delimited list of category manager categories that contain + registered ``Metrics.Provider`` records. Read below for how provider + registration works. + +If the entire service is disabled, you lose data collection. This means +that **local** data analysis won't be available because there is no data +to analyze! Keep in mind that Firefox Health Report can be useful even +if it's not submitting data to remote servers! + +Logging +------- + +The following preferences allow you to control the logging behavior of +Firefox Health Report. + +logging.consoleEnabled + Whether to write log messages to the web console. This is true by + default. + +logging.consoleLevel + The minimum log level FHR messages must have to be written to the + web console. By default, only FHR warnings or errors will be written + to the web console. During normal/expected operation, no messages of + this type should be produced. + +logging.dumpEnabled + Whether to write log messages via ``dump()``. If true, FHR will write + messages to stdout/stderr. + + This is typically only enabled when developing FHR. + +logging.dumpLevel + The minimum log level messages must have to be written via + ``dump()``. + +State +----- + +currentDaySubmissionFailureCount + How many submission failures the client has encountered while + attempting to upload the most recent document. + +lastDataSubmissionFailureTime + The time of the last failed document upload. + +lastDataSubmissionRequestedTime + The time of the last document upload attempt. + +lastDataSubmissionSuccessfulTime + The time of the last successful document upload. + +nextDataSubmissionTime + The time the next data submission is scheduled for. FHR will not + attempt to upload a new document before this time. + +pendingDeleteRemoteData + Whether the client currently has a pending request to delete remote + data. If true, the client will attempt to delete all remote data + before an upload is performed. + +FHR stores various state in preferences. + +Registering Providers +===================== + +Firefox Health Report providers are registered via the category manager. +See ``HealthReportComponents.manifest`` for providers defined in this +directory. + +Essentially, the category manager receives the name of a JS type and the +URI of a JSM to import that exports this symbol. At run-time, the +providers registered in the category manager are instantiated. + +Providers are registered via the category manager to make registration +simple and less prone to errors. Any XPCOM component can create a +category manager entry. Therefore, new data providers can be added +without having to touch core Firefox Health Report code. Additionally, +category manager registration means providers are more likely to be +registered on FHR's terms, when it wants. If providers were registered +in code at application run-time, there would be the risk of other +components prematurely instantiating FHR (causing a performance hit if +performed at an inopportune time) or semi-complicated code around +observers or listeners. Category manager entries are only 1 line per +provider and leave FHR in control: they are simple and safe. + +Document Generation and Lifecycle +================================= + +FHR will attempt to submit a JSON document containing data every 24 wall +clock hours. + +At upload time, FHR will query the database for **all** information from +the last 180 days and assemble this data into a JSON document. We +attempt to upload this JSON document with a client-generated UUID to the +configured server. + +Before we attempt upload, the generated UUID is stored in the JSON state +file on local disk. At this point, the client assumes the document with +that UUID has been successfully stored on the server. + +If the client is aware of other document UUIDs that presumably exist on +the server, those UUIDs are sent with the upload request so the client +can request those UUIDs be deleted. This helps ensure that each client +only has 1 document/UUID on the server at any one time. + +Importance of Persisting UUIDs +------------------------------ + +The choices of how, where, and when document UUIDs are stored and updated +are very important. One should not attempt to change things unless she +has a very detailed understanding of why things are the way they are. + +The client is purposefully very conservative about forgetting about +generated UUIDs. In other words, once a UUID is generated, the client +deliberately holds on to that UUID until it's very confident that UUID +is no longer stored on the server. The reason we do this is because +*orphaned* documents/UUIDs on the server can lead to faulty analysis, +such as over-reporting the number of Firefox installs that stop being +used. + +When uploading a new UUID, we update the state and save the state file +to disk *before* an upload attempt because if the upload succeeds but +the response never makes it back to the client, we want the client to +know about the uploaded UUID so it can delete it later to prevent an +orphan. + +We maintain a list of UUIDs locally (not simply the last UUID) because +multiple upload attempts could fail the same way as the previous +paragraph describes and we have no way of knowing which (if any) +actually succeeded. The safest approach is to assume every document +produced managed to get uploaded some how. + +We store the UUIDs on a file on disk and not anywhere else because we +want storage to be robust. We originally stored UUIDs in preferences, +which only flush to disk periodically. Writes to preferences were +apparently getting lost. We switched to writing directly to files to +eliminate this window. diff --git a/toolkit/components/telemetry/docs/fhr/dataformat.rst b/toolkit/components/telemetry/docs/fhr/dataformat.rst new file mode 100644 index 000000000..b067f9d0c --- /dev/null +++ b/toolkit/components/telemetry/docs/fhr/dataformat.rst @@ -0,0 +1,1997 @@ +.. _healthreport_dataformat: + +============== +Payload Format +============== + +Currently, the Firefox Health Report is submitted as a compressed JSON +document. The root JSON element is an object. A *version* field defines +the version of the payload which in turn defines the expected contents +the object. + +As of 2013-07-03, desktop submits Version 2, and Firefox for Android submits +Version 3 payloads. + +Version 3 +========= + +Version 3 is a complete rebuild of the document format. Events are tracked in +an "environment". Environments are computed from a large swath of local data +(e.g., add-ons, CPU count, versions), and a new environment comes into being +when one of its attributes changes. + +Client documents, then, will include descriptions of many environments, and +measurements will be attributed to one particular environment. + +A map of environments is present at the top level of the document, with the +current named "current" in the map. Each environment has a hash identifier and +a set of attributes. The current environment is completely described, and has +its hash present in a "hash" attribute. All other environments are represented +as a tree diff from the current environment, with their hash as the key in the +"environments" object. + +A removed add-on has the value 'null'. + +There is no "last" data at present. + +Daily data is hierarchical: by day, then by environment, and then by +measurement, and is present in "data", just as in v2. + +Leading by example:: + + { + "lastPingDate": "2013-06-29", + "thisPingDate": "2013-07-03", + "version": 3, + "environments": { + "current": { + "org.mozilla.sysinfo.sysinfo": { + "memoryMB": 1567, + "cpuCount": 4, + "architecture": "armeabi-v7a", + "_v": 1, + "version": "4.1.2", + "name": "Android" + }, + "org.mozilla.profile.age": { + "_v": 1, + "profileCreation": 15827 + }, + "org.mozilla.addons.active": { + "QuitNow@TWiGSoftware.com": { + "appDisabled": false, + "userDisabled": false, + "scope": 1, + "updateDay": 15885, + "foreignInstall": false, + "hasBinaryComponents": false, + "blocklistState": 0, + "type": "extension", + "installDay": 15885, + "version": "1.18.02" + }, + "{dbbf9331-b713-6eda-1006-205efead09dc}": { + "appDisabled": false, + "userDisabled": "askToActivate", + "scope": 8, + "updateDay": 15779, + "foreignInstall": true, + "blocklistState": 0, + "type": "plugin", + "installDay": 15779, + "version": "11.1 r115" + }, + "desktopbydefault@bnicholson.mozilla.org": { + "appDisabled": false, + "userDisabled": true, + "scope": 1, + "updateDay": 15870, + "foreignInstall": false, + "hasBinaryComponents": false, + "blocklistState": 0, + "type": "extension", + "installDay": 15870, + "version": "1.1" + }, + "{6e092a7f-ba58-4abb-88c1-1a4e50b217e4}": { + "appDisabled": false, + "userDisabled": false, + "scope": 1, + "updateDay": 15828, + "foreignInstall": false, + "hasBinaryComponents": false, + "blocklistState": 0, + "type": "extension", + "installDay": 15828, + "version": "1.1.0" + }, + "{46551EC9-40F0-4e47-8E18-8E5CF550CFB8}": { + "appDisabled": false, + "userDisabled": true, + "scope": 1, + "updateDay": 15879, + "foreignInstall": false, + "hasBinaryComponents": false, + "blocklistState": 0, + "type": "extension", + "installDay": 15879, + "version": "1.3.2" + }, + "_v": 1 + }, + "org.mozilla.appInfo.appinfo": { + "_v": 3, + "appLocale": "en_us", + "osLocale": "en_us", + "distribution": "", + "acceptLangIsUserSet": 0, + "isTelemetryEnabled": 1, + "isBlocklistEnabled": 1 + }, + "geckoAppInfo": { + "updateChannel": "nightly", + "id": "{aa3c5121-dab2-40e2-81ca-7ea25febc110}", + "os": "Android", + "platformBuildID": "20130703031323", + "platformVersion": "25.0a1", + "vendor": "Mozilla", + "name": "fennec", + "xpcomabi": "arm-eabi-gcc3", + "appBuildID": "20130703031323", + "_v": 1, + "version": "25.0a1" + }, + "hash": "tB4Pnnep9yTxnMDymc3dAB2RRB0=", + "org.mozilla.addons.counts": { + "extension": 4, + "plugin": 1, + "_v": 1, + "theme": 0 + } + }, + "k2O3hlreMeS7L1qtxeMsYWxgWWQ=": { + "geckoAppInfo": { + "platformBuildID": "20130630031138", + "appBuildID": "20130630031138", + "_v": 1 + }, + "org.mozilla.appInfo.appinfo": { + "_v": 2, + } + }, + "1+KN9TutMpzdl4TJEl+aCxK+xcw=": { + "geckoAppInfo": { + "platformBuildID": "20130626031100", + "appBuildID": "20130626031100", + "_v": 1 + }, + "org.mozilla.addons.active": { + "QuitNow@TWiGSoftware.com": null, + "{dbbf9331-b713-6eda-1006-205efead09dc}": null, + "desktopbydefault@bnicholson.mozilla.org": null, + "{6e092a7f-ba58-4abb-88c1-1a4e50b217e4}": null, + "{46551EC9-40F0-4e47-8E18-8E5CF550CFB8}": null, + "_v": 1 + }, + "org.mozilla.addons.counts": { + "extension": 0, + "plugin": 0, + "_v": 1 + } + } + }, + "data": { + "last": {}, + "days": { + "2013-07-03": { + "tB4Pnnep9yTxnMDymc3dAB2RRB0=": { + "org.mozilla.appSessions": { + "normal": [ + { + "r": "P", + "d": 2, + "sj": 653 + }, + { + "r": "P", + "d": 22 + }, + { + "r": "P", + "d": 5 + }, + { + "r": "P", + "d": 0 + }, + { + "r": "P", + "sg": 3560, + "d": 171, + "sj": 518 + }, + { + "r": "P", + "d": 16 + }, + { + "r": "P", + "d": 1079 + } + ], + "_v": "4" + } + }, + "k2O3hlreMeS7L1qtxeMsYWxgWWQ=": { + "org.mozilla.appSessions": { + "normal": [ + { + "r": "P", + "d": 27 + }, + { + "r": "P", + "d": 19 + }, + { + "r": "P", + "d": 55 + } + ], + "_v": "4" + }, + "org.mozilla.searches.counts": { + "bartext": { + "google": 1 + }, + "_v": "4" + }, + "org.mozilla.experiment": { + "lastActive": "some.experiment.id" + "_v": "1" + } + } + } + } + } + } + +App sessions in Version 3 +------------------------- + +Sessions are divided into "normal" and "abnormal". Session objects are stored as discrete JSON:: + + "org.mozilla.appSessions": { + _v: 4, + "normal": [ + {"r":"P", "d": 123}, + ], + "abnormal": [ + {"r":"A", "oom": true, "stopped": false} + ] + } + +Keys are: + +"r" + reason. Values are "P" (activity paused), "A" (abnormal termination). +"d" + duration. Value in seconds. +"sg" + Gecko startup time (msec). Present if this is a clean launch. This + corresponds to the telemetry timer *FENNEC_STARTUP_TIME_GECKOREADY*. +"sj" + Java activity init time (msec). Present if this is a clean launch. This + corresponds to the telemetry timer *FENNEC_STARTUP_TIME_JAVAUI*, + and includes initialization tasks beyond initial + *onWindowFocusChanged*. + +Abnormal terminations will be missing a duration and will feature these keys: + +"oom" + was the session killed by an OOM exception? +"stopped" + was the session stopped gently? + +Version 3.2 +----------- + +As of Firefox 35, the search counts measurement is now bumped to v6, including the *activity* location for the search activity. + +Version 3.1 +----------- + +As of Firefox 27, *appinfo* is now bumped to v3, including *osLocale*, +*appLocale* (currently always the same as *osLocale*), *distribution* (a string +containing the distribution ID and version, separated by a colon), and +*acceptLangIsUserSet*, an integer-boolean that describes whether the user set +an *intl.accept_languages* preference. + +The search counts measurement is now at version 5, which indicates that +non-partner searches are recorded. You'll see identifiers like "other-Foo Bar" +rather than "other". + + +Version 3.2 +----------- + +In Firefox 32, Firefox for Android includes a device configuration section +in the environment description:: + + "org.mozilla.device.config": { + "hasHardwareKeyboard": false, + "screenXInMM": 58, + "screenLayout": 2, + "uiType": "default", + "screenYInMM": 103, + "_v": 1, + "uiMode": 1 + } + +Of these, the only keys that need explanation are: + +uiType + One of "default", "smalltablet", "largetablet". +uiMode + A mask of the Android *Configuration.uiMode* value, e.g., + *UI_MODE_TYPE_CAR*. +screenLayout + A mask of the Android *Configuration.screenLayout* value. One of the + *SCREENLAYOUT_SIZE_* constants. + +Note that screen dimensions can be incorrect due to device inaccuracies and platform limitations. + +Other notable differences from Version 2 +---------------------------------------- + +* There is no default browser indicator on Android. +* Add-ons include a *blocklistState* attribute, as returned by AddonManager. +* Searches are now version 4, and are hierarchical: how the search was started + (bartext, barkeyword, barsuggest), and then counts per provider. + +Version 2 +========= + +Version 2 is the same as version 1 with the exception that it has an additional +top-level field, *geckoAppInfo*, which contains basic application info. + +geckoAppInfo +------------ + +This field is an object that is a simple map of string keys and values +describing basic application metadata. It is very similar to the *appinfo* +measurement in the *last* section. The difference is this field is almost +certainly guaranteed to exist whereas the one in the data part of the +payload may be omitted in certain scenarios (such as catastrophic client +error). + +Its keys are as follows: + +appBuildID + The build ID/date of the application. e.g. "20130314113542". + +version + The value of nsXREAppData.version. This is the application's version. e.g. + "21.0.0". + +vendor + The value of nsXREAppData.vendor. Can be empty an empty string. For + official Mozilla builds, this will be "Mozilla". + +name + The value of nsXREAppData.name. For official Firefox builds, this + will be "Firefox". + +id + The value of nsXREAppData.ID. + +platformVersion + The version of the Gecko platform (as opposed to the app version). For + Firefox, this is almost certainly equivalent to the *version* field. + +platformBuildID + The build ID/date of the Gecko platfor (as opposed to the app version). + This is commonly equivalent to *appBuildID*. + +os + The name of the operating system the application is running on. + +xpcomabi + The binary architecture of the build. + +updateChannel + The name of the channel used for application updates. Official Mozilla + builds have one of the values {release, beta, aurora, nightly}. Local and + test builds have *default* as the channel. + +Version 1 +========= + +Top-level Properties +-------------------- + +The main JSON object contains the following properties: + +lastPingDate + UTC date of the last upload. If this is the first upload from this client, + this will not be present. + +thisPingDate + UTC date when this payload was constructed. + +version + Integer version of this payload format. Currently only 1 is defined. + +clientID + An identifier that identifies the client that is submitting data. + + This property may not be present in older clients. + + See :ref:`healthreport_identifiers` for more info on identifiers. + +clientIDVersion + Integer version associated with the generation semantics for the + ``clientID``. + + If the value is ``1``, ``clientID`` is a randomly-generated UUID. + + This property may not be present in older clients. + +data + Object holding data constituting health report. + +Data Properties +--------------- + +The bulk of the health report is contained within the *data* object. This +object has the following keys: + +days + Object mapping UTC days to measurements from that day. Keys are in the + *YYYY-MM-DD* format. e.g. "2013-03-14" + +last + Object mapping measurement names to their values. + + +The value of *days* and *last* are objects mapping measurement names to that +measurement's values. The values are always objects. Each object contains +a *_v* property. This property defines the version of this measurement. +Additional non-underscore-prefixed properties are defined by the measurement +itself (see sections below). + +Example +------- + +Here is an example JSON document for version 1:: + + { + "version": 1, + "thisPingDate": "2013-03-11", + "lastPingDate": "2013-03-10", + "data": { + "last": { + "org.mozilla.addons.active": { + "masspasswordreset@johnathan.nightingale": { + "userDisabled": false, + "appDisabled": false, + "version": "1.05", + "type": "extension", + "scope": 1, + "foreignInstall": false, + "hasBinaryComponents": false, + "installDay": 14973, + "updateDay": 15317 + }, + "places-maintenance@bonardo.net": { + "userDisabled": false, + "appDisabled": false, + "version": "1.3", + "type": "extension", + "scope": 1, + "foreignInstall": false, + "hasBinaryComponents": false, + "installDay": 15268, + "updateDay": 15379 + }, + "_v": 1 + }, + "org.mozilla.appInfo.appinfo": { + "_v": 1, + "appBuildID": "20130309030841", + "distributionID": "", + "distributionVersion": "", + "hotfixVersion": "", + "id": "{ec8030f7-c20a-464f-9b0e-13a3a9e97384}", + "locale": "en-US", + "name": "Firefox", + "os": "Darwin", + "platformBuildID": "20130309030841", + "platformVersion": "22.0a1", + "updateChannel": "nightly", + "vendor": "Mozilla", + "version": "22.0a1", + "xpcomabi": "x86_64-gcc3" + }, + "org.mozilla.profile.age": { + "_v": 1, + "profileCreation": 12444 + }, + "org.mozilla.appSessions.current": { + "_v": 3, + "startDay": 15773, + "activeTicks": 522, + "totalTime": 70858, + "main": 1245, + "firstPaint": 2695, + "sessionRestored": 3436 + }, + "org.mozilla.sysinfo.sysinfo": { + "_v": 1, + "cpuCount": 8, + "memoryMB": 16384, + "architecture": "x86-64", + "name": "Darwin", + "version": "12.2.1" + } + }, + "days": { + "2013-03-11": { + "org.mozilla.addons.counts": { + "_v": 1, + "extension": 15, + "plugin": 12, + "theme": 1 + }, + "org.mozilla.places.places": { + "_v": 1, + "bookmarks": 757, + "pages": 104858 + }, + "org.mozilla.appInfo.appinfo": { + "_v": 1, + "isDefaultBrowser": 1 + } + }, + "2013-03-10": { + "org.mozilla.addons.counts": { + "_v": 1, + "extension": 15, + "plugin": 12, + "theme": 1 + }, + "org.mozilla.places.places": { + "_v": 1, + "bookmarks": 757, + "pages": 104857 + }, + "org.mozilla.searches.counts": { + "_v": 1, + "google.urlbar": 4 + }, + "org.mozilla.appInfo.appinfo": { + "_v": 1, + "isDefaultBrowser": 1 + } + } + } + } + } + +Measurements +============ + +The bulk of payloads consists of measurement data. An individual measurement +is merely a collection of related values e.g. *statistics about the Places +database* or *system information*. + +Each measurement has an integer version number attached. When the fields in +a measurement or the semantics of data within that measurement change, the +version number is incremented. + +All measurements are defined alphabetically in the sections below. + +org.mozilla.addons.addons +------------------------- + +This measurement contains information about the currently-installed add-ons. + +Version 2 +^^^^^^^^^ + +This version adds the human-readable fields *name* and *description*, both +coming directly from the Addon instance as most properties in version 1. +Also, all plugin details are now in org.mozilla.addons.plugins. + +Version 1 +^^^^^^^^^ + +The measurement object is a mapping of add-on IDs to objects containing +add-on metadata. + +Each add-on contains the following properties: + +* userDisabled +* appDisabled +* version +* type +* scope +* foreignInstall +* hasBinaryComponents +* installDay +* updateDay + +With the exception of *installDay* and *updateDay*, all these properties +come direct from the Addon instance. See https://developer.mozilla.org/en-US/docs/Addons/Add-on_Manager/Addon. +*installDay* and *updateDay* are the number of days since UNIX epoch of +the add-ons *installDate* and *updateDate* properties, respectively. + +Notes +^^^^^ + +Add-ons that have opted out of AMO updates via the +*extensions._id_.getAddons.cache.enabled* preference are, since Bug 868306 +(Firefox 24), included in the list of submitted add-ons. + +Example +^^^^^^^ +:: + + "org.mozilla.addons.addons": { + "_v": 2, + "{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}": { + "userDisabled": false, + "appDisabled": false, + "name": "Adblock Plus", + "version": "2.4.1", + "type": "extension", + "scope": 1, + "description": "Ads were yesterday!", + "foreignInstall": false, + "hasBinaryComponents": false, + "installDay": 16093, + "updateDay": 16093 + }, + "{e4a8a97b-f2ed-450b-b12d-ee082ba24781}": { + "userDisabled": true, + "appDisabled": false, + "name": "Greasemonkey", + "version": "1.14", + "type": "extension", + "scope": 1, + "description": "A User Script Manager for Firefox", + "foreignInstall": false, + "hasBinaryComponents": false, + "installDay": 16093, + "updateDay": 16093 + } + } + +org.mozilla.addons.plugins +-------------------------- + +This measurement contains information about the currently-installed plugins. + +Version 1 +^^^^^^^^^ + +The measurement object is a mapping of plugin IDs to objects containing +plugin metadata. + +The plugin ID is constructed of the plugins filename, name, version and +description. Every plugin has at least a filename and a name. + +Each plugin contains the following properties: + +* name +* version +* description +* blocklisted +* disabled +* clicktoplay +* mimeTypes +* updateDay + +With the exception of *updateDay* and *mimeTypes*, all these properties come +directly from ``nsIPluginTag`` via ``nsIPluginHost``. +*updateDay* is the number of days since UNIX epoch of the plugins last modified +time. +*mimeTypes* is the list of mimetypes the plugin supports, see +``nsIPluginTag.getMimeTypes()``. + +Example +^^^^^^^ + +:: + + "org.mozilla.addons.plugins": { + "_v": 1, + "Flash Player.plugin:Shockwave Flash:12.0.0.38:Shockwave Flash 12.0 r0": { + "mimeTypes": [ + "application/x-shockwave-flash", + "application/futuresplash" + ], + "name": "Shockwave Flash", + "version": "12.0.0.38", + "description": "Shockwave Flash 12.0 r0", + "blocklisted": false, + "disabled": false, + "clicktoplay": false + }, + "Default Browser.plugin:Default Browser Helper:537:Provides information about the default web browser": { + "mimeTypes": [ + "application/apple-default-browser" + ], + "name": "Default Browser Helper", + "version": "537", + "description": "Provides information about the default web browser", + "blocklisted": false, + "disabled": true, + "clicktoplay": false + } + } + +org.mozilla.addons.counts +------------------------- + +This measurement contains information about historical add-on counts. + +Version 1 +^^^^^^^^^ + +The measurement object consists of counts of different add-on types. The +properties are: + +extension + Integer count of installed extensions. +plugin + Integer count of installed plugins. +theme + Integer count of installed themes. +lwtheme + Integer count of installed lightweigh themes. + +Notes +^^^^^ + +Add-ons opted out of AMO updates are included in the counts. This differs from +the behavior of the active add-ons measurement. + +If no add-ons of a particular type are installed, the property for that type +will not be present (as opposed to an explicit property with value of 0). + +Example +^^^^^^^ + +:: + + "2013-03-14": { + "org.mozilla.addons.counts": { + "_v": 1, + "extension": 21, + "plugin": 4, + "theme": 1 + } + } + + + +org.mozilla.appInfo.appinfo +--------------------------- + +This measurement contains basic XUL application and Gecko platform +information. It is reported in the *last* section. + +Version 2 +^^^^^^^^^ + +In addition to fields present in version 1, this version has the following +fields appearing in the *days* section: + +isBlocklistEnabled + Whether the blocklist ping is enabled. This is an integer, 0 or 1. + This does not indicate whether the blocklist ping was sent but merely + whether the application will try to send the blocklist ping. + +isTelemetryEnabled + Whether Telemetry is enabled. This is an integer, 0 or 1. + +Version 1 +^^^^^^^^^ + +The measurement object contains mostly string values describing the +current application and build. The properties are: + +* vendor +* name +* id +* version +* appBuildID +* platformVersion +* platformBuildID +* os +* xpcomabi +* updateChannel +* distributionID +* distributionVersion +* hotfixVersion +* locale +* isDefaultBrowser + +Notes +^^^^^ + +All of the properties appear in the *last* section except for +*isDefaultBrowser*, which appears under *days*. + +Example +^^^^^^^ + +This example comes from an official OS X Nightly build:: + + "org.mozilla.appInfo.appinfo": { + "_v": 1, + "appBuildID": "20130311030946", + "distributionID": "", + "distributionVersion": "", + "hotfixVersion": "", + "id": "{ec8030f7-c20a-464f-9b0e-13a3a9e97384}", + "locale": "en-US", + "name": "Firefox", + "os": "Darwin", + "platformBuildID": "20130311030946", + "platformVersion": "22.0a1", + "updateChannel": "nightly", + "vendor": "Mozilla", + "version": "22.0a1", + "xpcomabi": "x86_64-gcc3" + }, + +org.mozilla.appInfo.update +-------------------------- + +This measurement contains information about the application update mechanism +in the application. + +Version 1 +^^^^^^^^^ + +The following daily values are reported: + +enabled + Whether automatic application update checking is enabled. 1 for yes, + 0 for no. +autoDownload + Whether automatic download of available updates is enabled. + +Notes +^^^^^ + +This measurement was merged to mozilla-central for JS FHR on 2013-07-15. + +Example +^^^^^^^ + +:: + + "2013-07-15": { + "org.mozilla.appInfo.update": { + "_v": 1, + "enabled": 1, + "autoDownload": 1, + } + } + +org.mozilla.appInfo.versions +---------------------------- + +This measurement contains a history of application version numbers. + +Version 2 +^^^^^^^^^ + +Version 2 reports more fields than version 1 and is not backwards compatible. +The following fields are present in version 2: + +appVersion + An array of application version strings. +appBuildID + An array of application build ID strings. +platformVersion + An array of platform version strings. +platformBuildID + An array of platform build ID strings. + +When the application is upgraded, the new version and/or build IDs are +appended to their appropriate fields. + +Version 1 +^^^^^^^^^ + +When the application version (*version* from *org.mozilla.appinfo.appinfo*) +changes, we record the new version on the day the change was seen. The new +versions for a day are recorded in an array under the *version* property. + +Notes +^^^^^ + +If the application isn't upgraded, this measurement will not be present. +This means this measurement will not be present for most days if a user is +on the release channel (since updates are typically released every 6 weeks). +However, users on the Nightly and Aurora channels will likely have a lot +of these entries since those builds are updated every day. + +Values for this measurement are collected when performing the daily +collection (typically occurs at upload time). As a result, it's possible +the actual upgrade day may not be attributed to the proper day - the +reported day may lag behind. + +The app and platform versions and build IDs should be identical for most +clients. If they are different, we are possibly looking at a *Frankenfox*. + +Example +^^^^^^^ + +:: + + "2013-03-27": { + "org.mozilla.appInfo.versions": { + "_v": 2, + "appVersion": [ + "22.0.0" + ], + "appBuildID": [ + "20130325031100" + ], + "platformVersion": [ + "22.0.0" + ], + "platformBuildID": [ + "20130325031100" + ] + } + } + +org.mozilla.appSessions.current +------------------------------- + +This measurement contains information about the currently running XUL +application's session. + +Version 3 +^^^^^^^^^ + +This measurement has the following properties: + +startDay + Integer days since UNIX epoch when this session began. +activeTicks + Integer count of *ticks* the session was active for. Gecko periodically + sends out a signal when the session is active. Session activity + involves keyboard or mouse interaction with the application. Each tick + represents a window of 5 seconds where there was interaction. +totalTime + Integer seconds the session has been alive. +main + Integer milliseconds it took for the Gecko process to start up. +firstPaint + Integer milliseconds from process start to first paint. +sessionRestored + Integer milliseconds from process start to session restore. + +Example +^^^^^^^ + +:: + + "org.mozilla.appSessions.current": { + "_v": 3, + "startDay": 15775, + "activeTicks": 4282, + "totalTime": 249422, + "main": 851, + "firstPaint": 3271, + "sessionRestored": 5998 + } + +org.mozilla.appSessions.previous +-------------------------------- + +This measurement contains information about previous XUL application sessions. + +Version 3 +^^^^^^^^^ + +This measurement contains per-day lists of all the sessions started on that +day. The following properties may be present: + +cleanActiveTicks + Active ticks of sessions that were properly shut down. +cleanTotalTime + Total number of seconds for sessions that were properly shut down. +abortedActiveTicks + Active ticks of sessions that were not properly shut down. +abortedTotalTime + Total number of seconds for sessions that were not properly shut down. +main + Time in milliseconds from process start to main process initialization. +firstPaint + Time in milliseconds from process start to first paint. +sessionRestored + Time in milliseconds from process start to session restore. + +Notes +^^^^^ + +Sessions are recorded on the date on which they began. + +If a session was aborted/crashed, the total time may be less than the actual +total time. This is because we don't always update total time during periods +of inactivity and the abort/crash could occur after a long period of idle, +before we've updated the total time. + +The lengths of the arrays for {cleanActiveTicks, cleanTotalTime}, +{abortedActiveTicks, abortedTotalTime}, and {main, firstPaint, sessionRestored} +should all be identical. + +The length of the clean sessions plus the length of the aborted sessions should +be equal to the length of the {main, firstPaint, sessionRestored} properties. + +It is not possible to distinguish the main, firstPaint, and sessionRestored +values from a clean vs aborted session: they are all lumped together. + +For sessions spanning multiple UTC days, it's not possible to know which +days the session was active for. It's possible a week long session only +had activity for 2 days and there's no way for us to tell which days. + +Example +^^^^^^^ + +:: + + "org.mozilla.appSessions.previous": { + "_v": 3, + "cleanActiveTicks": [ + 78, + 1785 + ], + "cleanTotalTime": [ + 4472, + 88908 + ], + "main": [ + 32, + 952 + ], + "firstPaint": [ + 2755, + 3497 + ], + "sessionRestored": [ + 5149, + 5520 + ] + } + +org.mozilla.crashes.crashes +--------------------------- + +This measurement contains a historical record of application crashes. + +Version 6 +^^^^^^^^^ + +This version adds tracking for out-of-memory (OOM) crashes in the main process. +An OOM crash will be counted as both main-crash and main-crash-oom. + +This measurement will be reported on each day there was a crash or crash +submission. Records may contain the following fields, whose values indicate +the number of crashes, hangs, or submissions that occurred on the given day: + +* content-crash +* content-crash-submission-succeeded +* content-crash-submission-failed +* content-hang +* content-hang-submission-succeeded +* content-hang-submission-failed +* gmplugin-crash +* gmplugin-crash-submission-succeeded +* gmplugin-crash-submission-failed +* main-crash +* main-crash-oom +* main-crash-submission-succeeded +* main-crash-submission-failed +* main-hang +* main-hang-submission-succeeded +* main-hang-submission-failed +* plugin-crash +* plugin-crash-submission-succeeded +* plugin-crash-submission-failed +* plugin-hang +* plugin-hang-submission-succeeded +* plugin-hang-submission-failed + +Version 5 +^^^^^^^^^ + +This version adds support for Gecko media plugin (GMP) crashes. + +This measurement will be reported on each day there was a crash or crash +submission. Records may contain the following fields, whose values indicate +the number of crashes, hangs, or submissions that occurred on the given day: + +* content-crash +* content-crash-submission-succeeded +* content-crash-submission-failed +* content-hang +* content-hang-submission-succeeded +* content-hang-submission-failed +* gmplugin-crash +* gmplugin-crash-submission-succeeded +* gmplugin-crash-submission-failed +* main-crash +* main-crash-submission-succeeded +* main-crash-submission-failed +* main-hang +* main-hang-submission-succeeded +* main-hang-submission-failed +* plugin-crash +* plugin-crash-submission-succeeded +* plugin-crash-submission-failed +* plugin-hang +* plugin-hang-submission-succeeded +* plugin-hang-submission-failed + +Version 4 +^^^^^^^^^ + +This version follows up from version 3, adding submissions which are now +tracked by the :ref:`crashes_crashmanager`. + +This measurement will be reported on each day there was a crash or crash +submission. Records may contain the following fields, whose values indicate +the number of crashes, hangs, or submissions that occurred on the given day: + +* main-crash +* main-crash-submission-succeeded +* main-crash-submission-failed +* main-hang +* main-hang-submission-succeeded +* main-hang-submission-failed +* content-crash +* content-crash-submission-succeeded +* content-crash-submission-failed +* content-hang +* content-hang-submission-succeeded +* content-hang-submission-failed +* plugin-crash +* plugin-crash-submission-succeeded +* plugin-crash-submission-failed +* plugin-hang +* plugin-hang-submission-succeeded +* plugin-hang-submission-failed + +Version 3 +^^^^^^^^^ + +This version follows up from version 2, building on improvements to +the :ref:`crashes_crashmanager`. + +This measurement will be reported on each day there was a +crash. Records may contain the following fields, whose values indicate +the number of crashes or hangs that occurred on the given day: + +* main-crash +* main-hang +* content-crash +* content-hang +* plugin-crash +* plugin-hang + +Version 2 +^^^^^^^^^ + +The switch to version 2 coincides with the introduction of the +:ref:`crashes_crashmanager`, which provides a more robust source of +crash data. + +This measurement will be reported on each day there was a crash. The +following fields may be present in each record: + +mainCrash + The number of main process crashes that occurred on the given day. + +Yes, version 2 does not track submissions like version 1. It is very +likely submissions will be re-added later. + +Also absent from version 2 are plugin crashes and hangs. These will be +re-added, likely in version 3. + +Version 1 +^^^^^^^^^ + +This measurement will be reported on each day there was a crash. The +following properties are reported: + +pending + The number of crash reports that haven't been submitted. +submitted + The number of crash reports that were submitted. + +Notes +^^^^^ + +Main process crashes are typically submitted immediately after they +occur (by checking a box in the crash reporter, which should appear +automatically after a crash). If the crash reporter submits the crash +successfully, we get a submitted crash. Else, we leave it as pending. + +A pending crash does not mean it will eventually be submitted. + +Pending crash reports can be submitted post-crash by going to +about:crashes. + +If a pending crash is submitted via about:crashes, the submitted count +increments but the pending count does not decrement. This is because FHR +does not know which pending crash was just submitted and therefore it does +not know which day's pending crash to decrement. + +Example +^^^^^^^ + +:: + + "org.mozilla.crashes.crashes": { + "_v": 1, + "pending": 1, + "submitted": 2 + }, + "org.mozilla.crashes.crashes": { + "_v": 2, + "mainCrash": 2 + } + "org.mozilla.crashes.crashes": { + "_v": 4, + "main-crash": 2, + "main-crash-submission-succeeded": 1, + "main-crash-submission-failed": 1, + "main-hang": 1, + "plugin-crash": 2 + } + +org.mozilla.healthreport.submissions +------------------------------------ + +This measurement contains a history of FHR's own data submission activity. +It was added in Firefox 23 in early May 2013. + +Version 2 +^^^^^^^^^ + +This is the same as version 1 except an additional field has been added. + +uploadAlreadyInProgress + A request for upload was initiated while another upload was in progress. + This should not occur in well-behaving clients. It (along with a lock + preventing simultaneous upload) was added to ensure this never occurs. + +Version 1 +^^^^^^^^^ + +Daily counts of upload events are recorded. + +firstDocumentUploadAttempt + An attempt was made to upload the client's first document to the server. + These are uploads where the client is not aware of a previous document ID + on the server. Unless the client had disabled upload, there should be at + most one of these in the history of the client. + +continuationUploadAttempt + An attempt was made to upload a document that replaces an existing document + on the server. Most upload attempts should be attributed to this as opposed + to *firstDocumentUploadAttempt*. + +uploadSuccess + The upload attempt recorded by *firstDocumentUploadAttempt* or + *continuationUploadAttempt* was successful. + +uploadTransportFailure + An upload attempt failed due to transport failure (network unavailable, + etc). + +uploadServerFailure + An upload attempt failed due to a server-reported failure. Ideally these + are failures reported by the FHR server itself. However, intermediate + proxies, firewalls, etc may trigger this depending on how things are + configured. + +uploadClientFailure + An upload attempt failued due to an error/exception in the client. + This almost certainly points to a bug in the client. + +The result for an upload attempt is always attributed to the same day as +the attempt, even if the result occurred on a different day from the attempt. +Therefore, the sum of the result counts should equal the result of the attempt +counts. + +org.mozilla.hotfix.update +------------------------- + +This measurement contains results from the Firefox update hotfix. + +The Firefox update hotfix bypasses the built-in application update mechanism +and installs a modern Firefox. + +Version 1 +^^^^^^^^^ + +The fields in this measurement are dynamically created based on which +versions of the update hotfix state file are found on disk. + +The general format of the fields is ``<version>.<thing>`` where ``version`` +is a hotfix version like ``v20140527`` and ``thing`` is a key from the +hotfix state file, e.g. ``upgradedFrom``. Here are some of the ``things`` +that can be defined. + +upgradedFrom + String identifying the Firefox version that the hotfix upgraded from. + e.g. ``16.0`` or ``17.0.1``. + +uninstallReason + String with enumerated values identifying why the hotfix was uninstalled. + Value will be ``STILL_INSTALLED`` if the hotfix is still installed. + +downloadAttempts + Integer number of times the hotfix started downloading an installer. + Download resumes are part of this count. + +downloadFailures + Integer count of times a download supposedly completed but couldn't + be validated. This likely represents something wrong with the network + connection. The ratio of this to ``downloadAttempts`` should be low. + +installAttempts + Integer count of times the hotfix attempted to run the installer. + This should ideally be 1. It should only be greater than 1 if UAC + elevation was cancelled or not allowed. + +installFailures + Integer count of total installation failures this client experienced. + Can be 0. ``installAttempts - installFailures`` implies install successes. + +notificationsShown + Integer count of times a notification was displayed to the user that + they are running an older Firefox. + +org.mozilla.places.places +------------------------- + +This measurement contains information about the Places database (where Firefox +stores its history and bookmarks). + +Version 1 +^^^^^^^^^ + +Daily counts of items in the database are reported in the following properties: + +bookmarks + Integer count of bookmarks present. +pages + Integer count of pages in the history database. + +Example +^^^^^^^ + +:: + + "org.mozilla.places.places": { + "_v": 1, + "bookmarks": 388, + "pages": 94870 + } + +org.mozilla.profile.age +----------------------- + +This measurement contains information about the current profile's age (and +in version 2, the profile's most recent reset date) + +Version 2 +^^^^^^^^^ + +*profileCreation* and *profileReset* properties are present. Both define +the integer days since UNIX epoch that the current profile was created or +reset accordingly. + +Version 1 +^^^^^^^^^ + +A single *profileCreation* property is present. It defines the integer +days since UNIX epoch that the current profile was created. + +Notes +^^^^^ + +It is somewhat difficult to obtain a reliable *profile born date* due to a +number of factors, but since Version 2, improvements have been made - on a +"profile reset" we copy the profileCreation date from the old profile and +record the time of the reset in profileReset. + +Example +^^^^^^^ + +:: + + "org.mozilla.profile.age": { + "_v": 2, + "profileCreation": 15176 + "profileReset": 15576 + } + +org.mozilla.searches.counts +--------------------------- + +This measurement contains information about searches performed in the +application. + +Version 6 (mobile) +^^^^^^^^^^^^^^^^^^ + +This adds two new search locations: *widget* and *activity*, corresponding to the search widget and search activity respectively. + +Version 2 +^^^^^^^^^ + +This behaves like version 1 except we added all search engines that +Mozilla has a partner agreement with. Like version 1, we concatenate +a search engine ID with a search origin. + +Another difference with version 2 is we should no longer misattribute +a search to the *other* bucket if the search engine name is localized. + +The set of search engine providers is: + +* amazon-co-uk +* amazon-de +* amazon-en-GB +* amazon-france +* amazon-it +* amazon-jp +* amazondotcn +* amazondotcom +* amazondotcom-de +* aol-en-GB +* aol-web-search +* bing +* eBay +* eBay-de +* eBay-en-GB +* eBay-es +* eBay-fi +* eBay-france +* eBay-hu +* eBay-in +* eBay-it +* google +* google-jp +* google-ku +* google-maps-zh-TW +* mailru +* mercadolibre-ar +* mercadolibre-cl +* mercadolibre-mx +* seznam-cz +* twitter +* twitter-de +* twitter-ja +* yahoo +* yahoo-NO +* yahoo-answer-zh-TW +* yahoo-ar +* yahoo-bid-zh-TW +* yahoo-br +* yahoo-ch +* yahoo-cl +* yahoo-de +* yahoo-en-GB +* yahoo-es +* yahoo-fi +* yahoo-france +* yahoo-fy-NL +* yahoo-id +* yahoo-in +* yahoo-it +* yahoo-jp +* yahoo-jp-auctions +* yahoo-mx +* yahoo-sv-SE +* yahoo-zh-TW +* yandex +* yandex-ru +* yandex-slovari +* yandex-tr +* yandex.by +* yandex.ru-be + +And of course, *other*. + +The sources for searches remain: + +* abouthome +* contextmenu +* searchbar +* urlbar + +The measurement will only be populated with providers and sources that +occurred that day. + +If a user switches locales, searches from default providers on the older +locale will still be supported. However, if that same search engine is +added by the user to the new build and is *not* a default search engine +provider, its searches will be attributed to the *other* bucket. + +Version 1 +^^^^^^^^^ + +We record counts of performed searches grouped by search engine and search +origin. Only search engines with which Mozilla has a business relationship +are explicitly counted. All other search engines are grouped into an +*other* bucket. + +The following search engines are explicitly counted: + +* Amazon.com +* Bing +* Google +* Yahoo +* Other + +The following search origins are distinguished: + +about:home + Searches initiated from the search text box on about:home. +context menu + Searches initiated from the context menu (highlight text, right click, + and select "search for...") +search bar + Searches initiated from the search bar (the text field next to the + Awesomebar) +url bar + Searches initiated from the awesomebar/url bar. + +Due to the localization of search engine names, non en-US locales may wrongly +attribute searches to the *other* bucket. This is fixed in version 2. + +Example +^^^^^^^ + +:: + + "org.mozilla.searches.counts": { + "_v": 1, + "google.searchbar": 3, + "google.urlbar": 7 + }, + +org.mozilla.searches.engines +---------------------------- + +This measurement contains information about search engines. + +Version 1 +^^^^^^^^^ + +This version debuted with Firefox 31 on desktop. It contains the +following properties: + +default + Daily string identifier or name of the default search engine provider. + + This field will only be collected if Telemetry is enabled. If + Telemetry is enabled and then later disabled, this field may + disappear from future days in the payload. + + The special value ``NONE`` could occur if there is no default search + engine. + + The special value ``UNDEFINED`` could occur if a default search + engine exists but its identifier could not be determined. + + This field's contents are + ``Services.search.defaultEngine.identifier`` (if defined) or + ``"other-"`` + ``Services.search.defaultEngine.name`` if not. + In other words, search engines without an ``.identifier`` + are prefixed with ``other-``. + +Version 2 +^^^^^^^^^ + +Starting with Firefox 40, there is an additional optional value: + +cohort + Daily cohort string identifier, recorded if the user is part of + search defaults A/B testing. + +org.mozilla.sync.sync +--------------------- + +This daily measurement contains information about the Sync service. + +Values should be recorded for every day FHR measurements occurred. + +Version 1 +^^^^^^^^^ + +This version debuted with Firefox 30 on desktop. It contains the following +properties: + +enabled + Daily numeric indicating whether Sync is configured and enabled. 1 if so, + 0 otherwise. + +preferredProtocol + String version of the maximum Sync protocol version the client supports. + This will be ``1.1`` for for legacy Sync and ``1.5`` for clients that + speak the Firefox Accounts protocol. + +actualProtocol + The actual Sync protocol version the client is configured to use. + + This will be ``1.1`` if the client is configured with the legacy Sync + service or if the client only supports ``1.1``. + + It will be ``1.5`` if the client supports ``1.5`` and either a) the + client is not configured b) the client is using Firefox Accounts Sync. + +syncStart + Count of sync operations performed. + +syncSuccess + Count of sync operations that completed successfully. + +syncError + Count of sync operations that did not complete successfully. + + This is a measure of overall sync success. This does *not* reflect + recoverable errors (such as record conflict) that can occur during + sync. This is thus a rough proxy of whether the sync service is + operating without error. + +org.mozilla.sync.devices +------------------------ + +This daily measurement contains information about the device type composition +for the configured Sync account. + +Version 1 +^^^^^^^^^ + +Version 1 was introduced with Firefox 30. + +Field names are dynamic according to the client-reported device types from +Sync records. All fields are daily last seen integer values corresponding to +the number of devices of that type. + +Common values include: + +desktop + Corresponds to a Firefox desktop client. + +mobile + Corresponds to a Fennec client. + +org.mozilla.sync.migration +-------------------------- + +This daily measurement contains information about sync migration (that is, the +semi-automated process of migrating a legacy sync account to an FxA account.) + +Measurements will start being recorded after a migration is offered by the +sync server and stop after migration is complete or the user elects to "unlink" +their sync account. In other words, it is expected that users with Sync setup +for FxA or with sync unconfigured will not collect data, and that for users +where data is collected, the collection will only be for a relatively short +period. + +Version 1 +^^^^^^^^^ + +Version 1 was introduced with Firefox 37 and includes the following properties: + +state + Corresponds to either a STATE_USER_* string or a STATE_INTERNAL_* string in + FxaMigration.jsm. This reflects a state where we are waiting for the user, + or waiting for some internal process to complete on the way to completing + the migration. + +declined + Corresponds to the number of times the user closed the migration infobar. + +unlinked + Set if the user declined to migrate and instead "unlinked" Sync from the + browser. + +accepted + Corresponds to the number of times the user explicitly elected to start or + continue the migration - it counts how often the user clicked on any UI + created specifically for migration. The "ideal" UX for migration would see + this at exactly 1, some known edge-cases (eg, browser restart required to + finish) could expect this to be 2, and anything more means we are doing + something wrong. + +org.mozilla.sysinfo.sysinfo +--------------------------- + +This measurement contains basic information about the system the application +is running on. + +Version 2 +^^^^^^^^^ + +This version debuted with Firefox 29 on desktop. + +A single property was introduced. + +isWow64 + If present, this property indicates whether the machine supports WoW64. + This property can be used to identify whether the host machine is 64-bit. + + This property is only present on Windows machines. It is the preferred way + to identify 32- vs 64-bit support in that environment. + +Version 1 +^^^^^^^^^ + +The following properties may be available: + +cpuCount + Integer number of CPUs/cores in the machine. +memoryMB + Integer megabytes of memory in the machine. +manufacturer + The manufacturer of the device. +device + The name of the device (like model number). +hardware + Unknown. +name + OS name. +version + OS version. +architecture + OS architecture that the application is built for. This is not the + actual system architecture. + +Example +^^^^^^^ + +:: + + "org.mozilla.sysinfo.sysinfo": { + "_v": 1, + "cpuCount": 8, + "memoryMB": 8192, + "architecture": "x86-64", + "name": "Darwin", + "version": "12.2.0" + } + + +org.mozilla.translation.translation +----------------------------------- + +This daily measurement contains information about the usage of the translation +feature. It is a special telemetry measurement which will only be recorded in +FHR if telemetry is enabled. + +Version 1 +^^^^^^^^^ + +Daily counts are reported in the following properties: + +translationOpportunityCount + Integer count of the number of opportunities there were to translate a page. +missedTranslationOpportunityCount + Integer count of the number of missed opportunities there were to translate a page. + A missed opportunity is when the page language is not supported by the translation + provider. +pageTranslatedCount + Integer count of the number of pages translated. +charactersTranslatedCount + Integer count of the number of characters translated. +detectedLanguageChangedBefore + Integer count of the number of times the user manually adjusted the detected + language before translating. +detectedLanguageChangedAfter + Integer count of the number of times the user manually adjusted the detected + language after having first translated the page. +targetLanguageChanged + Integer count of the number of times the user manually adjusted the target + language. +deniedTranslationOffer + Integer count of the number of times the user opted-out offered + page translation, either by the Not Now button or by the notification's + close button in the "offer" state. +autoRejectedTranlationOffer + Integer count of the number of times the user is not offered page + translation because they had previously clicked "Never translate this + language" or "Never translate this site". +showOriginalContent + Integer count of the number of times the user activated the Show Original + command. + +Additional daily counts broken down by language are reported in the following +properties: + +translationOpportunityCountsByLanguage + A mapping from language to count of opportunities to translate that + language. +missedTranslationOpportunityCountsByLanguage + A mapping from language to count of missed opportunities to translate that + language. +pageTranslatedCountsByLanguage + A mapping from language to the counts of pages translated from that + language. Each language entry will be an object containing a "total" member + along with individual counts for each language translated to. + +Other properties: + +detectLanguageEnabled + Whether automatic language detection is enabled. This is an integer, 0 or 1. +showTranslationUI + Whether the translation feature UI will be shown. This is an integer, 0 or 1. + +Example +^^^^^^^ + +:: + + "org.mozilla.translation.translation": { + "_v": 1, + "detectLanguageEnabled": 1, + "showTranslationUI": 1, + "translationOpportunityCount": 134, + "missedTranslationOpportunityCount": 32, + "pageTranslatedCount": 6, + "charactersTranslatedCount": "1126", + "detectedLanguageChangedBefore": 1, + "detectedLanguageChangedAfter": 2, + "targetLanguageChanged": 0, + "deniedTranslationOffer": 3, + "autoRejectedTranlationOffer": 1, + "showOriginalContent": 2, + "translationOpportunityCountsByLanguage": { + "fr": 100, + "es": 34 + }, + "missedTranslationOpportunityCountsByLanguage": { + "it": 20, + "nl": 10, + "fi": 2 + }, + "pageTranslatedCountsByLanguage": { + "fr": { + "total": 6, + "es": 5, + "en": 1 + } + } + } + + +org.mozilla.experiments.info +---------------------------------- + +Daily measurement reporting information about the Telemetry Experiments service. + +Version 1 +^^^^^^^^^ + +Property: + +lastActive + ID of the final Telemetry Experiment that is active on a given day, if any. + + +Version 2 +^^^^^^^^^ + +Adds an additional optional property: + +lastActiveBranch + If the experiment uses branches, the branch identifier string. + +Example +^^^^^^^ + +:: + + "org.mozilla.experiments.info": { + "_v": 2, + "lastActive": "some.experiment.id", + "lastActiveBranch": "control" + } + +org.mozilla.uitour.treatment +---------------------------- + +Daily measurement reporting information about treatment tagging done +by the UITour module. + +Version 1 +^^^^^^^^^ + +Daily text values in the following properties: + +<tag>: + Array of discrete strings corresponding to calls for setTreatmentTag(tag, value). + +Example +^^^^^^^ + +:: + + "org.mozilla.uitour.treatment": { + "_v": 1, + "treatment": [ + "optin", + "optin-DNT" + ], + "another-tag": [ + "foobar-value" + ] + } + +org.mozilla.passwordmgr.passwordmgr +----------------------------------- + +Daily measurement reporting information about the Password Manager + +Version 1 +^^^^^^^^^ + +Property: + +numSavedPasswords + number of passwords saved in the Password Manager + +enabled + Whether or not the user has disabled the Password Manager in prefernces + +Example +^^^^^^^ + +:: + + "org.mozilla.passwordmgr.passwordmgr": { + "_v": 1, + "numSavedPasswords": 5, + "enabled": 0, + } + +Version 2 +^^^^^^^^^ + +More detailed measurements of login forms & their behavior + +numNewSavedPasswordsInSession + Number of passwords saved to the password manager this session. + +numSuccessfulFills + Number of times the password manager filled in password fields for user this session. + +numTotalLoginsEncountered + Number of times a login form was encountered by the user in the session. + +Example +^^^^^^^ + +:: + "org.mozilla.passwordmgr.passwordmgr": { + "_v": 2, + "numSavedPasswords": 32, + "enabled": 1, + "numNewSavedPasswords": 5, + "numSuccessfulFills": 11, + "numTotalLoginsEncountered": 23, + } diff --git a/toolkit/components/telemetry/docs/fhr/identifiers.rst b/toolkit/components/telemetry/docs/fhr/identifiers.rst new file mode 100644 index 000000000..82ad0e49e --- /dev/null +++ b/toolkit/components/telemetry/docs/fhr/identifiers.rst @@ -0,0 +1,83 @@ +.. _healthreport_identifiers: + +=========== +Identifiers +=========== + +Firefox Health Report records some identifiers to keep track of clients +and uploaded documents. + +Identifier Types +================ + +Document/Upload IDs +------------------- + +A random UUID called the *Document ID* or *Upload ID* is generated when the FHR +client creates or uploads a new document. + +When clients generate a new *Document ID*, they persist this ID to disk +**before** the upload attempt. + +As part of the upload, the client sends all old *Document IDs* to the server +and asks the server to delete them. In well-behaving clients, the server +has a single record for each client with a randomly-changing *Document ID*. + +Client IDs +---------- + +A *Client ID* is an identifier that **attempts** to uniquely identify an +individual FHR client. Please note the emphasis on *attempts* in that last +sentence: *Client IDs* do not guarantee uniqueness. + +The *Client ID* is generated when the client first runs or as needed. + +The *Client ID* is transferred to the server as part of every upload. The +server is thus able to affiliate multiple document uploads with a single +*Client ID*. + +Client ID Versions +^^^^^^^^^^^^^^^^^^ + +The semantics for how a *Client ID* is generated are versioned. + +Version 1 + The *Client ID* is a randomly-generated UUID. + +History of Identifiers +====================== + +In the beginning, there were just *Document IDs*. The thinking was clients +would clean up after themselves and leave at most 1 active document on the +server. + +Unfortunately, this did not work out. Using brute force analysis to +deduplicate records on the server, a number of interesting patterns emerged. + +Orphaning + Clients would upload a new payload while not deleting the old payload. + +Divergent records + Records would share data up to a certain date and then the data would + almost completely diverge. This appears to be indicative of profile + copying. + +Rollback + Records would share data up to a certain date. Each record in this set + would contain data for a day or two but no extra data. This could be + explained by filesystem rollback on the client. + +A significant percentage of the records on the server belonged to +misbehaving clients. Identifying these records was extremely resource +intensive and error-prone. These records were undermining the ability +to use Firefox Health Report data. + +Thus, the *Client ID* was born. The intent of the *Client ID* was to +uniquely identify clients so the extreme effort required and the +questionable reliability of deduplicating server data would become +problems of the past. + +The *Client ID* was originally a randomly-generated UUID (version 1). This +allowed detection of orphaning and rollback. However, these version 1 +*Client IDs* were still susceptible to use on multiple profiles and +machines if the profile was copied. diff --git a/toolkit/components/telemetry/docs/fhr/index.rst b/toolkit/components/telemetry/docs/fhr/index.rst new file mode 100644 index 000000000..497385dd8 --- /dev/null +++ b/toolkit/components/telemetry/docs/fhr/index.rst @@ -0,0 +1,34 @@ +================================ +Firefox Health Report (Obsolete) +================================ + +**Firefox Health Report (FHR) is obsolete and no longer ships with Firefox. +This documentation will live here for a few more cycles.** + +Firefox Health Report is a background service that collects application +metrics and periodically submits them to a central server. The core +parts of the service are implemented in this directory. However, the +actual XPCOM service is implemented in the +``data_reporting_service`. + +The core types can actually be instantiated multiple times and used to +power multiple data submission services within a single Gecko +application. In other words, everything in this directory is effectively +a reusable library. However, the terminology and some of the features +are very specific to what the Firefox Health Report feature requires. + +.. toctree:: + :maxdepth: 1 + + architecture + dataformat + identifiers + +Legal and Privacy Concerns +========================== + +Because Firefox Health Report collects and submits data to remote +servers and is an opt-out feature, there are legal and privacy +concerns over what data may be collected and submitted. **Additions or +changes to submitted data should be signed off by responsible +parties.** |