1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
|
Task Kinds
==========
This section lists and documents the available task kinds.
build
------
Builds are tasks that produce an installer or other output that can be run by
users or automated tests. This is more restrictive than most definitions of
"build" in a Mozilla context: it does not include tasks that run build-like
actions for static analysis or to produce instrumented artifacts.
artifact-build
--------------
This kind performs an artifact build: one based on precompiled binaries
discovered via the TaskCluster index. This task verifies that such builds
continue to work correctly.
hazard
------
Hazard builds are similar to "regular' builds, but use a compiler extension to
extract a bunch of data from the build and then analyze that data looking for
hazardous behaviors.
l10n
----
TBD (Callek)
source-check
------------
Source-checks are tasks that look at the Gecko source directly to check
correctness. This can include linting, Python unit tests, source-code
analysis, or measurement work -- basically anything that does not require a
build.
upload-symbols
--------------
Upload-symbols tasks run after builds and upload the symbols files generated by
build tasks to Socorro for later use in crash analysis.
valgrind
--------
Valgrind tasks produce builds instrumented by valgrind.
static-analysis
---------------
Static analysis builds use the compiler to perform some detailed analysis of
the source code while building. The useful output from these tasks are their
build logs, and while they produce a binary, they do not upload it as an
artifact.
toolchain
---------
Toolchain builds create the compiler toolchains used to build Firefox. These
will eventually be dependencies of the builds themselves, but for the moment
are run manually via try pushes and the results uploaded to tooltool.
spidermonkey
------------
Spidermonkey tasks check out the full gecko source tree, then compile only the
spidermonkey portion. Each task runs specific tests after the build.
marionette-harness
------------------
TBD (Maja)
Tests
-----
Test tasks for Gecko products are divided into several kinds, but share a
common implementation. The process goes like this, based on a set of YAML
files named in ``kind.yml``:
* For each build task, determine the related test platforms based on the build
platform. For example, a Windows 2010 build might be tested on Windows 7
and Windows 10. Each test platform specifies a "test set" indicating which
tests to run. This is configured in the file named
``test-platforms.yml``.
* Each test set is expanded to a list of tests to run. This is configured in
the file named by ``test-sets.yml``.
* Each named test is looked up in the file named by ``tests.yml`` to find a
test description. This test description indicates what the test does, how
it is reported to treeherder, and how to perform the test, all in a
platform-independent fashion.
* Each test description is converted into one or more tasks. This is
performed by a sequence of transforms defined in the ``transforms`` key in
``kind.yml``. See :doc:`transforms`: for more information on these
transforms.
* The resulting tasks become a part of the task graph.
.. important::
This process generates *all* test jobs, regardless of tree or try syntax.
It is up to a later stage of the task-graph generation (the target set) to
select the tests that will actually be performed.
desktop-test
............
The ``desktop-test`` kind defines tests for Desktop builds. Its ``tests.yml``
defines the full suite of desktop tests and their particulars, leaving it to
the transforms to determine how those particulars apply to Linux, OS X, and
Windows.
android-test
............
The ``android-test`` kind defines tests for Android builds.
It is very similar to ``desktop-test``, but the details of running the tests
differ substantially, so they are defined separately.
docker-image
------------
Tasks of the ``docker-image`` kind build the Docker images in which other
Docker tasks run.
The tasks to generate each docker image have predictable labels:
``build-docker-image-<name>``.
Docker images are built from subdirectories of ``testing/docker``, using
``docker build``. There is currently no capability for one Docker image to
depend on another in-tree docker image, without uploading the latter to a
Docker repository
The task definition used to create the image-building tasks is given in
``image.yml`` in the kind directory, and is interpreted as a :doc:`YAML
Template <yaml-templates>`.
|
