Add m-esr52 at 52.6.0

5 files changed, 511 insertions, 0 deletions

diff --git a/python/mach/docs/commands.rst b/python/mach/docs/commands.rst
new file mode 100644
index 000000000..af2973dd7
--- /dev/null
+++ b/python/mach/docs/commands.rst
@@ -0,0 +1,145 @@
+.. _mach_commands:
+
+=====================
+Implementing Commands
+=====================
+
+Mach commands are defined via Python decorators.
+
+All the relevant decorators are defined in the *mach.decorators* module.
+The important decorators are as follows:
+
+:py:func:`CommandProvider <mach.decorators.CommandProvider>`
+  A class decorator that denotes that a class contains mach
+  commands. The decorator takes no arguments.
+
+:py:func:`Command <mach.decorators.Command>`
+  A method decorator that denotes that the method should be called when
+  the specified command is requested. The decorator takes a command name
+  as its first argument and a number of additional arguments to
+  configure the behavior of the command.
+
+:py:func:`CommandArgument <mach.decorators.CommandArgument>`
+  A method decorator that defines an argument to the command. Its
+  arguments are essentially proxied to ArgumentParser.add_argument()
+
+:py:func:`SubCommand <mach.decorators.SubCommand>`
+  A method decorator that denotes that the method should be a
+  sub-command to an existing ``@Command``. The decorator takes the
+  parent command name as its first argument and the sub-command name
+  as its second argument.
+
+  ``@CommandArgument`` can be used on ``@SubCommand`` instances just
+  like they can on ``@Command`` instances.
+
+Classes with the ``@CommandProvider`` decorator **must** have an
+``__init__`` method that accepts 1 or 2 arguments. If it accepts 2
+arguments, the 2nd argument will be a
+:py:class:`mach.base.CommandContext` instance.
+
+Here is a complete example:
+
+.. code-block:: python
+
+   from mach.decorators import (
+       CommandArgument,
+       CommandProvider,
+       Command,
+   )
+
+   @CommandProvider
+   class MyClass(object):
+       @Command('doit', help='Do ALL OF THE THINGS.')
+       @CommandArgument('--force', '-f', action='store_true',
+           help='Force doing it.')
+       def doit(self, force=False):
+           # Do stuff here.
+
+When the module is loaded, the decorators tell mach about all handlers.
+When mach runs, it takes the assembled metadata from these handlers and
+hooks it up to the command line driver. Under the hood, arguments passed
+to the decorators are being used to help mach parse command arguments,
+formulate arguments to the methods, etc. See the documentation in the
+:py:mod:`mach.base` module for more.
+
+The Python modules defining mach commands do not need to live inside the
+main mach source tree.
+
+Conditionally Filtering Commands
+================================
+
+Sometimes it might only make sense to run a command given a certain
+context. For example, running tests only makes sense if the product
+they are testing has been built, and said build is available. To make
+sure a command is only runnable from within a correct context, you can
+define a series of conditions on the
+:py:func:`Command <mach.decorators.Command>` decorator.
+
+A condition is simply a function that takes an instance of the
+:py:func:`mach.decorators.CommandProvider` class as an argument, and
+returns ``True`` or ``False``. If any of the conditions defined on a
+command return ``False``, the command will not be runnable. The
+docstring of a condition function is used in error messages, to explain
+why the command cannot currently be run.
+
+Here is an example:
+
+.. code-block:: python
+
+   from mach.decorators import (
+       CommandProvider,
+       Command,
+   )
+
+   def build_available(cls):
+       """The build needs to be available."""
+       return cls.build_path is not None
+
+    @CommandProvider
+   class MyClass(MachCommandBase):
+       def __init__(self, build_path=None):
+           self.build_path = build_path
+
+       @Command('run_tests', conditions=[build_available])
+       def run_tests(self):
+           # Do stuff here.
+
+It is important to make sure that any state needed by the condition is
+available to instances of the command provider.
+
+By default all commands without any conditions applied will be runnable,
+but it is possible to change this behaviour by setting
+``require_conditions`` to ``True``:
+
+.. code-block:: python
+
+   m = mach.main.Mach()
+   m.require_conditions = True
+
+Minimizing Code in Commands
+===========================
+
+Mach command modules, classes, and methods work best when they are
+minimal dispatchers. The reason is import bloat. Currently, the mach
+core needs to import every Python file potentially containing mach
+commands for every command invocation. If you have dozens of commands or
+commands in modules that import a lot of Python code, these imports
+could slow mach down and waste memory.
+
+It is thus recommended that mach modules, classes, and methods do as
+little work as possible. Ideally the module should only import from
+the :py:mod:`mach` package. If you need external modules, you should
+import them from within the command method.
+
+To keep code size small, the body of a command method should be limited
+to:
+
+1. Obtaining user input (parsing arguments, prompting, etc)
+2. Calling into some other Python package
+3. Formatting output
+
+Of course, these recommendations can be ignored if you want to risk
+slower performance.
+
+In the future, the mach driver may cache the dispatching information or
+have it intelligently loaded to facilitate lazy loading.
diff --git a/python/mach/docs/driver.rst b/python/mach/docs/driver.rst
new file mode 100644
index 000000000..022ebe657
--- /dev/null
+++ b/python/mach/docs/driver.rst
@@ -0,0 +1,51 @@
+.. _mach_driver:
+
+=======
+Drivers
+=======
+
+Entry Points
+============
+
+It is possible to use setuptools' entry points to load commands
+directly from python packages. A mach entry point is a function which
+returns a list of files or directories containing mach command
+providers. e.g.:
+
+.. code-block:: python
+
+   def list_providers():
+       providers = []
+       here = os.path.abspath(os.path.dirname(__file__))
+       for p in os.listdir(here):
+           if p.endswith('.py'):
+               providers.append(os.path.join(here, p))
+       return providers
+
+See http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins
+for more information on creating an entry point. To search for entry
+point plugins, you can call
+:py:meth:`mach.main.Mach.load_commands_from_entry_point`. e.g.:
+
+.. code-block:: python
+
+   mach.load_commands_from_entry_point("mach.external.providers")
+
+Adding Global Arguments
+=======================
+
+Arguments to mach commands are usually command-specific. However,
+mach ships with a handful of global arguments that apply to all
+commands.
+
+It is possible to extend the list of global arguments. In your
+*mach driver*, simply call
+:py:meth:`mach.main.Mach.add_global_argument`. e.g.:
+
+.. code-block:: python
+
+   mach = mach.main.Mach(os.getcwd())
+
+   # Will allow --example to be specified on every mach command.
+   mach.add_global_argument('--example', action='store_true',
+       help='Demonstrate an example global argument.')
diff --git a/python/mach/docs/index.rst b/python/mach/docs/index.rst
new file mode 100644
index 000000000..cd2056333
--- /dev/null
+++ b/python/mach/docs/index.rst
@@ -0,0 +1,75 @@
+====
+mach
+====
+
+Mach (German for *do*) is a generic command dispatcher for the command
+line.
+
+To use mach, you install the mach core (a Python package), create an
+executable *driver* script (named whatever you want), and write mach
+commands. When the *driver* is executed, mach dispatches to the
+requested command handler automatically.
+
+Features
+========
+
+On a high level, mach is similar to using argparse with subparsers (for
+command handling). When you dig deeper, mach offers a number of
+additional features:
+
+Distributed command definitions
+  With optparse/argparse, you have to define your commands on a central
+  parser instance. With mach, you annotate your command methods with
+  decorators and mach finds and dispatches to them automatically.
+
+Command categories
+  Mach commands can be grouped into categories when displayed in help.
+  This is currently not possible with argparse.
+
+Logging management
+  Mach provides a facility for logging (both classical text and
+  structured) that is available to any command handler.
+
+Settings files
+  Mach provides a facility for reading settings from an ini-like file
+  format.
+
+Components
+==========
+
+Mach is conceptually composed of the following components:
+
+core
+  The mach core is the core code powering mach. This is a Python package
+  that contains all the business logic that makes mach work. The mach
+  core is common to all mach deployments.
+
+commands
+  These are what mach dispatches to. Commands are simply Python methods
+  registered as command names. The set of commands is unique to the
+  environment mach is deployed in.
+
+driver
+  The *driver* is the entry-point to mach. It is simply an executable
+  script that loads the mach core, tells it where commands can be found,
+  then asks the mach core to handle the current request. The driver is
+  unique to the deployed environment. But, it's usually based on an
+  example from this source tree.
+
+Project State
+=============
+
+mach was originally written as a command dispatching framework to aid
+Firefox development. While the code is mostly generic, there are still
+some pieces that closely tie it to Mozilla/Firefox. The goal is for
+these to eventually be removed and replaced with generic features so
+mach is suitable for anybody to use. Until then, mach may not be the
+best fit for you.
+
+.. toctree::
+   :maxdepth: 1
+
+   commands
+   driver
+   logging
+   settings
diff --git a/python/mach/docs/logging.rst b/python/mach/docs/logging.rst
new file mode 100644
index 000000000..ff245cf03
--- /dev/null
+++ b/python/mach/docs/logging.rst
@@ -0,0 +1,100 @@
+.. _mach_logging:
+
+=======
+Logging
+=======
+
+Mach configures a built-in logging facility so commands can easily log
+data.
+
+What sets the logging facility apart from most loggers you've seen is
+that it encourages structured logging. Instead of conventional logging
+where simple strings are logged, the internal logging mechanism logs all
+events with the following pieces of information:
+
+* A string *action*
+* A dict of log message fields
+* A formatting string
+
+Essentially, instead of assembling a human-readable string at
+logging-time, you create an object holding all the pieces of data that
+will constitute your logged event. For each unique type of logged event,
+you assign an *action* name.
+
+Depending on how logging is configured, your logged event could get
+written a couple of different ways.
+
+JSON Logging
+============
+
+Where machines are the intended target of the logging data, a JSON
+logger is configured. The JSON logger assembles an array consisting of
+the following elements:
+
+* Decimal wall clock time in seconds since UNIX epoch
+* String *action* of message
+* Object with structured message data
+
+The JSON-serialized array is written to a configured file handle.
+Consumers of this logging stream can just perform a readline() then feed
+that into a JSON deserializer to reconstruct the original logged
+message. They can key off the *action* element to determine how to
+process individual events. There is no need to invent a parser.
+Convenient, isn't it?
+
+Logging for Humans
+==================
+
+Where humans are the intended consumer of a log message, the structured
+log message are converted to more human-friendly form. This is done by
+utilizing the *formatting* string provided at log time. The logger
+simply calls the *format* method of the formatting string, passing the
+dict containing the message's fields.
+
+When *mach* is used in a terminal that supports it, the logging facility
+also supports terminal features such as colorization. This is done
+automatically in the logging layer - there is no need to control this at
+logging time.
+
+In addition, messages intended for humans typically prepends every line
+with the time passed since the application started.
+
+Logging HOWTO
+=============
+
+Structured logging piggybacks on top of Python's built-in logging
+infrastructure provided by the *logging* package. We accomplish this by
+taking advantage of *logging.Logger.log()*'s *extra* argument. To this
+argument, we pass a dict with the fields *action* and *params*. These
+are the string *action* and dict of message fields, respectively. The
+formatting string is passed as the *msg* argument, like normal.
+
+If you were logging to a logger directly, you would do something like:
+
+.. code-block:: python
+
+   logger.log(logging.INFO, 'My name is {name}',
+       extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
+
+The JSON logging would produce something like::
+
+   [1339985554.306338, "my_name", {"name": "Gregory"}]
+
+Human logging would produce something like::
+
+   0.52 My name is Gregory
+
+Since there is a lot of complexity using logger.log directly, it is
+recommended to go through a wrapping layer that hides part of the
+complexity for you. The easiest way to do this is by utilizing the
+LoggingMixin:
+
+.. code-block:: python
+
+   import logging
+   from mach.mixin.logging import LoggingMixin
+
+   class MyClass(LoggingMixin):
+       def foo(self):
+            self.log(logging.INFO, 'foo_start', {'bar': True},
+                'Foo performed. Bar: {bar}')
diff --git a/python/mach/docs/settings.rst b/python/mach/docs/settings.rst
new file mode 100644
index 000000000..b51dc54a2
--- /dev/null
+++ b/python/mach/docs/settings.rst
@@ -0,0 +1,140 @@
+.. _mach_settings:
+
+========
+Settings
+========
+
+Mach can read settings in from a set of configuration files. These
+configuration files are either named ``machrc`` or ``.machrc`` and
+are specified by the bootstrap script. In mozilla-central, these files
+can live in ``~/.mozbuild`` and/or ``topsrcdir``.
+
+Settings can be specified anywhere, and used both by mach core or
+individual commands.
+
+
+Core Settings
+=============
+
+These settings are implemented by mach core.
+
+* alias - Create a command alias. This is useful if you want to alias a command to something else, optionally including some defaults. It can either be used to create an entire new command, or provide defaults for an existing one. For example:
+
+.. parsed-literal::
+
+    [alias]
+    mochitest = mochitest -f browser
+    browser-test = mochitest -f browser
+
+
+Defining Settings
+=================
+
+Settings need to be explicitly defined, along with their type,
+otherwise mach will throw when trying to access them.
+
+To define settings, use the :func:`~decorators.SettingsProvider`
+decorator in an existing mach command module. E.g:
+
+.. code-block:: python
+
+    from mach.decorators import SettingsProvider
+
+    @SettingsProvider
+    class ArbitraryClassName(object):
+        config_settings = [
+            ('foo.bar', 'string'),
+            ('foo.baz', 'int', 0, set([0,1,2])),
+        ]
+
+``@SettingsProvider``'s must specify a variable called ``config_settings``
+that returns a list of tuples. Alternatively, it can specify a function
+called ``config_settings`` that returns a list of tuples.
+
+Each tuple is of the form:
+
+.. code-block:: python
+
+    ('<section>.<option>', '<type>', default, extra)
+
+``type`` is a string and can be one of:
+string, boolean, int, pos_int, path
+
+``default`` is optional, and provides a default value in case none was
+specified by any of the configuration files.
+
+``extra`` is also optional and is a dict containing additional key/value
+pairs to add to the setting's metadata. The following keys may be specified
+in the ``extra`` dict:
+    * ``choices`` - A set of allowed values for the setting.
+
+Wildcards
+---------
+
+Sometimes a section should allow arbitrarily defined options from the user, such
+as the ``alias`` section mentioned above. To define a section like this, use ``*``
+as the option name. For example:
+
+.. parsed-literal::
+
+    ('foo.*', 'string')
+
+This allows configuration files like this:
+
+.. parsed-literal::
+
+    [foo]
+    arbitrary1 = some string
+    arbitrary2 = some other string
+
+
+Documenting Settings
+====================
+
+All settings must at least be documented in the en_US locale. Otherwise,
+running ``mach settings`` will raise. Mach uses gettext to perform localization.
+
+A handy command exists to generate the localization files:
+
+.. parsed-literal::
+
+    mach settings locale-gen <section>
+
+You'll be prompted to add documentation for all options in section with the
+en_US locale. To add documentation in another locale, pass in ``--locale``.
+
+
+Accessing Settings
+==================
+
+Now that the settings are defined and documented, they're accessible from
+individual mach commands if the command receives a context in its constructor.
+For example:
+
+.. code-block:: python
+
+    from mach.decorators import (
+        Command,
+        CommandProvider,
+        SettingsProvider,
+    )
+
+    @SettingsProvider
+    class ExampleSettings(object):
+        config_settings = [
+            ('a.b', 'string', 'default'),
+            ('foo.bar', 'string'),
+            ('foo.baz', 'int', 0, {'choices': set([0,1,2])}),
+        ]
+
+    @CommandProvider
+    class Commands(object):
+        def __init__(self, context):
+            self.settings = context.settings
+
+        @Command('command', category='misc',
+                 description='Prints a setting')
+        def command(self):
+            print(self.settings.a.b)
+            for option in self.settings.foo:
+                print(self.settings.foo[option])