.. workflow:: package_publish

Workflow ``package_publish``
============================

This workflow publishes source and/or binary packages to a given target
suite.  It is normally expected to be used as a sub-workflow.

Permission considerations
~~~~~~~~~~~~~~~~~~~~~~~~~

Copying artifacts requires both the ability to read from the source and the
ability to write to the destination (either directly or via a workflow).

After artifacts have been made public, it's helpful to be able to see the
work request that created them, without having to somehow also copy the work
request around.  To achieve this, the permission predicate that checks
whether a user can see a work request may check whether any of the artifacts
produced by the work request are visible to that user, and return True in
that case even if the work request itself would not ordinarily be visible.

.. note::

   It may be surprising that this rule is "any of the artifacts produced by
   the work request" rather than "all of the artifacts produced by the work
   request"; but there isn't usually anywhere useful to copy
   :artifact:`debusine:work-request-debug-logs` artifacts to, and making
   only some of the artifacts produced by a work request public seems
   unlikely to be a realistic unembargoing use case.

While build logs may expose additional information not in the output
artifacts (such as build-dependencies where security updates are also being
prepared), similar information might easily be exposed by the output
artifacts themselves anyway, so the onus is on people who make artifacts
public to check that it is safe to do so.

.. todo::

   Permission checks for this workflow are not yet implemented.

Resource accounting considerations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We want to be able to track the resource usage of workspaces and scopes.  If
artifacts are copied between workspaces (and hence perhaps between scopes),
then the same files may exist in multiple workspaces, complicating this kind
of analysis.  The question is likely to be something along the lines of "how
much data does Debusine need to store on behalf of this workspace or scope
that it would not otherwise need to store?".

A reasonable first cut would be to track the origin of copies, and to
account an artifact's files to a workspace (and its containing scope) if the
artifact is in that workspace and is no longer in its origin workspace.  We
therefore add a nullable ``Artifact.original_artifact`` foreign key, with
``on_delete=SET_NULL``.

Some other variations are possible, and are not made more difficult by this
design.  For example, we may wish to account for each workspace's usage
without considering whether files have been copied from or to other
workspaces (in which case the total file store size may be less than the sum
of the sizes of all workspaces); or to calculate the "unique" size of a
workspace as the total size of all files that appear only in that workspace.

Workflow definition
~~~~~~~~~~~~~~~~~~~

* ``task_data``:

  * ``source_artifact`` (:ref:`lookup-single`, optional): a
    :artifact:`debian:source-package` or :artifact:`debian:upload` artifact
    representing the source package (the former is used when the workflow is
    started based on a ``.dsc`` rather than a ``.changes``)
  * ``binary_artifacts`` (:ref:`lookup-multiple`, optional): a list of
    :artifact:`debian:upload` artifacts representing the binary packages
  * ``target_suite`` (:ref:`lookup-single`, required): the
    :collection:`debian:suite` collection to publish packages to
  * ``unembargo`` (boolean, defaults to False): if True, allow publishing
    artifacts from private workspaces to public suites
  * ``replace`` (boolean, defaults to False): if True, replace existing
    similar items
  * ``suite_variables`` (dictionary, optional): pass these variables when
    adding items to the target suite collection; see
    :collection:`debian:suite` for the available variable names
  * ``update_indexes`` (boolean, defaults to True): if True, update indexes
    in the target suite after copying items

At least one of ``source_artifact`` and ``binary_artifacts`` must be set.

The workflow computes dynamic metadata as:

.. dynamic_data::
  :method: debusine.server.workflows.package_publish::PackagePublishWorkflow.build_dynamic_data

``target_suite`` is looked up relative to this workflow's workspace.  As a
result, it must either be part of this workspace's inheritance chain, or
else be identified by ID (``NNN`` or ``NNN@collections``).

The ``component``, ``section``, and (for binary packages) ``priority``
variables need to be set when adding packages to the target suite.  They are
determined as follows:

* default to the values from the artifact:

  * for a :artifact:`debian:upload` source artifact, use the component and
    section listed for the ``.dsc`` file in the ``.changes`` file
  * for a :artifact:`debian:source-package` source artifact, then its
    component and section defaults are unavailable without unpacking the
    source package, so set the ``component`` to ``main`` and the ``section``
    to ``misc``
  * for a binary artifact, use the `Section
    <https://www.debian.org/doc/debian-policy/ch-controlfields.html#section>`__
    field in the ``.deb`` to set the component and section and the `Priority
    <https://www.debian.org/doc/debian-policy/ch-controlfields.html#priority>`__
    field to set the priority, falling back to ``main``, ``misc``, and
    ``optional`` respectively

* merge in the values from ``suite_variables``, which take priority over the
  defaults above

The workflow creates a :task:`CopyCollectionItems` task.  The ``copies``
field in its task data is as follows:

* ``source_items``: the union of whichever of ``{source_artifact}`` and
  ``{binary_artifacts}`` are set
* ``target_collection``: ``{target_suite}``
* ``unembargo``: ``{unembargo}``
* ``replace``: ``{replace}``
* ``variables``: ``{suite_variables}``

Any of the lookups in ``source_items`` may result in :bare-data:`promises
<debusine:promise>`, and in that case the workflow adds corresponding
dependencies.

If ``binary_artifacts`` is set and the source and target workspaces have
different instances of the :collection:`debian:package-build-logs`
collection, then the workflow also adds an entry to ``copies`` as follows:

* ``source_items``:

  .. code-block:: yaml

      collection: {source build logs collection}
      lookup__same_work_request: {binary_artifacts}

* ``target_collection``: target build logs collection
* ``unembargo``: ``{unembargo}``
* ``replace``: ``{replace}``

If ``binary_artifacts`` is set and the source and target workspaces have
different instances of the :collection:`debusine:task-history` collection,
then the workflow also adds an entry to ``copies`` as follows:

* ``source_items``:

  .. code-block:: yaml

      collection: {source task history collection}
      lookup__same_workflow: {binary_artifacts}

* ``target_collection``: target task history collection
* ``unembargo``: ``{unembargo}``
* ``replace``: ``{replace}``

If ``update_indexes`` is True, then once the :task:`CopyCollectionItems`
task has completed, the workflow triggers an :workflow:`update_suites`
workflow in the target workspace, with ``only_suites`` set to a list
containing the name of the target suite.
