<ddclass="field-odd"><p>Brian C. Lane <<aclass="reference external"href="mailto:bcl%40redhat.com">bcl<span>@</span>redhat<span>.</span>com</a>></p>
<h2>Installation<aclass="headerlink"href="#installation"title="Permalink to this headline">¶</a></h2>
<p>The best way to install <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> is to use <codeclass="docutils literal notranslate"><spanclass="pre">sudo</span><spanclass="pre">dnf</span><spanclass="pre">install</span>
<li><p>Create a <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code> user and group by running <codeclass="docutils literal notranslate"><spanclass="pre">useradd</span><spanclass="pre">weldr</span></code></p></li>
<li><p>Remove any pre-existing socket directory with <codeclass="docutils literal notranslate"><spanclass="pre">rm</span><spanclass="pre">-rf</span><spanclass="pre">/run/weldr/</span></code>
A new directory with correct permissions will be created the first time the server runs.</p></li>
<li><p>Enable the socket activation with <codeclass="docutils literal notranslate"><spanclass="pre">systemctl</span><spanclass="pre">enable</span><spanclass="pre">lorax-composer.socket</span>
<p>NOTE: You can also run it directly with <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span><spanclass="pre">/path/to/blueprints</span></code>. However,
<codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> does not react well to being started both on the command line and via
socket activation at the same time. It is therefore recommended that you run it directly
on the command line only for testing or development purposes. For real use or development
of other projects that simply use the API, you should stick to socket activation only.</p>
<h2>Logs<aclass="headerlink"href="#logs"title="Permalink to this headline">¶</a></h2>
<p>Logs are stored under <codeclass="docutils literal notranslate"><spanclass="pre">/var/log/lorax-composer/</span></code> and include all console
messages as well as extra debugging info and API requests.</p>
</div>
<divclass="section"id="security">
<h2>Security<aclass="headerlink"href="#security"title="Permalink to this headline">¶</a></h2>
<p>Some security related issues that you should be aware of before running <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>:</p>
<li><p>One of the API server threads needs to retain root privileges in order to run Anaconda.</p></li>
<li><p>Only allow authorized users access to the <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code> group and socket.</p></li>
<h2>How it Works<aclass="headerlink"href="#how-it-works"title="Permalink to this headline">¶</a></h2>
<p>The server runs as root, and as <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code>. Communication with it is via a unix
domain socket (<codeclass="docutils literal notranslate"><spanclass="pre">/run/weldr/api.socket</span></code> by default). The directory and socket
are owned by <codeclass="docutils literal notranslate"><spanclass="pre">root:weldr</span></code> so that any user in the <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code> group can use the API
to control <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>.</p>
<p>At startup the server will check for the correct permissions and
ownership of a pre-existing directory, or it will create a new one if it
cmdline by passing it the <codeclass="docutils literal notranslate"><spanclass="pre">--socket</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">--group</span></code> arguments.</p>
<p>It will then drop root privileges for the API thread and run as the <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code>
user. The queue and compose thread still runs as root because it needs to be
able to mount/umount files and run Anaconda.</p>
</div>
<divclass="section"id="composing-images">
<h2>Composing Images<aclass="headerlink"href="#composing-images"title="Permalink to this headline">¶</a></h2>
<p>The <aclass="reference external"href="https://github.com/weldr/welder-web/">welder-web</a> GUI project can be used to construct
blueprints and create composes using a web browser.</p>
<h2>Blueprints<aclass="headerlink"href="#blueprints"title="Permalink to this headline">¶</a></h2>
<p>Blueprints are simple text files in <aclass="reference external"href="https://github.com/toml-lang/toml">TOML</a> format that describe
which packages, and what versions, to install into the image. They can also define a limited set
of customizations to make to the final image.</p>
<p>Example blueprints can be found in the <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code><aclass="reference external"href="https://github.com/weldr/lorax/tree/master/tests/pylorax/blueprints/">test suite</a>, with a simple one
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">name</span></code> field is the name of the blueprint. It can contain spaces, but they will be converted to <codeclass="docutils literal notranslate"><spanclass="pre">-</span></code>
when it is written to disk. It should be short and descriptive.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">description</span></code> can be a longer description of the blueprint, it is only used for display purposes.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">version</span></code> is a <aclass="reference external"href="https://semver.org/">semver compatible</a> version number. If
a new blueprint is uploaded with the same <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code> the server will
automatically bump the PATCH level of the <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code>. If the <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code>
set to <codeclass="docutils literal notranslate"><spanclass="pre">0.1.0</span></code> when the existing blueprint <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code> is <codeclass="docutils literal notranslate"><spanclass="pre">0.0.1</span></code> will
result in the new blueprint being stored as <codeclass="docutils literal notranslate"><spanclass="pre">version</span><spanclass="pre">0.1.0</span></code>.</p>
<h3>[[packages]] and [[modules]]<aclass="headerlink"href="#packages-and-modules"title="Permalink to this headline">¶</a></h3>
<p>These entries describe the package names and matching version glob to be installed into the image.</p>
<p>The names must match the names exactly, and the versions can be an exact match
or a filesystem-like glob of the version using <codeclass="docutils literal notranslate"><spanclass="pre">*</span></code> wildcards and <codeclass="docutils literal notranslate"><spanclass="pre">?</span></code>
<p>NOTE: Currently there are no differences between <codeclass="docutils literal notranslate"><spanclass="pre">packages</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">modules</span></code>
in <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>. Both are treated like an rpm package dependency.</p>
<p>For example, to install <codeclass="docutils literal notranslate"><spanclass="pre">tmux-2.9a</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">openssh-server-8.*</span></code>, you would add
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">groups</span></code> entries describe a group of packages to be installed into the image. Package groups are
<p>For example, if you want to install the <codeclass="docutils literal notranslate"><spanclass="pre">anaconda-tools</span></code> group you would add this to your
<p><codeclass="docutils literal notranslate"><spanclass="pre">groups</span></code> is a TOML list, so each group needs to be listed separately, like <codeclass="docutils literal notranslate"><spanclass="pre">packages</span></code> but with
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">[customizations]</span></code> section can be used to configure the hostname of the final image. eg.:</p>
<h4>[customizations.kernel]<aclass="headerlink"href="#customizations-kernel"title="Permalink to this headline">¶</a></h4>
<p>This allows you to append arguments to the bootloader's kernel commandline. This will not have any
effect on <codeclass="docutils literal notranslate"><spanclass="pre">tar</span></code> or <codeclass="docutils literal notranslate"><spanclass="pre">ext4-filesystem</span></code> images since they do not include a bootloader.</p>
<h4>[[customizations.user]]<aclass="headerlink"href="#customizations-user"title="Permalink to this headline">¶</a></h4>
<p>Add a user to the image, and/or set their ssh key.
All fields for this section are optional except for the <codeclass="docutils literal notranslate"><spanclass="pre">name</span></code>, here is a complete example:</p>
<p>If the password starts with <codeclass="docutils literal notranslate"><spanclass="pre">$6$</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">$5$</span></code>, or <codeclass="docutils literal notranslate"><spanclass="pre">$2b$</span></code> it will be stored as
an encrypted password. Otherwise it will be treated as a plain text password.</p>
<h4>[[customizations.group]]<aclass="headerlink"href="#customizations-group"title="Permalink to this headline">¶</a></h4>
<p>Add a group to the image. <codeclass="docutils literal notranslate"><spanclass="pre">name</span></code> is required and <codeclass="docutils literal notranslate"><spanclass="pre">gid</span></code> is optional:</p>
<p>The values supported by <codeclass="docutils literal notranslate"><spanclass="pre">timezone</span></code> can be listed by running <codeclass="docutils literal notranslate"><spanclass="pre">timedatectl</span><spanclass="pre">list-timezones</span></code>.</p>
<p>If no timezone is setup the system will default to using <cite>UTC</cite>. The ntp servers are also
optional and will default to using the distribution defaults which are fine for most uses.</p>
<p>In some image types there are already NTP servers setup, eg. Google cloud image, and they
cannot be overridden because they are required to boot in the selected environment. But the
timezone will be updated to the one selected in the blueprint.</p>
</div>
<divclass="section"id="customizations-locale">
<h4>[customizations.locale]<aclass="headerlink"href="#customizations-locale"title="Permalink to this headline">¶</a></h4>
<p>Customize the locale settings for the system:</p>
<p>The values supported by <codeclass="docutils literal notranslate"><spanclass="pre">languages</span></code> can be listed by running <codeclass="docutils literal notranslate"><spanclass="pre">localectl</span><spanclass="pre">list-locales</span></code> from
the command line.</p>
<p>The values supported by <codeclass="docutils literal notranslate"><spanclass="pre">keyboard</span></code> can be listed by running <codeclass="docutils literal notranslate"><spanclass="pre">localectl</span><spanclass="pre">list-keymaps</span></code> from
the command line.</p>
<p>Multiple languages can be added. The first one becomes the
primary, and the others are added as secondary. One or the other of <codeclass="docutils literal notranslate"><spanclass="pre">languages</span></code>
or <codeclass="docutils literal notranslate"><spanclass="pre">keyboard</span></code> must be included (or both) in the section.</p>
</div>
<divclass="section"id="customizations-firewall">
<h4>[customizations.firewall]<aclass="headerlink"href="#customizations-firewall"title="Permalink to this headline">¶</a></h4>
<p>By default the firewall blocks all access except for services that enable their ports explicitly,
like <codeclass="docutils literal notranslate"><spanclass="pre">sshd</span></code>. This command can be used to open other ports or services. Ports are configured using
<p>Numeric ports, or their names from <codeclass="docutils literal notranslate"><spanclass="pre">/etc/services</span></code> can be used in the <codeclass="docutils literal notranslate"><spanclass="pre">ports</span></code> enabled/disabled lists.</p>
<p>The blueprint settings extend any existing settings in the image templates, so if <codeclass="docutils literal notranslate"><spanclass="pre">sshd</span></code> is
already enabled it will extend the list of ports with the ones listed by the blueprint.</p>
<p>If the distribution uses <codeclass="docutils literal notranslate"><spanclass="pre">firewalld</span></code> you can specify services listed by <codeclass="docutils literal notranslate"><spanclass="pre">firewall-cmd</span><spanclass="pre">--get-services</span></code>
in a <codeclass="docutils literal notranslate"><spanclass="pre">customizations.firewall.services</span></code> section:</p>
<p>Remember that the <codeclass="docutils literal notranslate"><spanclass="pre">firewall.services</span></code> are different from the names in <codeclass="docutils literal notranslate"><spanclass="pre">/etc/services</span></code>.</p>
<p>Both are optional, if they are not used leave them out or set them to an empty list <codeclass="docutils literal notranslate"><spanclass="pre">[]</span></code>. If you
only want the default firewall setup this section can be omitted from the blueprint.</p>
<p>NOTE: The <codeclass="docutils literal notranslate"><spanclass="pre">Google</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">OpenStack</span></code> templates explicitly disable the firewall for their environment.
This cannot be overridden by the blueprint.</p>
</div>
<divclass="section"id="customizations-services">
<h4>[customizations.services]<aclass="headerlink"href="#customizations-services"title="Permalink to this headline">¶</a></h4>
<p>This section can be used to control which services are enabled at boot time.
Some image types already have services enabled or disabled in order for the
image to work correctly, and cannot be overridden. eg. <codeclass="docutils literal notranslate"><spanclass="pre">ami</span></code> requires
<codeclass="docutils literal notranslate"><spanclass="pre">sshd</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">chronyd</span></code>, and <codeclass="docutils literal notranslate"><spanclass="pre">cloud-init</span></code>. Without them the image will not
boot. Blueprint services are added to, not replacing, the list already in the
templates, if any.</p>
<p>The service names are systemd service units. You may specify any systemd unit
file accepted by <codeclass="docutils literal notranslate"><spanclass="pre">systemctl</span><spanclass="pre">enable</span></code> eg. <codeclass="docutils literal notranslate"><spanclass="pre">cockpit.socket</span></code>:</p>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">[[repos.git]]</span></code> entries are used to add files from a <aclass="reference external"href="https://git-scm.com/">git repository</a>
repository to the created image. The repository is cloned, the specified <codeclass="docutils literal notranslate"><spanclass="pre">ref</span></code> is checked out
and an rpm is created to install the files to a <codeclass="docutils literal notranslate"><spanclass="pre">destination</span></code> path. The rpm includes a summary
with the details of the repository and reference used to create it. The rpm is also included in the
image build metadata.</p>
<p>To create an rpm named <codeclass="docutils literal notranslate"><spanclass="pre">server-config-1.0-1.noarch.rpm</span></code> you would add this to your blueprint:</p>
<li><p>rpmname: Name of the rpm to create, also used as the prefix name in the tar archive</p></li>
<li><p>rpmversion: Version of the rpm, eg. "1.0.0"</p></li>
<li><p>rpmrelease: Release of the rpm, eg. "1"</p></li>
<li><p>summary: Summary string for the rpm</p></li>
<li><p>repo: URL of the get repo to clone and create the archive from</p></li>
<li><p>ref: Git reference to check out. eg. origin/branch-name, git tag, or git commit hash</p></li>
<li><p>destination: Path to install the / of the git repo at when installing the rpm</p></li>
</ul>
<p>An rpm will be created with the contents of the git repository referenced, with the files
being installed under <codeclass="docutils literal notranslate"><spanclass="pre">/opt/server/</span></code> in this case.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">ref</span></code> can be any valid git reference for use with <codeclass="docutils literal notranslate"><spanclass="pre">git</span><spanclass="pre">archive</span></code>. eg. to use the head
of a branch set it to <codeclass="docutils literal notranslate"><spanclass="pre">origin/branch-name</span></code>, a tag name, or a commit hash.</p>
<p>Note that the repository is cloned in full each time a build is started, so pointing to a
repository with a large amount of history may take a while to clone and use a significant
amount of disk space. The clone is temporary and is removed once the rpm is created.</p>
<h2>Adding Output Types<aclass="headerlink"href="#adding-output-types"title="Permalink to this headline">¶</a></h2>
<p><codeclass="docutils literal notranslate"><spanclass="pre">livemedia-creator</span></code> supports a large number of output types, and only some of
these are currently available via <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>. To add a new output type to
lorax-composer a kickstart file needs to be added to <codeclass="docutils literal notranslate"><spanclass="pre">./share/composer/</span></code>. The
name of the kickstart is what will be used by the <codeclass="docutils literal notranslate"><spanclass="pre">/compose/types</span></code> route, and the
<codeclass="docutils literal notranslate"><spanclass="pre">compose_type</span></code> field of the POST to start a compose. It also needs to have
code added to the <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.compose_args"title="pylorax.api.compose.compose_args"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.compose_args()</span></code></a> function. The
<aclass="reference internal"href="pylorax.html#pylorax.installer.novirt_install"title="pylorax.installer.novirt_install"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.installer.novirt_install()</span></code></a> when it runs the compose. When the
<aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.move_compose_results"title="pylorax.api.compose.move_compose_results"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.move_compose_results()</span></code></a> handles this for each type.
<li><p>A <codeclass="docutils literal notranslate"><spanclass="pre">partitioned-disk.ks</span></code> file in <codeclass="docutils literal notranslate"><spanclass="pre">./share/composer/</span></code></p></li>
<li><p>A new entry in the _MAP in <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.compose_args"title="pylorax.api.compose.compose_args"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.compose_args()</span></code></a></p></li>
<li><p>Add a bit of code to <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.move_compose_results"title="pylorax.api.compose.move_compose_results"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.move_compose_results()</span></code></a> to move the disk image from
the compose directory to the results directory.</p></li>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">partitioned-disk.ks</span></code> is pretty similar to the example minimal kickstart
in <codeclass="docutils literal notranslate"><spanclass="pre">./docs/fedora-minimal.ks</span></code>. You should remove the <codeclass="docutils literal notranslate"><spanclass="pre">url</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">repo</span></code>
commands, they will be added by the compose process. Make sure the bootloader
packages are included in the <codeclass="docutils literal notranslate"><spanclass="pre">%packages</span></code> section at the end of the kickstart,
and you will want to leave off the <codeclass="docutils literal notranslate"><spanclass="pre">%end</span></code> so that the compose can append the
list of packages from the blueprint.</p>
<p>The new <codeclass="docutils literal notranslate"><spanclass="pre">_MAP</span></code> entry should be a copy of one of the existing entries, but with <codeclass="docutils literal notranslate"><spanclass="pre">make_disk</span></code> set
to <codeclass="docutils literal notranslate"><spanclass="pre">True</span></code>. Make sure that none of the other <codeclass="docutils literal notranslate"><spanclass="pre">make_*</span></code> options are <codeclass="docutils literal notranslate"><spanclass="pre">True</span></code>. The <codeclass="docutils literal notranslate"><spanclass="pre">image_name</span></code> is
what the name of the final image will be.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">move_compose_results()</span></code> can be as simple as moving the output file into
the results directory, or it could do some post-processing on it. The end of
the function should always clean up the <codeclass="docutils literal notranslate"><spanclass="pre">./compose/</span></code> directory, removing any
unneeded extra files. This is especially true for the <codeclass="docutils literal notranslate"><spanclass="pre">live-iso</span></code> since it produces
the contents of the iso as well as the boot.iso itself.</p>
the <codeclass="docutils literal notranslate"><spanclass="pre">*.repo</span></code> files from <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code> into
<codeclass="docutils literal notranslate"><spanclass="pre">/var/lib/lorax/composer/repos.d/</span></code> at startup, these are immutable system
repositories and cannot be deleted or changed. If you want to add additional
repos you can put them into <codeclass="docutils literal notranslate"><spanclass="pre">/var/lib/lorax/composer/repos.d/</span></code> or use the
<codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/*</span></code> API routes to create them.</p>
<p>The new source can be added by doing a POST to the <codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/new</span></code>
route using JSON (with <cite>Content-Type</cite> header set to <cite>application/json</cite>) or TOML
(with it set to <cite>text/x-toml</cite>). The format of the source looks like this (in
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">proxy</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">gpgkey_urls</span></code> entries are optional. All of the others are required. The supported
<p>If <codeclass="docutils literal notranslate"><spanclass="pre">check_ssl</span></code> is true the https certificates must be valid. If they are self-signed you can either set
this to false, or add your Certificate Authority to the host system.</p>
<p>If <codeclass="docutils literal notranslate"><spanclass="pre">check_gpg</span></code> is true the GPG key must either be installed on the host system, or <codeclass="docutils literal notranslate"><spanclass="pre">gpgkey_urls</span></code>
should point to it.</p>
<p>You can edit an existing source (other than system sources), by doing a POST to the <codeclass="docutils literal notranslate"><spanclass="pre">new</span></code> route
with the new version of the source. It will overwrite the previous one.</p>
<p>A list of existing sources is available from <codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/list</span></code>, and detailed info
on a source can be retrieved with the <codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/info/<source-name></span></code> route. By default
it returns JSON but it can also return TOML if <codeclass="docutils literal notranslate"><spanclass="pre">?format=toml</span></code> is added to the request.</p>
<p>Non-system sources can be deleted by doing a <codeclass="docutils literal notranslate"><spanclass="pre">DELETE</span></code> request to the
<p>The documentation for the source API routes can be <aclass="reference external"href="pylorax.api.html#api-v0-projects-source-list">found here</a></p>
<p>The configured sources are used for all blueprint depsolve operations, and for composing images.
When adding additional sources you must make sure that the packages in the source do not
conflict with any other package sources, otherwise depsolving will fail.</p>
<h3>DVD ISO Package Source<aclass="headerlink"href="#dvd-iso-package-source"title="Permalink to this headline">¶</a></h3>
<p>In some situations the system may want to <em>only</em> use a DVD iso as the package
source, not the repos from the network. <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">anaconda</span></code>
understand <codeclass="docutils literal notranslate"><spanclass="pre">file://</span></code> URLs so you can mount an iso on the host, and replace the
system repo files with a configuration file pointing to the DVD.</p>
<li><p>Stop the <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer.service</span></code> if it is running</p></li>
<li><p>Move the repo files in <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code> someplace safe</p></li>
<li><p>Create a new <codeclass="docutils literal notranslate"><spanclass="pre">iso.repo</span></code> file in <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code>:</p>
<li><p>Remove all the cached repo files from <codeclass="docutils literal notranslate"><spanclass="pre">/var/lib/lorax/composer/repos/</span></code></p></li>
<li><p>Restart the <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer.service</span></code></p></li>
<li><p>Check the output of <codeclass="docutils literal notranslate"><spanclass="pre">composer-cli</span><spanclass="pre">status</span><spanclass="pre">show</span></code> for any output specific depsolve errors.
For example, the DVD usually does not include <codeclass="docutils literal notranslate"><spanclass="pre">grub2-efi-*-cdboot-*</span></code> so the live-iso image
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
