<trclass="field-odd field"><thclass="field-name">Authors:</th><tdclass="field-body">Brian C. Lane <<aclass="reference external"href="mailto:bcl%40redhat.com">bcl<span>@</span>redhat<span>.</span>com</a>></td>
</tr>
</tbody>
</table>
<p><codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> is an API server that allows you to build disk images using
<aclass="reference internal"href="#blueprints">Blueprints</a> to describe the package versions to be installed into the image.
It is compatible with the Weldr project’s bdcs-api REST protocol. More
information on Weldr can be found <aclass="reference external"href="http://www.weldr.io">on the Weldr blog</a>.</p>
<p>Behind the scenes it uses <aclass="reference external"href="livemedia-creator.html">livemedia-creator</a> and
<aclass="reference external"href="https://anaconda-installer.readthedocs.io/en/latest/">Anaconda</a> to handle the
<h2>Installation<aclass="headerlink"href="#installation"title="Permalink to this headline">¶</a></h2>
<p>The best way to install <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> is to use <codeclass="docutils literal notranslate"><spanclass="pre">sudo</span><spanclass="pre">dnf</span><spanclass="pre">install</span>
<spanclass="pre">lorax-composer</span><spanclass="pre">composer-cli</span></code>, this will setup the weldr user and install the
systemd socket activation service. You will then need to enable it with <codeclass="docutils literal notranslate"><spanclass="pre">sudo</span>
<h2>Quickstart<aclass="headerlink"href="#quickstart"title="Permalink to this headline">¶</a></h2>
<olclass="arabic simple">
<li>Create a <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code> user and group by running <codeclass="docutils literal notranslate"><spanclass="pre">useradd</span><spanclass="pre">weldr</span></code></li>
<li>Remove any pre-existing socket directory with <codeclass="docutils literal notranslate"><spanclass="pre">rm</span><spanclass="pre">-rf</span><spanclass="pre">/run/weldr/</span></code>
A new directory with correct permissions will be created the first time the server runs.</li>
<li>Enable the socket activation with <codeclass="docutils literal notranslate"><spanclass="pre">systemctl</span><spanclass="pre">enable</span><spanclass="pre">lorax-composer.socket</span>
<p>NOTE: You can also run it directly with <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span><spanclass="pre">/path/to/blueprints</span></code>. However,
<codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> does not react well to being started both on the command line and via
socket activation at the same time. It is therefore recommended that you run it directly
on the command line only for testing or development purposes. For real use or development
of other projects that simply use the API, you should stick to socket activation only.</p>
<h2>Logs<aclass="headerlink"href="#logs"title="Permalink to this headline">¶</a></h2>
<p>Logs are stored under <codeclass="docutils literal notranslate"><spanclass="pre">/var/log/lorax-composer/</span></code> and include all console
messages as well as extra debugging info and API requests.</p>
</div>
<divclass="section"id="security">
<h2>Security<aclass="headerlink"href="#security"title="Permalink to this headline">¶</a></h2>
<p>Some security related issues that you should be aware of before running <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>:</p>
<ulclass="simple">
<li>One of the API server threads needs to retain root privileges in order to run Anaconda.</li>
<li>Only allow authorized users access to the <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code> group and socket.</li>
</ul>
<p>Since Anaconda kickstarts are used there is the possibility that a user could
inject commands into a blueprint that would result in the kickstart executing
arbitrary code on the host. Only authorized users should be allowed to build
images using <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>.</p>
<h2>How it Works<aclass="headerlink"href="#how-it-works"title="Permalink to this headline">¶</a></h2>
<p>The server runs as root, and as <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code>. Communication with it is via a unix
domain socket (<codeclass="docutils literal notranslate"><spanclass="pre">/run/weldr/api.socket</span></code> by default). The directory and socket
are owned by <codeclass="docutils literal notranslate"><spanclass="pre">root:weldr</span></code> so that any user in the <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code> group can use the API
to control <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>.</p>
<p>At startup the server will check for the correct permissions and
ownership of a pre-existing directory, or it will create a new one if it
doesn’t exist. The socket path and group owner’s name can be changed from the
cmdline by passing it the <codeclass="docutils literal notranslate"><spanclass="pre">--socket</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">--group</span></code> arguments.</p>
<p>It will then drop root privileges for the API thread and run as the <codeclass="docutils literal notranslate"><spanclass="pre">weldr</span></code>
user. The queue and compose thread still runs as root because it needs to be
able to mount/umount files and run Anaconda.</p>
</div>
<divclass="section"id="composing-images">
<h2>Composing Images<aclass="headerlink"href="#composing-images"title="Permalink to this headline">¶</a></h2>
<p>The <aclass="reference external"href="https://github.com/weldr/welder-web/">welder-web</a> GUI project can be used to construct
blueprints and create composes using a web browser.</p>
<p>Or use the command line with <aclass="reference external"href="composer-cli.html">composer-cli</a>.</p>
</div>
<divclass="section"id="blueprints">
<h2>Blueprints<aclass="headerlink"href="#blueprints"title="Permalink to this headline">¶</a></h2>
<p>Blueprints are simple text files in <aclass="reference external"href="https://github.com/toml-lang/toml">TOML</a> format that describe
which packages, and what versions, to install into the image. They can also define a limited set
of customizations to make to the final image.</p>
<p>Example blueprints can be found in the <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code><aclass="reference external"href="https://github.com/weldr/lorax/tree/master/tests/pylorax/blueprints/">test suite</a>, with a simple one
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">name</span></code> field is the name of the blueprint. It can contain spaces, but they will be converted to <codeclass="docutils literal notranslate"><spanclass="pre">-</span></code>
when it is written to disk. It should be short and descriptive.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">description</span></code> can be a longer description of the blueprint, it is only used for display purposes.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">version</span></code> is a <aclass="reference external"href="https://semver.org/">semver compatible</a> version number. If
a new blueprint is uploaded with the same <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code> the server will
automatically bump the PATCH level of the <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code>. If the <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code>
doesn’t match it will be used as is. eg. Uploading a blueprint with <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code>
set to <codeclass="docutils literal notranslate"><spanclass="pre">0.1.0</span></code> when the existing blueprint <codeclass="docutils literal notranslate"><spanclass="pre">version</span></code> is <codeclass="docutils literal notranslate"><spanclass="pre">0.0.1</span></code> will
result in the new blueprint being stored as <codeclass="docutils literal notranslate"><spanclass="pre">version</span><spanclass="pre">0.1.0</span></code>.</p>
<h3>[[packages]] and [[modules]]<aclass="headerlink"href="#packages-and-modules"title="Permalink to this headline">¶</a></h3>
<p>These entries describe the package names and matching version glob to be installed into the image.</p>
<p>The names must match the names exactly, and the versions can be an exact match
or a filesystem-like glob of the version using <codeclass="docutils literal notranslate"><spanclass="pre">*</span></code> wildcards and <codeclass="docutils literal notranslate"><spanclass="pre">?</span></code>
character matching.</p>
<p>NOTE: As of lorax-composer-29.2-1 the versions are not used for depsolving,
that is planned for a future release. And currently there are no differences
between <codeclass="docutils literal notranslate"><spanclass="pre">packages</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">modules</span></code> in <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>.</p>
<h3>Customizations<aclass="headerlink"href="#customizations"title="Permalink to this headline">¶</a></h3>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">[[customizations]]</span></code> section can be used to configure the hostname of the final image. eg.:</p>
<p>The key will be added to the user’s authorized_keys file.</p>
</div>
<divclass="section"id="customizations-user">
<h4>[[customizations.user]]<aclass="headerlink"href="#customizations-user"title="Permalink to this headline">¶</a></h4>
<p>Add a user to the image, and/or set their ssh key.
All fields for this section are optional except for the <codeclass="docutils literal notranslate"><spanclass="pre">name</span></code>, here is a complete example:</p>
<p>If the password starts with <codeclass="docutils literal notranslate"><spanclass="pre">$6$</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">$5$</span></code>, or <codeclass="docutils literal notranslate"><spanclass="pre">$2b$</span></code> it will be stored as
an encrypted password. Otherwise it will be treated as a plain text password.</p>
</div>
<divclass="section"id="customizations-group">
<h4>[[customizations.group]]<aclass="headerlink"href="#customizations-group"title="Permalink to this headline">¶</a></h4>
<p>Add a group to the image. <codeclass="docutils literal notranslate"><spanclass="pre">name</span></code> is required and <codeclass="docutils literal notranslate"><spanclass="pre">gid</span></code> is optional:</p>
<h2>Adding Output Types<aclass="headerlink"href="#adding-output-types"title="Permalink to this headline">¶</a></h2>
<p><codeclass="docutils literal notranslate"><spanclass="pre">livemedia-creator</span></code> supports a large number of output types, and only some of
these are currently available via <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code>. To add a new output type to
lorax-composer a kickstart file needs to be added to <codeclass="docutils literal notranslate"><spanclass="pre">./share/composer/</span></code>. The
name of the kickstart is what will be used by the <codeclass="docutils literal notranslate"><spanclass="pre">/compose/types</span></code> route, and the
<codeclass="docutils literal notranslate"><spanclass="pre">compose_type</span></code> field of the POST to start a compose. It also needs to have
code added to the <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.compose_args"title="pylorax.api.compose.compose_args"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.compose_args()</span></code></a> function. The
<aclass="reference internal"href="pylorax.html#pylorax.installer.novirt_install"title="pylorax.installer.novirt_install"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.installer.novirt_install()</span></code></a> when it runs the compose. When the
<aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.move_compose_results"title="pylorax.api.compose.move_compose_results"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.move_compose_results()</span></code></a> handles this for each type.
<li>A new entry in the _MAP in <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.compose_args"title="pylorax.api.compose.compose_args"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.compose_args()</span></code></a></li>
<li>Add a bit of code to <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.move_compose_results"title="pylorax.api.compose.move_compose_results"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">pylorax.api.compose.move_compose_results()</span></code></a> to move the disk image from
the compose directory to the results directory.</li>
</ul>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">partitioned-disk.ks</span></code> is pretty similar to the example minimal kickstart
in <codeclass="docutils literal notranslate"><spanclass="pre">./docs/fedora-minimal.ks</span></code>. You should remove the <codeclass="docutils literal notranslate"><spanclass="pre">url</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">repo</span></code>
commands, they will be added by the compose process. Make sure the bootloader
packages are included in the <codeclass="docutils literal notranslate"><spanclass="pre">%packages</span></code> section at the end of the kickstart,
and you will want to leave off the <codeclass="docutils literal notranslate"><spanclass="pre">%end</span></code> so that the compose can append the
list of packages from the blueprint.</p>
<p>The new <codeclass="docutils literal notranslate"><spanclass="pre">_MAP</span></code> entry should be a copy of one of the existing entries, but with <codeclass="docutils literal notranslate"><spanclass="pre">make_disk</span></code> set
to <codeclass="docutils literal notranslate"><spanclass="pre">True</span></code>. Make sure that none of the other <codeclass="docutils literal notranslate"><spanclass="pre">make_*</span></code> options are <codeclass="docutils literal notranslate"><spanclass="pre">True</span></code>. The <codeclass="docutils literal notranslate"><spanclass="pre">image_name</span></code> is
what the name of the final image will be.</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">move_compose_results()</span></code> can be as simple as moving the output file into
the results directory, or it could do some post-processing on it. The end of
the function should always clean up the <codeclass="docutils literal notranslate"><spanclass="pre">./compose/</span></code> directory, removing any
unneeded extra files. This is especially true for the <codeclass="docutils literal notranslate"><spanclass="pre">live-iso</span></code> since it produces
the contents of the iso as well as the boot.iso itself.</p>
<h2>Package Sources<aclass="headerlink"href="#package-sources"title="Permalink to this headline">¶</a></h2>
<p>By default lorax-composer uses the host’s configured repositories. It copies
the <codeclass="docutils literal notranslate"><spanclass="pre">*.repo</span></code> files from <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code> into
<codeclass="docutils literal notranslate"><spanclass="pre">/var/lib/lorax/composer/repos.d/</span></code> at startup, these are immutable system
repositories and cannot be deleted or changed. If you want to add additional
repos you can put them into <codeclass="docutils literal notranslate"><spanclass="pre">/var/lib/lorax/composer/repos.d/</span></code> or use the
<codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/*</span></code> API routes to create them.</p>
<p>The new source can be added by doing a POST to the <codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/new</span></code>
route using JSON (with <cite>Content-Type</cite> header set to <cite>application/json</cite>) or TOML
(with it set to <cite>text/x-toml</cite>). The format of the source looks like this (in
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">proxy</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">gpgkey_urls</span></code> entries are optional. All of the others are required. The supported
types for the urls are:</p>
<ulclass="simple">
<li><codeclass="docutils literal notranslate"><spanclass="pre">yum-baseurl</span></code> is a URL to a yum repository.</li>
<li><codeclass="docutils literal notranslate"><spanclass="pre">yum-mirrorlist</span></code> is a URL for a mirrorlist.</li>
<li><codeclass="docutils literal notranslate"><spanclass="pre">yum-metalink</span></code> is a URL for a metalink.</li>
</ul>
<p>If <codeclass="docutils literal notranslate"><spanclass="pre">check_ssl</span></code> is true the https certificates must be valid. If they are self-signed you can either set
this to false, or add your Certificate Authority to the host system.</p>
<p>If <codeclass="docutils literal notranslate"><spanclass="pre">check_gpg</span></code> is true the GPG key must either be installed on the host system, or <codeclass="docutils literal notranslate"><spanclass="pre">gpgkey_urls</span></code>
should point to it.</p>
<p>You can edit an existing source (other than system sources), by doing a POST to the <codeclass="docutils literal notranslate"><spanclass="pre">new</span></code> route
with the new version of the source. It will overwrite the previous one.</p>
<p>A list of existing sources is available from <codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/list</span></code>, and detailed info
on a source can be retrieved with the <codeclass="docutils literal notranslate"><spanclass="pre">/api/v0/projects/source/info/<source-name></span></code> route. By default
it returns JSON but it can also return TOML if <codeclass="docutils literal notranslate"><spanclass="pre">?format=toml</span></code> is added to the request.</p>
<p>Non-system sources can be deleted by doing a <codeclass="docutils literal notranslate"><spanclass="pre">DELETE</span></code> request to the
<p>The documentation for the source API routes can be <aclass="reference external"href="pylorax.api.html#api-v0-projects-source-list">found here</a></p>
<p>The configured sources are used for all blueprint depsolve operations, and for composing images.
When adding additional sources you must make sure that the packages in the source do not
conflict with any other package sources, otherwise depsolving will fail.</p>
<h3>DVD ISO Package Source<aclass="headerlink"href="#dvd-iso-package-source"title="Permalink to this headline">¶</a></h3>
<p>In some situations the system may want to <em>only</em> use a DVD iso as the package
source, not the repos from the network. <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">anaconda</span></code>
understand <codeclass="docutils literal notranslate"><spanclass="pre">file://</span></code> URLs so you can mount an iso on the host, and replace the
system repo files with a configuration file pointing to the DVD.</p>
<ul>
<li><pclass="first">Stop the <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer.service</span></code> if it is running</p>
</li>
<li><pclass="first">Move the repo files in <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code> someplace safe</p>
</li>
<li><pclass="first">Create a new <codeclass="docutils literal notranslate"><spanclass="pre">iso.repo</span></code> file in <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code>:</p>
<li><pclass="first">Remove all the cached repo files from <codeclass="docutils literal notranslate"><spanclass="pre">/var/lib/lorax/composer/repos/</span></code></p>
</li>
<li><pclass="first">Restart the <codeclass="docutils literal notranslate"><spanclass="pre">lorax-composer.service</span></code></p>
</li>
<li><pclass="first">Check the output of <codeclass="docutils literal notranslate"><spanclass="pre">composer-cli</span><spanclass="pre">status</span><spanclass="pre">show</span></code> for any output specific depsolve errors.
For example, the DVD usually does not include <codeclass="docutils literal notranslate"><spanclass="pre">grub2-efi-*-cdboot-*</span></code> so the live-iso image
type will not be available.</p>
</li>
</ul>
<p>If you want to <em>add</em> the DVD source to the existing sources you can do that by
mounting the iso and creating a source file to point to it as described in the
<aclass="reference internal"href="#package-sources">Package Sources</a> documentation. In that case there is no need to remove the other
sources from <codeclass="docutils literal notranslate"><spanclass="pre">/etc/yum.repos.d/</span></code> or clear the cached repos.</p>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
