<trclass="field-odd field"><thclass="field-name">Authors:</th><tdclass="field-body">Brian C. Lane <<aclass="reference external"href="mailto:bcl%40redhat.com">bcl<span>@</span>redhat<span>.</span>com</a>></td>
<p><ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt> is an API server that allows you to build disk images using
<aclass="reference internal"href="#blueprints">Blueprints</a> to describe the package versions to be installed into the image.
It is compatible with the Weldr project’s bdcs-api REST protocol. More
information on Weldr can be found <aclass="reference external"href="http://www.weldr.io">on the Weldr blog</a>.</p>
<p>Behind the scenes it uses <aclass="reference external"href="livemedia-creator.html">livemedia-creator</a> and
<aclass="reference external"href="https://anaconda-installer.readthedocs.io/en/latest/">Anaconda</a> to handle the
installation and configuration of the images.</p>
<divclass="section"id="important-things-to-note">
<h2>Important Things To Note<aclass="headerlink"href="#important-things-to-note"title="Permalink to this headline">¶</a></h2>
<ulclass="simple">
<li>SELinux must be in Permissive mode. Anaconda requires SELinux be in permissive mode
for image creation to work correctly. You can either edit the setting in the
<ttclass="docutils literal"><spanclass="pre">/etc/sysconfig/selinux</span></tt> file, or run <ttclass="docutils literal"><spanclass="pre">setenforce</span><spanclass="pre">0</span></tt> before starting lorax-composer.</li>
<li>Some output types require packages from the RHEL 7 Optional repository. See the
<aclass="reference external"href="https://access.redhat.com/solutions/392003">Red Hat Enterprise Linux 7 documentation</a>
for information on how to enable it. Otherwise you will see image creation fail to
depsolve even if the blueprint itself is correct.</li>
</ul>
</div>
<divclass="section"id="installation">
<h2>Installation<aclass="headerlink"href="#installation"title="Permalink to this headline">¶</a></h2>
<p>The best way to install <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt> is to use <ttclass="docutils literal"><spanclass="pre">sudo</span><spanclass="pre">dnf</span><spanclass="pre">install</span>
<spanclass="pre">lorax-composer</span><spanclass="pre">composer-cli</span></tt>, this will setup the weldr user and install the
systemd socket activation service. You will then need to enable it with <ttclass="docutils literal"><spanclass="pre">sudo</span>
<h2>Quickstart<aclass="headerlink"href="#quickstart"title="Permalink to this headline">¶</a></h2>
<olclass="arabic simple">
<li>Create a <ttclass="docutils literal"><spanclass="pre">weldr</span></tt> user and group by running <ttclass="docutils literal"><spanclass="pre">useradd</span><spanclass="pre">weldr</span></tt></li>
<li>Remove any pre-existing socket directory with <ttclass="docutils literal"><spanclass="pre">rm</span><spanclass="pre">-rf</span><spanclass="pre">/run/weldr/</span></tt>
A new directory with correct permissions will be created the first time the server runs.</li>
<li>Enable the socket activation with <ttclass="docutils literal"><spanclass="pre">systemctl</span><spanclass="pre">enable</span><spanclass="pre">lorax-composer.socket</span>
<p>NOTE: You can also run it directly with <ttclass="docutils literal"><spanclass="pre">lorax-composer</span><spanclass="pre">/path/to/blueprints</span></tt>. However,
<ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt> does not react well to being started both on the command line and via
socket activation at the same time. It is therefore recommended that you run it directly
on the command line only for testing or development purposes. For real use or development
of other projects that simply use the API, you should stick to socket activation only.</p>
<h2>Logs<aclass="headerlink"href="#logs"title="Permalink to this headline">¶</a></h2>
<p>Logs are stored under <ttclass="docutils literal"><spanclass="pre">/var/log/lorax-composer/</span></tt> and include all console
messages as well as extra debugging info and API requests.</p>
</div>
<divclass="section"id="security">
<h2>Security<aclass="headerlink"href="#security"title="Permalink to this headline">¶</a></h2>
<p>Some security related issues that you should be aware of before running <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt>:</p>
<ulclass="simple">
<li>One of the API server threads needs to retain root privileges in order to run Anaconda.</li>
<li>SELinux must be set to Permissive or disabled to allow <ttclass="docutils literal"><spanclass="pre">livemedia-creator</span></tt> to run Anaconda.</li>
<li>Only allow authorized users access to the <ttclass="docutils literal"><spanclass="pre">weldr</span></tt> group and socket.</li>
</ul>
<p>Since Anaconda kickstarts are used there is the possibility that a user could
inject commands into a blueprint that would result in the kickstart executing
arbitrary code on the host. Only authorized users should be allowed to build
images using <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt>.</p>
</div>
<divclass="section"id="how-it-works">
<h2>How it Works<aclass="headerlink"href="#how-it-works"title="Permalink to this headline">¶</a></h2>
<p>The server runs as root, and as <ttclass="docutils literal"><spanclass="pre">weldr</span></tt>. Communication with it is via a unix
domain socket (<ttclass="docutils literal"><spanclass="pre">/run/weldr/api.socket</span></tt> by default). The directory and socket
are owned by <ttclass="docutils literal"><spanclass="pre">root:weldr</span></tt> so that any user in the <ttclass="docutils literal"><spanclass="pre">weldr</span></tt> group can use the API
to control <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt>.</p>
<p>At startup the server will check for the correct permissions and
ownership of a pre-existing directory, or it will create a new one if it
doesn’t exist. The socket path and group owner’s name can be changed from the
cmdline by passing it the <ttclass="docutils literal"><spanclass="pre">--socket</span></tt> and <ttclass="docutils literal"><spanclass="pre">--group</span></tt> arguments.</p>
<p>It will then drop root privileges for the API thread and run as the <ttclass="docutils literal"><spanclass="pre">weldr</span></tt>
user. The queue and compose thread still runs as root because it needs to be
<p>The <aclass="reference external"href="https://github.com/weldr/welder-web/">welder-web</a> GUI project can be used to construct
blueprints and create composes using a web browser.</p>
<p>Or use the command line with <aclass="reference external"href="composer-cli.html">composer-cli</a>.</p>
</div>
<divclass="section"id="blueprints">
<h2>Blueprints<aclass="headerlink"href="#blueprints"title="Permalink to this headline">¶</a></h2>
<p>Blueprints are simple text files in <aclass="reference external"href="https://github.com/toml-lang/toml">TOML</a> format that describe
which packages, and what versions, to install into the image. They can also define a limited set
of customizations to make to the final image.</p>
<p>Example blueprints can be found in the <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt><aclass="reference external"href="https://github.com/weldr/lorax/tree/master/tests/pylorax/blueprints/">test suite</a>, with a simple one
<p>The <ttclass="docutils literal"><spanclass="pre">name</span></tt> field is the name of the blueprint. It can contain spaces, but they will be converted to <ttclass="docutils literal"><spanclass="pre">-</span></tt>
when it is written to disk. It should be short and descriptive.</p>
<p><ttclass="docutils literal"><spanclass="pre">description</span></tt> can be a longer description of the blueprint, it is only used for display purposes.</p>
<p><ttclass="docutils literal"><spanclass="pre">version</span></tt> is a <aclass="reference external"href="https://semver.org/">semver compatible</a> version number. If
a new blueprint is uploaded with the same <ttclass="docutils literal"><spanclass="pre">version</span></tt> the server will
automatically bump the PATCH level of the <ttclass="docutils literal"><spanclass="pre">version</span></tt>. If the <ttclass="docutils literal"><spanclass="pre">version</span></tt>
doesn’t match it will be used as is. eg. Uploading a blueprint with <ttclass="docutils literal"><spanclass="pre">version</span></tt>
set to <ttclass="docutils literal"><spanclass="pre">0.1.0</span></tt> when the existing blueprint <ttclass="docutils literal"><spanclass="pre">version</span></tt> is <ttclass="docutils literal"><spanclass="pre">0.0.1</span></tt> will
result in the new blueprint being stored as <ttclass="docutils literal"><spanclass="pre">version</span><spanclass="pre">0.1.0</span></tt>.</p>
<divclass="section"id="packages-and-modules">
<h3>[[packages]] and [[modules]]<aclass="headerlink"href="#packages-and-modules"title="Permalink to this headline">¶</a></h3>
<p>These entries describe the package names and matching version glob to be installed into the image.</p>
<p>The names must match the names exactly, and the versions can be an exact match
or a filesystem-like glob of the version using <ttclass="docutils literal"><spanclass="pre">*</span></tt> wildcards and <ttclass="docutils literal"><spanclass="pre">?</span></tt>
character matching.</p>
<p>NOTE: As of lorax-composer-29.2-1 the versions are not used for depsolving,
that is planned for a future release. And currently there are no differences
between <ttclass="docutils literal"><spanclass="pre">packages</span></tt> and <ttclass="docutils literal"><spanclass="pre">modules</span></tt> in <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt>.</p>
<h3>Customizations<aclass="headerlink"href="#customizations"title="Permalink to this headline">¶</a></h3>
<p>The <ttclass="docutils literal"><spanclass="pre">[[customizations]]</span></tt> section can be used to configure the hostname of the final image. eg.:</p>
<p>If the password starts with <ttclass="docutils literal"><spanclass="pre">$6$</span></tt>, <ttclass="docutils literal"><spanclass="pre">$5$</span></tt>, or <ttclass="docutils literal"><spanclass="pre">$2b$</span></tt> it will be stored as
an encrypted password. Otherwise it will be treated as a plain text password.</p>
</div>
<divclass="section"id="customizations-group">
<h4>[[customizations.group]]<aclass="headerlink"href="#customizations-group"title="Permalink to this headline">¶</a></h4>
<p>Add a group to the image. <ttclass="docutils literal"><spanclass="pre">name</span></tt> is required and <ttclass="docutils literal"><spanclass="pre">gid</span></tt> is optional:</p>
lorax-composer a kickstart file needs to be added to <ttclass="docutils literal"><spanclass="pre">./share/composer/</span></tt>. The
name of the kickstart is what will be used by the <ttclass="docutils literal"><spanclass="pre">/compose/types</span></tt> route, and the
<ttclass="docutils literal"><spanclass="pre">compose_type</span></tt> field of the POST to start a compose. It also needs to have
code added to the <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.compose_args"title="pylorax.api.compose.compose_args"><ttclass="xref py py-func docutils literal"><spanclass="pre">pylorax.api.compose.compose_args()</span></tt></a> function. The
<ttclass="docutils literal"><spanclass="pre">_MAP</span></tt> entry in this function defines what lorax-composer will pass to
<aclass="reference internal"href="pylorax.html#pylorax.installer.novirt_install"title="pylorax.installer.novirt_install"><ttclass="xref py py-func docutils literal"><spanclass="pre">pylorax.installer.novirt_install()</span></tt></a> when it runs the compose. When the
compose is finished the output files need to be copied out of the build
<aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.move_compose_results"title="pylorax.api.compose.move_compose_results"><ttclass="xref py py-func docutils literal"><spanclass="pre">pylorax.api.compose.move_compose_results()</span></tt></a> handles this for each type.
You should move them instead of copying to save space.</p>
<p>If the new output type does not have support in livemedia-creator it should be
added there first. This will make the output available to the widest number of
<h3>Example: Add partitioned disk support<aclass="headerlink"href="#example-add-partitioned-disk-support"title="Permalink to this headline">¶</a></h3>
<p>Partitioned disk support is something that livemedia-creator already supports
via the <ttclass="docutils literal"><spanclass="pre">--make-disk</span></tt> cmdline argument. To add this to lorax-composer it
needs 3 things:</p>
<ulclass="simple">
<li>A <ttclass="docutils literal"><spanclass="pre">partitioned-disk.ks</span></tt> file in <ttclass="docutils literal"><spanclass="pre">./share/composer/</span></tt></li>
<li>A new entry in the _MAP in <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.compose_args"title="pylorax.api.compose.compose_args"><ttclass="xref py py-func docutils literal"><spanclass="pre">pylorax.api.compose.compose_args()</span></tt></a></li>
<li>Add a bit of code to <aclass="reference internal"href="pylorax.api.html#pylorax.api.compose.move_compose_results"title="pylorax.api.compose.move_compose_results"><ttclass="xref py py-func docutils literal"><spanclass="pre">pylorax.api.compose.move_compose_results()</span></tt></a> to move the disk image from
the compose directory to the results directory.</li>
</ul>
<p>The <ttclass="docutils literal"><spanclass="pre">partitioned-disk.ks</span></tt> is pretty similar to the example minimal kickstart
in <ttclass="docutils literal"><spanclass="pre">./docs/rhel7-minimal.ks</span></tt>. You should remove the <ttclass="docutils literal"><spanclass="pre">url</span></tt> and <ttclass="docutils literal"><spanclass="pre">repo</span></tt>
commands, they will be added by the compose process. Make sure the bootloader
packages are included in the <ttclass="docutils literal"><spanclass="pre">%packages</span></tt> section at the end of the kickstart,
and you will want to leave off the <ttclass="docutils literal"><spanclass="pre">%end</span></tt> so that the compose can append the
<p>The new <ttclass="docutils literal"><spanclass="pre">_MAP</span></tt> entry should be a copy of one of the existing entries, but with <ttclass="docutils literal"><spanclass="pre">make_disk</span></tt> set
to <ttclass="docutils literal"><spanclass="pre">True</span></tt>. Make sure that none of the other <ttclass="docutils literal"><spanclass="pre">make_*</span></tt> options are <ttclass="docutils literal"><spanclass="pre">True</span></tt>. The <ttclass="docutils literal"><spanclass="pre">image_name</span></tt> is
what the name of the final image will be.</p>
<p><ttclass="docutils literal"><spanclass="pre">move_compose_results()</span></tt> can be as simple as moving the output file into
the results directory, or it could do some post-processing on it. The end of
the function should always clean up the <ttclass="docutils literal"><spanclass="pre">./compose/</span></tt> directory, removing any
unneeded extra files. This is especially true for the <ttclass="docutils literal"><spanclass="pre">live-iso</span></tt> since it produces
the contents of the iso as well as the boot.iso itself.</p>
<h2>Package Sources<aclass="headerlink"href="#package-sources"title="Permalink to this headline">¶</a></h2>
<p>By default lorax-composer uses the host’s configured repositories. It copies
the <ttclass="docutils literal"><spanclass="pre">*.repo</span></tt> files from <ttclass="docutils literal"><spanclass="pre">/etc/yum.repos.d/</span></tt> into
<ttclass="docutils literal"><spanclass="pre">/var/lib/lorax/composer/repos.d/</span></tt> at startup, these are immutable system
repositories and cannot be deleted or changed. If you want to add additional
repos you can put them into <ttclass="docutils literal"><spanclass="pre">/var/lib/lorax/composer/repos.d/</span></tt> or use the
<ttclass="docutils literal"><spanclass="pre">/api/v0/projects/source/*</span></tt> API routes to create them.</p>
<p>The new source can be added by doing a POST to the <ttclass="docutils literal"><spanclass="pre">/api/v0/projects/source/new</span></tt>
route using JSON (with <cite>Content-Type</cite> header set to <cite>application/json</cite>) or TOML
(with it set to <cite>text/x-toml</cite>). The format of the source looks like this (in
<p>The <ttclass="docutils literal"><spanclass="pre">proxy</span></tt> and <ttclass="docutils literal"><spanclass="pre">gpgkey_urls</span></tt> entries are optional. All of the others are required. The supported
types for the urls are:</p>
<ulclass="simple">
<li><ttclass="docutils literal"><spanclass="pre">yum-baseurl</span></tt> is a URL to a yum repository.</li>
<li><ttclass="docutils literal"><spanclass="pre">yum-mirrorlist</span></tt> is a URL for a mirrorlist.</li>
<li><ttclass="docutils literal"><spanclass="pre">yum-metalink</span></tt> is a URL for a metalink.</li>
</ul>
<p>If <ttclass="docutils literal"><spanclass="pre">check_ssl</span></tt> is true the https certificates must be valid. If they are self-signed you can either set
this to false, or add your Certificate Authority to the host system.</p>
<p>If <ttclass="docutils literal"><spanclass="pre">check_gpg</span></tt> is true the GPG key must either be installed on the host system, or <ttclass="docutils literal"><spanclass="pre">gpgkey_urls</span></tt>
should point to it.</p>
<p>You can edit an existing source (other than system sources), by doing a POST to the <ttclass="docutils literal"><spanclass="pre">new</span></tt> route
with the new version of the source. It will overwrite the previous one.</p>
<p>A list of existing sources is available from <ttclass="docutils literal"><spanclass="pre">/api/v0/projects/source/list</span></tt>, and detailed info
on a source can be retrieved with the <ttclass="docutils literal"><spanclass="pre">/api/v0/projects/source/info/<source-name></span></tt> route. By default
it returns JSON but it can also return TOML if <ttclass="docutils literal"><spanclass="pre">?format=toml</span></tt> is added to the request.</p>
<p>Non-system sources can be deleted by doing a <ttclass="docutils literal"><spanclass="pre">DELETE</span></tt> request to the
<p>The documentation for the source API routes can be <aclass="reference external"href="pylorax.api.html#api-v0-projects-source-list">found here</a></p>
<p>The configured sources are used for all blueprint depsolve operations, and for composing images.
When adding additional sources you must make sure that the packages in the source do not
conflict with any other package sources, otherwise depsolving will fail.</p>
<h3>DVD ISO Package Source<aclass="headerlink"href="#dvd-iso-package-source"title="Permalink to this headline">¶</a></h3>
<p>In some situations the system may want to <em>only</em> use a DVD iso as the package
source, not the repos from the network. <ttclass="docutils literal"><spanclass="pre">lorax-composer</span></tt> and <ttclass="docutils literal"><spanclass="pre">anaconda</span></tt>
understand <ttclass="docutils literal"><spanclass="pre">file://</span></tt> URLs so you can mount an iso on the host, and replace the
system repo files with a configuration file pointing to the DVD.</p>
<ul>
<li><pclass="first">Stop the <ttclass="docutils literal"><spanclass="pre">lorax-composer.service</span></tt> if it is running</p>
</li>
<li><pclass="first">Move the repo files in <ttclass="docutils literal"><spanclass="pre">/etc/yum.repos.d/</span></tt> someplace safe</p>
</li>
<li><pclass="first">Create a new <ttclass="docutils literal"><spanclass="pre">iso.repo</span></tt> file in <ttclass="docutils literal"><spanclass="pre">/etc/yum.repos.d/</span></tt>:</p>
<li><pclass="first">Remove all the cached repo files from <ttclass="docutils literal"><spanclass="pre">/var/lib/lorax/composer/repos/</span></tt></p>
</li>
<li><pclass="first">Restart the <ttclass="docutils literal"><spanclass="pre">lorax-composer.service</span></tt></p>
</li>
<li><pclass="first">Check the output of <ttclass="docutils literal"><spanclass="pre">composer-cli</span><spanclass="pre">status</span><spanclass="pre">show</span></tt> for any output specific depsolve errors.
For example, the DVD usually does not include <ttclass="docutils literal"><spanclass="pre">grub2-efi-*-cdboot-*</span></tt> so the live-iso image
type will not be available.</p>
</li>
</ul>
<p>If you want to <em>add</em> the DVD source to the existing sources you can do that by
mounting the iso and creating a source file to point to it as described in the
<aclass="reference internal"href="#package-sources">Package Sources</a> documentation. In that case there is no need to remove the other
sources from <ttclass="docutils literal"><spanclass="pre">/etc/yum.repos.d/</span></tt> or clear the cached repos.</p>
