From 2c81a4c1dbf792f8e857030271ed2cadd5ed7368 Mon Sep 17 00:00:00 2001 From: "Brian C. Lane" Date: Thu, 28 Jun 2018 15:25:49 -0700 Subject: [PATCH] lorax-composer: Update documentation This updates the documentation to include composer-cli instead of curl, warnings about SELinux and needing the RHEL 7 Optional package repo, as well as other documentation improvements from the master branch. --- docs/composer-cli.rst | 62 +++++++++++++ docs/index.rst | 1 + docs/lorax-composer.rst | 201 +++++++++++++++++++++++++++++++++------- 3 files changed, 233 insertions(+), 31 deletions(-) create mode 100644 docs/composer-cli.rst diff --git a/docs/composer-cli.rst b/docs/composer-cli.rst new file mode 100644 index 00000000..366e01ad --- /dev/null +++ b/docs/composer-cli.rst @@ -0,0 +1,62 @@ +composer-cli +============ + +:Authors: + Brian C. Lane + +``composer-cli`` is used to interact with the ``lorax-composer`` API server, managing blueprints, exploring available packages, and building new images. + +It requires `lorax-composer `_ to be installed on the +local system, and the user running it needs to be a member of the ``weldr`` +group. They do not need to be root, but all of the `security precautions +`_ apply. + +composer-cli cmdline arguments +------------------------------ + +.. argparse:: + :ref: composer.cli.cmdline.composer_cli_parser + :prog: composer-cli + +Edit a Blueprint +---------------- + +Start out by listing the available blueprints using ``composer-cli blueprints +list``, pick one and save it to the local directory by running ``composer-cli +blueprints save http-server``. If there are no blueprints available you can +copy one of the examples `from the test suite +`_. + +Edit the file (it will be saved with a .toml extension) and change the +description, add a package or module to it. Send it back to the server by +running ``composer-cli blueprints push http-server.toml``. You can verify that it was +saved by viewing the changelog - ``composer-cli blueprints changes http-server``. + +Build an image +---------------- + +Build a ``qcow2`` disk image from this blueprint by running ``composer-cli +compose start http-server qcow2``. It will print a UUID that you can use to +keep track of the build. You can also cancel the build if needed. + +The available types of images is displayed by ``composer-cli compose types``. +Currently this consists of: ext4-filesystem, live-iso, partitioned-disk, qcow2, +tar + +Monitor the build status +------------------------ + +Monitor it using ``composer-cli compose status``, which will show the status of +all the builds on the system. You can view the end of the anaconda build logs +once it is in the ``RUNNING`` state using ``composer-cli compose log UUID`` +where UUID is the UUID returned by the start command. + +Once the build is in the ``FINISHED`` state you can download the image. + +Download the image +------------------ + +Downloading the final image is done with ``composer-cli compose image UUID`` and it will +save the qcow2 image as ``UUID-disk.qcow2`` which you can then use to boot a VM like this:: + + qemu-kvm --name test-image -m 1024 -hda ./UUID-disk.qcow2 diff --git a/docs/index.rst b/docs/index.rst index acd79678..67f2b9ed 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -15,6 +15,7 @@ Contents: lorax livemedia-creator lorax-composer + composer-cli product-images modules diff --git a/docs/lorax-composer.rst b/docs/lorax-composer.rst index 737d07bb..580338f4 100644 --- a/docs/lorax-composer.rst +++ b/docs/lorax-composer.rst @@ -4,29 +4,37 @@ lorax-composer :Authors: Brian C. Lane -lorax-composer is an API server that is compatible with the Weldr project's -bdcs-api REST protocol. More information on Weldr can be found `on the Weldr -blog `_. +``lorax-composer`` is an API server that allows you to build disk images using +`Blueprints`_ to describe the package versions to be installed into the image. +It is compatible with the Weldr project's bdcs-api REST protocol. More +information on Weldr can be found `on the Weldr blog `_. -The server runs as root, and communication with it is via a unix domain socket -(``/run/weldr/api.socket`` by default). The directory and socket are owned by -root:weldr so that any user in the weldr group can use the API to control -lorax-composer. +Behind the scenes it uses `livemedia-creator `_ and +`Anaconda `_ to handle the +installation and configuration of the images. -When starting the server it will check for the correct permissions and -ownership of a pre-existing directory, or it will create a new one if it -doesn't exist. The socket path and group owner's name can be changed from the -cmdline by passing it the ``--socket`` and ``--group`` arguments. +Important Things To Note +------------------------ -As of version 19.7.7 it will drop root privileges for the API thread. The queue -and compose thread still runs as root because it needs to be able to -mount/umount files and run Anaconda. +* SELinux must be in Permissive mode. Anaconda requires SELinux be in permissive mode + for image creation to work correctly. You can either edit the setting in the + ``/etc/sysconfig/selinux`` file, or run ``setenforce 0`` before starting lorax-composer. -Logs ----- +* Some output types require packages from the RHEL 7 Optional repository. See the + `Red Hat Enterprise Linux 7 documentation `_ + for information on how to enable it. Otherwise you will see image creation fail to + depsolve even if the blueprint itself is correct. -Logs are stored under ``/var/log/lorax-composer/`` and include all console -messages as well as extra debugging info and API requests. +Installation +------------ + +The best way to install ``lorax-composer`` is to use ``sudo dnf install +lorax-composer composer-cli``, this will setup the weldr user and install the +systemd socket activation service. You will then need to enable it with ``sudo +systemctl enable lorax-composer.socket && sudo systemctl start +lorax-composer.socket``. This will leave the server off until the first request +is made. Systemd will then launch the server and it will remain running until +the system is rebooted. Quickstart ---------- @@ -34,31 +42,162 @@ Quickstart 1. Create a ``weldr`` user and group by running ``useradd weldr`` 2. Remove any pre-existing socket directory with ``rm -rf /run/weldr/`` A new directory with correct permissions will be created the first time the server runs. -3. Either start it via systemd with ``systemctl start lorax-composer`` or - run it directly with ``lorax-composer /path/to/blueprints/`` +3. Enable the socket activation with ``systemctl enable lorax-composer.socket + && sudo systemctl start lorax-composer.socket`` or run it directly with + ``lorax-composer /path/to/blueprints/`` -The ``/path/to/blueprints/`` is where the blueprint's git repo will be created, and -all the blueprints created with the ``/api/v0/blueprints/new`` route will be stored. -If there are blueprint ``.toml`` files in the top level of the directory they will -be imported into the blueprint git storage. +The ``/path/to/blueprints/`` directory is where the blueprints' git repo will +be created, and all the blueprints created with the ``/api/v0/blueprints/new`` +route will be stored. If there are blueprint ``.toml`` files in the top level +of the directory they will be imported into the blueprint git storage when +``lorax-composer`` starts. + +Logs +---- + +Logs are stored under ``/var/log/lorax-composer/`` and include all console +messages as well as extra debugging info and API requests. + +Security +-------- + +Some security related issues that you should be aware of before running ``lorax-composer``: + +* One of the API server threads needs to retain root privileges in order to run Anaconda. +* SELinux must be set to Permissive or disabled to allow ``livemedia-creator`` to run Anaconda. +* Only allow authorized users access to the ``weldr`` group and socket. + +Since Anaconda kickstarts are used there is the possibility that a user could +inject commands into a blueprint that would result in the kickstart executing +arbitrary code on the host. Only authorized users should be allowed to build +images using ``lorax-composer``. + +How it Works +------------ + +The server runs as root, and as ``weldr``. Communication with it is via a unix +domain socket (``/run/weldr/api.socket`` by default). The directory and socket +are owned by ``root:weldr`` so that any user in the ``weldr`` group can use the API +to control ``lorax-composer``. + +At startup the server will check for the correct permissions and +ownership of a pre-existing directory, or it will create a new one if it +doesn't exist. The socket path and group owner's name can be changed from the +cmdline by passing it the ``--socket`` and ``--group`` arguments. + +It will then drop root privileges for the API thread and run as the ``weldr`` +user. The queue and compose thread still runs as root because it needs to be +able to mount/umount files and run Anaconda. Composing Images ---------------- -As of version 19.7.7 lorax-composer can create ``tar`` output images. You can use curl to start -a compose like this:: +The `welder-web `_ GUI project can be used to construct +blueprints and create composes using a web browser. - curl --unix-socket /run/weldr/api.socket -X POST -H "Content-Type: application/json" -d '{"blueprint_name": "http-server", "compose_type": "tar", "branch": "master"}' http:///api/v0/compose +Or use the command line with `composer-cli `_. -And then monitor it by passing the returned build UUID to ``/compose/status/``. +Blueprints +---------- + +Blueprints are simple text files in `TOML `_ format that describe +which packages, and what versions, to install into the image. They can also define a limited set +of customizations to make to the final image. + +Example blueprints can be found in the ``lorax-composer`` `test suite +`_, with a simple one +looking like this:: + + name = "base" + description = "A base system with bash" + version = "0.0.1" + + [[packages]] + name = "bash" + version = "4.4.*" + +The ``name`` field is the name of the blueprint. It can contain spaces, but they will be converted to ``-`` +when it is written to disk. It should be short and descriptive. + +``description`` can be a longer description of the blueprint, it is only used for display purposes. + +``version`` is a `semver compatible `_ version number. If +a new blueprint is uploaded with the same ``version`` the server will +automatically bump the PATCH level of the ``version``. If the ``version`` +doesn't match it will be used as is. eg. Uploading a blueprint with ``version`` +set to ``0.1.0`` when the existing blueprint ``version`` is ``0.0.1`` will +result in the new blueprint being stored as ``version 0.1.0``. + +[[packages]] and [[modules]] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +These entries describe the package names and matching version glob to be installed into the image. + +The names must match the names exactly, and the versions can be an exact match +or a filesystem-like glob of the version using ``*`` wildcards and ``?`` +character matching. + +NOTE: As of lorax-composer-29.2-1 the versions are not used for depsolving, +that is planned for a future release. And currently there are no differences +between ``packages`` and ``modules`` in ``lorax-composer``. + +Customizations +~~~~~~~~~~~~~~ + +The ``[[customizations]]`` section can be used to configure the hostname of the final image. eg.:: + + [[customizations]] + hostname = "baseimage" + + +[[customizations.sshkey]] +************************* + +Set an existing user's ssh key in the final image:: + + [[customizations.sshkey]] + user = "root" + key = "PUBLIC SSH KEY" + +The key will be added to the user's authorized_keys file. + + +[[customizations.user]] +*********************** + +Add a user to the image, and/or set their ssh key. +All fields for this section are optional except for the ``name``, here is a complete example:: + + [[customizations.user]] + name = "admin" + description = "Administrator account" + password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..." + key = "PUBLIC SSH KEY" + home = "/srv/widget/" + shell = "/usr/bin/bash" + groups = ["widget", "users", "wheel"] + uid = 1200 + gid = 1200 + +If the password starts with ``$6$``, ``$5$``, or ``$2b$`` it will be stored as +an encrypted password. Otherwise it will be treated as a plain text password. + + +[[customizations.group]] +************************ + +Add a group to the image. ``name`` is required and ``gid`` is optional:: + + [[customizations.group]] + name = "widget" + gid = 1130 -Version 19.7.10 adds support for ``live-iso`` and ``partitioned-disk`` Adding Output Types ------------------- -livemedia-creator supports a large number of output types, and only some of -these are currently available via lorax-composer. To add a new output type to +``livemedia-creator`` supports a large number of output types, and only some of +these are currently available via ``lorax-composer``. To add a new output type to lorax-composer a kickstart file needs to be added to ``./share/composer/``. The name of the kickstart is what will be used by the ``/compose/types`` route, and the ``compose_type`` field of the POST to start a compose. It also needs to have