543 lines
18 KiB
ReStructuredText
543 lines
18 KiB
ReStructuredText
composer-cli
|
|
============
|
|
|
|
:Authors:
|
|
Brian C. Lane <bcl@redhat.com>
|
|
|
|
``composer-cli`` is an interactive tool for use with a WELDR API server,
|
|
managing blueprints, exploring available packages, and building new images. As
|
|
of Fedora 34, `osbuild-composer <https://osbuild.org>` is the recommended
|
|
server.
|
|
|
|
It requires the server to be installed on the local system, and the user
|
|
running it needs to be a member of the ``weldr`` group.
|
|
|
|
composer-cli cmdline arguments
|
|
------------------------------
|
|
|
|
.. argparse::
|
|
:ref: composer.cli.cmdline.composer_cli_parser
|
|
:prog: composer-cli
|
|
|
|
Edit a Blueprint
|
|
----------------
|
|
|
|
Start out by listing the available blueprints using ``composer-cli blueprints
|
|
list``, pick one and save it to the local directory by running ``composer-cli
|
|
blueprints save http-server``.
|
|
|
|
Edit the file (it will be saved with a .toml extension) and change the
|
|
description, add a package or module to it. Send it back to the server by
|
|
running ``composer-cli blueprints push http-server.toml``. You can verify that it was
|
|
saved by viewing the changelog - ``composer-cli blueprints changes http-server``.
|
|
|
|
See the `Example Blueprint`_ for an example.
|
|
|
|
Build an image
|
|
----------------
|
|
|
|
Build a ``qcow2`` disk image from this blueprint by running ``composer-cli
|
|
compose start http-server qcow2``. It will print a UUID that you can use to
|
|
keep track of the build. You can also cancel the build if needed.
|
|
|
|
The available types of images is displayed by ``composer-cli compose types``.
|
|
Currently this consists of: alibaba, ami, ext4-filesystem, google, hyper-v,
|
|
live-iso, openstack, partitioned-disk, qcow2, tar, vhd, vmdk
|
|
|
|
You can optionally start an upload of the finished image, see `Image Uploads`_ for
|
|
more information.
|
|
|
|
|
|
Monitor the build status
|
|
------------------------
|
|
|
|
Monitor it using ``composer-cli compose status``, which will show the status of
|
|
all the builds on the system. You can view the end of the anaconda build logs
|
|
once it is in the ``RUNNING`` state using ``composer-cli compose log UUID``
|
|
where UUID is the UUID returned by the start command.
|
|
|
|
Once the build is in the ``FINISHED`` state you can download the image.
|
|
|
|
Download the image
|
|
------------------
|
|
|
|
Downloading the final image is done with ``composer-cli compose image UUID`` and it will
|
|
save the qcow2 image as ``UUID-disk.qcow2`` which you can then use to boot a VM like this::
|
|
|
|
qemu-kvm --name test-image -m 1024 -hda ./UUID-disk.qcow2
|
|
|
|
|
|
Image Uploads
|
|
-------------
|
|
|
|
``composer-cli`` can upload the images to a number of services, including AWS,
|
|
OpenStack, and vSphere. The upload can be started when the build is finished,
|
|
by using ``composer-cli compose start ...`` or an existing image can be uploaded
|
|
with ``composer-cli upload start ...``. In order to access the service you need
|
|
to pass authentication details to composer-cli using a TOML file, or reference
|
|
a previously saved profile.
|
|
|
|
.. note::
|
|
With ``osbuild-composer`` you can only specify upload targets during
|
|
the compose process.
|
|
|
|
|
|
Providers
|
|
---------
|
|
|
|
Providers are the services providers with Ansible playbook support under
|
|
``/usr/share/lorax/lifted/providers/``, you will need to gather some provider
|
|
specific information in order to authenticate with it. You can view the
|
|
required fields using ``composer-cli providers template <PROVIDER>``, eg. for AWS
|
|
you would run::
|
|
|
|
composer-cli upload template aws
|
|
|
|
The output looks like this::
|
|
|
|
provider = "aws"
|
|
|
|
[settings]
|
|
aws_access_key = "AWS Access Key"
|
|
aws_bucket = "AWS Bucket"
|
|
aws_region = "AWS Region"
|
|
aws_secret_key = "AWS Secret Key"
|
|
|
|
Save this into an ``aws-credentials.toml`` file and use it when running ``start``.
|
|
|
|
AWS
|
|
^^^
|
|
|
|
The access key and secret key can be created by going to the
|
|
``IAM->Users->Security Credentials`` section and creating a new access key. The
|
|
secret key will only be shown when it is first created so make sure to record
|
|
it in a secure place. The region should be the region that you want to use the
|
|
AMI in, and the bucket can be an existing bucket, or a new one, following the
|
|
normal AWS bucket naming rules. It will be created if it doesn't already exist.
|
|
|
|
When uploading the image it is first uploaded to the s3 bucket, and then
|
|
converted to an AMI. If the conversion is successful the s3 object will be
|
|
deleted. If it fails, re-trying after correcting the problem will re-use the
|
|
object if you have not deleted it in the meantime, speeding up the process.
|
|
|
|
|
|
Profiles
|
|
--------
|
|
|
|
Profiles store the authentication settings associated with a specific provider.
|
|
Providers can have multiple profiles, as long as their names are unique. For
|
|
example, you may have one profile for testing and another for production
|
|
uploads.
|
|
|
|
Profiles are created by pushing the provider settings template to the server using
|
|
``composer-cli providers push <PROFILE.TOML>`` where ``PROFILE.TOML`` is the same as the
|
|
provider template, but with the addition of a ``profile`` field. For example, an AWS
|
|
profile named ``test-uploads`` would look like this::
|
|
|
|
provider = "aws"
|
|
profile = "test-uploads"
|
|
|
|
[settings]
|
|
aws_access_key = "AWS Access Key"
|
|
aws_bucket = "AWS Bucket"
|
|
aws_region = "AWS Region"
|
|
aws_secret_key = "AWS Secret Key"
|
|
|
|
You can view the profile by using ``composer-cli providers aws test-uploads``.
|
|
|
|
|
|
Build an image and upload results
|
|
---------------------------------
|
|
|
|
If you have a profile named ``test-uploads``::
|
|
|
|
composer-cli compose start example-http-server ami "http image" aws test-uploads
|
|
|
|
Or if you have the settings stored in a TOML file::
|
|
|
|
composer-cli compose start example-http-server ami "http image" aws-settings.toml
|
|
|
|
It will return the UUID of the image build, and the UUID of the upload. Once
|
|
the build has finished successfully it will start the upload process, which you
|
|
can monitor with ``composer-cli upload info <UPLOAD-UUID>``
|
|
|
|
You can also view the upload logs from the Ansible playbook with::
|
|
|
|
``composer-cli upload log <UPLOAD-UUID>``
|
|
|
|
The type of the image must match the type supported by the provider.
|
|
|
|
|
|
Upload an existing image
|
|
------------------------
|
|
|
|
You can upload previously built images, as long as they are in the ``FINISHED`` state, using ``composer-cli upload start ...```. If you have a profile named ``test-uploads``::
|
|
|
|
composer-cli upload start <UUID> "http-image" aws test-uploads
|
|
|
|
Or if you have the settings stored in a TOML file::
|
|
|
|
composer-cli upload start <UUID> "http-image" aws-settings.toml
|
|
|
|
This will output the UUID of the upload, which can then be used to monitor the status in the same way
|
|
described above.
|
|
|
|
|
|
Debugging
|
|
---------
|
|
|
|
There are a couple of arguments that can be helpful when debugging problems.
|
|
These are only meant for debugging and should not be used to script access to
|
|
the API. If you need to do that you can communicate with it directly in the
|
|
language of your choice.
|
|
|
|
``--json`` will return the server's response as a nicely formatted json output
|
|
instead of printing what the command would usually print.
|
|
|
|
``--test=1`` will cause a compose start to start creating an image, and then
|
|
end with a failed state.
|
|
|
|
``--test=2`` will cause a compose to start and then end with a finished state,
|
|
without actually composing anything.
|
|
|
|
|
|
Blueprint Reference
|
|
-------------------
|
|
|
|
Blueprints are simple text files in `TOML <https://github.com/toml-lang/toml>`_ format that describe
|
|
which packages, and what versions, to install into the image. They can also define a limited set
|
|
of customizations to make to the final image.
|
|
|
|
A basic blueprint looks like this::
|
|
|
|
name = "base"
|
|
description = "A base system with bash"
|
|
version = "0.0.1"
|
|
|
|
[[packages]]
|
|
name = "bash"
|
|
version = "4.4.*"
|
|
|
|
The ``name`` field is the name of the blueprint. It can contain spaces, but they will be converted to ``-``
|
|
when it is written to disk. It should be short and descriptive.
|
|
|
|
``description`` can be a longer description of the blueprint, it is only used for display purposes.
|
|
|
|
``version`` is a `semver compatible <https://semver.org/>`_ version number. If
|
|
a new blueprint is uploaded with the same ``version`` the server will
|
|
automatically bump the PATCH level of the ``version``. If the ``version``
|
|
doesn't match it will be used as is. eg. Uploading a blueprint with ``version``
|
|
set to ``0.1.0`` when the existing blueprint ``version`` is ``0.0.1`` will
|
|
result in the new blueprint being stored as ``version 0.1.0``.
|
|
|
|
[[packages]] and [[modules]]
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
These entries describe the package names and matching version glob to be installed into the image.
|
|
|
|
The names must match the names exactly, and the versions can be an exact match
|
|
or a filesystem-like glob of the version using ``*`` wildcards and ``?``
|
|
character matching.
|
|
|
|
.. note::
|
|
Currently there are no differences between ``packages`` and ``modules``
|
|
in ``osbuild-composer``. Both are treated like an rpm package dependency.
|
|
|
|
For example, to install ``tmux-2.9a`` and ``openssh-server-8.*``, you would add
|
|
this to your blueprint::
|
|
|
|
[[packages]]
|
|
name = "tmux"
|
|
version = "2.9a"
|
|
|
|
[[packages]]
|
|
name = "openssh-server"
|
|
version = "8.*"
|
|
|
|
|
|
|
|
[[groups]]
|
|
^^^^^^^^^^
|
|
|
|
The ``groups`` entries describe a group of packages to be installed into the image. Package groups are
|
|
defined in the repository metadata. Each group has a descriptive name used primarily for display
|
|
in user interfaces and an ID more commonly used in kickstart files. Here, the ID is the expected
|
|
way of listing a group.
|
|
|
|
Groups have three different ways of categorizing their packages: mandatory, default, and optional.
|
|
For purposes of blueprints, mandatory and default packages will be installed. There is no mechanism
|
|
for selecting optional packages.
|
|
|
|
For example, if you want to install the ``anaconda-tools`` group you would add this to your
|
|
blueprint::
|
|
|
|
[[groups]]
|
|
name="anaconda-tools"
|
|
|
|
``groups`` is a TOML list, so each group needs to be listed separately, like ``packages`` but with
|
|
no version number.
|
|
|
|
|
|
Customizations
|
|
^^^^^^^^^^^^^^
|
|
|
|
The ``[customizations]`` section can be used to configure the hostname of the final image. eg.::
|
|
|
|
[customizations]
|
|
hostname = "baseimage"
|
|
|
|
This is optional and may be left out to use the defaults.
|
|
|
|
|
|
[customizations.kernel]
|
|
***********************
|
|
|
|
This allows you to append arguments to the bootloader's kernel commandline. This will not have any
|
|
effect on ``tar`` or ``ext4-filesystem`` images since they do not include a bootloader.
|
|
|
|
For example::
|
|
|
|
[customizations.kernel]
|
|
append = "nosmt=force"
|
|
|
|
|
|
[[customizations.sshkey]]
|
|
*************************
|
|
|
|
Set an existing user's ssh key in the final image::
|
|
|
|
[[customizations.sshkey]]
|
|
user = "root"
|
|
key = "PUBLIC SSH KEY"
|
|
|
|
The key will be added to the user's authorized_keys file.
|
|
|
|
.. warning::
|
|
|
|
``key`` expects the entire content of ``~/.ssh/id_rsa.pub``
|
|
|
|
|
|
[[customizations.user]]
|
|
***********************
|
|
|
|
Add a user to the image, and/or set their ssh key.
|
|
All fields for this section are optional except for the ``name``, here is a complete example::
|
|
|
|
[[customizations.user]]
|
|
name = "admin"
|
|
description = "Administrator account"
|
|
password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31L..."
|
|
key = "PUBLIC SSH KEY"
|
|
home = "/srv/widget/"
|
|
shell = "/usr/bin/bash"
|
|
groups = ["widget", "users", "wheel"]
|
|
uid = 1200
|
|
gid = 1200
|
|
|
|
If the password starts with ``$6$``, ``$5$``, or ``$2b$`` it will be stored as
|
|
an encrypted password. Otherwise it will be treated as a plain text password.
|
|
|
|
.. warning::
|
|
|
|
``key`` expects the entire content of ``~/.ssh/id_rsa.pub``
|
|
|
|
|
|
[[customizations.group]]
|
|
************************
|
|
|
|
Add a group to the image. ``name`` is required and ``gid`` is optional::
|
|
|
|
[[customizations.group]]
|
|
name = "widget"
|
|
gid = 1130
|
|
|
|
|
|
[customizations.timezone]
|
|
*************************
|
|
|
|
Customizing the timezone and the NTP servers to use for the system::
|
|
|
|
[customizations.timezone]
|
|
timezone = "US/Eastern"
|
|
ntpservers = ["0.north-america.pool.ntp.org", "1.north-america.pool.ntp.org"]
|
|
|
|
The values supported by ``timezone`` can be listed by running ``timedatectl list-timezones``.
|
|
|
|
If no timezone is setup the system will default to using `UTC`. The ntp servers are also
|
|
optional and will default to using the distribution defaults which are fine for most uses.
|
|
|
|
In some image types there are already NTP servers setup, eg. Google cloud image, and they
|
|
cannot be overridden because they are required to boot in the selected environment. But the
|
|
timezone will be updated to the one selected in the blueprint.
|
|
|
|
|
|
[customizations.locale]
|
|
***********************
|
|
|
|
Customize the locale settings for the system::
|
|
|
|
[customizations.locale]
|
|
languages = ["en_US.UTF-8"]
|
|
keyboard = "us"
|
|
|
|
The values supported by ``languages`` can be listed by running ``localectl list-locales`` from
|
|
the command line.
|
|
|
|
The values supported by ``keyboard`` can be listed by running ``localectl list-keymaps`` from
|
|
the command line.
|
|
|
|
Multiple languages can be added. The first one becomes the
|
|
primary, and the others are added as secondary. One or the other of ``languages``
|
|
or ``keyboard`` must be included (or both) in the section.
|
|
|
|
|
|
[customizations.firewall]
|
|
*************************
|
|
|
|
By default the firewall blocks all access except for services that enable their ports explicitly,
|
|
like ``sshd``. This command can be used to open other ports or services. Ports are configured using
|
|
the port:protocol format::
|
|
|
|
[customizations.firewall]
|
|
ports = ["22:tcp", "80:tcp", "imap:tcp", "53:tcp", "53:udp"]
|
|
|
|
Numeric ports, or their names from ``/etc/services`` can be used in the ``ports`` enabled/disabled lists.
|
|
|
|
The blueprint settings extend any existing settings in the image templates, so if ``sshd`` is
|
|
already enabled it will extend the list of ports with the ones listed by the blueprint.
|
|
|
|
If the distribution uses ``firewalld`` you can specify services listed by ``firewall-cmd --get-services``
|
|
in a ``customizations.firewall.services`` section::
|
|
|
|
[customizations.firewall.services]
|
|
enabled = ["ftp", "ntp", "dhcp"]
|
|
disabled = ["telnet"]
|
|
|
|
Remember that the ``firewall.services`` are different from the names in ``/etc/services``.
|
|
|
|
Both are optional, if they are not used leave them out or set them to an empty list ``[]``. If you
|
|
only want the default firewall setup this section can be omitted from the blueprint.
|
|
|
|
NOTE: The ``Google`` and ``OpenStack`` templates explicitly disable the firewall for their environment.
|
|
This cannot be overridden by the blueprint.
|
|
|
|
[customizations.services]
|
|
*************************
|
|
|
|
This section can be used to control which services are enabled at boot time.
|
|
Some image types already have services enabled or disabled in order for the
|
|
image to work correctly, and cannot be overridden. eg. ``ami`` requires
|
|
``sshd``, ``chronyd``, and ``cloud-init``. Without them the image will not
|
|
boot. Blueprint services are added to, not replacing, the list already in the
|
|
templates, if any.
|
|
|
|
The service names are systemd service units. You may specify any systemd unit
|
|
file accepted by ``systemctl enable`` eg. ``cockpit.socket``::
|
|
|
|
[customizations.services]
|
|
enabled = ["sshd", "cockpit.socket", "httpd"]
|
|
disabled = ["postfix", "telnetd"]
|
|
|
|
|
|
[[repos.git]]
|
|
~~~~~~~~~~~~~
|
|
|
|
.. note::
|
|
Currently ``osbuild-composer`` does not support ``repos.git``
|
|
|
|
The ``[[repos.git]]`` entries are used to add files from a `git repository <https://git-scm.com/>`_
|
|
repository to the created image. The repository is cloned, the specified ``ref`` is checked out
|
|
and an rpm is created to install the files to a ``destination`` path. The rpm includes a summary
|
|
with the details of the repository and reference used to create it. The rpm is also included in the
|
|
image build metadata.
|
|
|
|
To create an rpm named ``server-config-1.0-1.noarch.rpm`` you would add this to your blueprint::
|
|
|
|
[[repos.git]]
|
|
rpmname="server-config"
|
|
rpmversion="1.0"
|
|
rpmrelease="1"
|
|
summary="Setup files for server deployment"
|
|
repo="PATH OF GIT REPO TO CLONE"
|
|
ref="v1.0"
|
|
destination="/opt/server/"
|
|
|
|
* rpmname: Name of the rpm to create, also used as the prefix name in the tar archive
|
|
* rpmversion: Version of the rpm, eg. "1.0.0"
|
|
* rpmrelease: Release of the rpm, eg. "1"
|
|
* summary: Summary string for the rpm
|
|
* repo: URL of the get repo to clone and create the archive from
|
|
* ref: Git reference to check out. eg. origin/branch-name, git tag, or git commit hash
|
|
* destination: Path to install the / of the git repo at when installing the rpm
|
|
|
|
An rpm will be created with the contents of the git repository referenced, with the files
|
|
being installed under ``/opt/server/`` in this case.
|
|
|
|
``ref`` can be any valid git reference for use with ``git archive``. eg. to use the head
|
|
of a branch set it to ``origin/branch-name``, a tag name, or a commit hash.
|
|
|
|
Note that the repository is cloned in full each time a build is started, so pointing to a
|
|
repository with a large amount of history may take a while to clone and use a significant
|
|
amount of disk space. The clone is temporary and is removed once the rpm is created.
|
|
|
|
Example Blueprint
|
|
-----------------
|
|
|
|
This example blueprint will install the ``tmux``, ``git``, and ``vim-enhanced``
|
|
packages. It will set the ``root`` ssh key, add the ``widget`` and ``admin``
|
|
users as well as a ``students`` group::
|
|
|
|
name = "example-custom-base"
|
|
description = "A base system with customizations"
|
|
version = "0.0.1"
|
|
|
|
[[packages]]
|
|
name = "tmux"
|
|
version = "*"
|
|
|
|
[[packages]]
|
|
name = "git"
|
|
version = "*"
|
|
|
|
[[packages]]
|
|
name = "vim-enhanced"
|
|
version = "*"
|
|
|
|
[customizations]
|
|
hostname = "custombase"
|
|
|
|
[[customizations.sshkey]]
|
|
user = "root"
|
|
key = "A SSH KEY FOR ROOT"
|
|
|
|
[[customizations.user]]
|
|
name = "widget"
|
|
description = "Widget process user account"
|
|
home = "/srv/widget/"
|
|
shell = "/usr/bin/false"
|
|
groups = ["dialout", "users"]
|
|
|
|
[[customizations.user]]
|
|
name = "admin"
|
|
description = "Widget admin account"
|
|
password = "$6$CHO2$3rN8eviE2t50lmVyBYihTgVRHcaecmeCk31LeOUleVK/R/aeWVHVZDi26zAH.o0ywBKH9Tc0/wm7sW/q39uyd1"
|
|
home = "/srv/widget/"
|
|
shell = "/usr/bin/bash"
|
|
groups = ["widget", "users", "students"]
|
|
uid = 1200
|
|
|
|
[[customizations.user]]
|
|
name = "plain"
|
|
password = "simple plain password"
|
|
|
|
[[customizations.user]]
|
|
name = "bart"
|
|
key = "SSH KEY FOR BART"
|
|
groups = ["students"]
|
|
|
|
[[customizations.group]]
|
|
name = "widget"
|
|
|
|
[[customizations.group]]
|
|
name = "students"
|