Commit Graph

151 Commits

Author SHA1 Message Date
Brian C. Lane
82c8c3a491 gevent has deprecated .wsgi, should use .pywsgi instead
https://github.com/gevent/gevent/blob/master/doc/api/gevent.wsgi.rst
(cherry picked from commit c9ca451568)
2018-06-11 16:54:59 -07:00
Brian C. Lane
ce715ad15f Check to make sure blueprints directory exists 2018-06-11 16:54:59 -07:00
Brian C. Lane
f26a6212d6 Add --proxy to lorax-composer cmdline
Overrides the [yum] proxy setting in the config file.
2018-06-11 16:54:59 -07:00
Brian C. Lane
9978503a69 Pass the --tmp value into run_creator and cleanup after a crash
Crashing can sometimes leave directories in /var/tmp/lmc-* so clean
those up after run_creator is finished.
2018-06-11 16:54:59 -07:00
Brian C. Lane
3a38a57ae1 Add --tmp to lorax-composer and set default tempdir
It was using /tmp/ which can fill up quickly when building images.
Default to /var/tmp unless the user passes --tmp /tmp/
2018-06-11 16:54:59 -07:00
Brian C. Lane
ffc3195d77 Add support for systemd socket activation
Instead of enabling lorax-composer.service enable lorax-composer.socket
and it will start lorax-composer on first access to
/run/weldr/api.socket
2018-06-11 16:54:59 -07:00
Brian C. Lane
651e111d1f Change the API code to use blueprint
This includes in the JSON output. The only exception is when using
functions from the recipes module. There are no plans to change that.
2018-06-11 16:54:59 -07:00
Brian C. Lane
de604b37a6 Add a pid file for lorax-composer
This will prevent accidentally running more than 1 instance.
Uses /run/lorax-composer.pid and checks to make sure that the PID
written to it isn't stale.
2018-06-11 16:54:58 -07:00
Brian C. Lane
59fd3934f3 Move core of livemedia-creator to run_creator()
We need to be able to share the output types from livemedia-creator with
lorax-composer, so move the core of the main() function into
run_creatoe(). Pass in the cmdline args or a DataHolder with them set.
2018-06-11 16:54:58 -07:00
Brian C. Lane
85ee784efc Only chown recipe directory if it already exists
Since creation of the directory has been moved after switching users
this now needs to check to make sure it exists first.
2018-06-11 16:54:58 -07:00
Brian C. Lane
a1427b1a31 Set the HOME variable to a directory the uid can access
libgit2 will look for $HOME/.gitconfig after dropping root privileges so
this needs to be accessible to the user.
2018-06-11 16:54:58 -07:00
Brian C. Lane
442dc43206 Open the git repo after dropping root privileges
Also make sure the recipe directory and its contents have correct
ownership, and change the default recipe path when using the systemd
service to /var/lib/lorax/composer/recipes/
2018-06-11 16:54:58 -07:00
Brian C. Lane
4ae2e1450d Exit on uid/gid errors before checking directory permissions
If the uid/gid check fails there is no sense in trying to examine the
ownership or permissions.
2018-06-11 16:54:58 -07:00
Brian C. Lane
eb2a7208c8 Move queue monitor startup into a function
This allows it to be used by the test framework.
2018-06-11 16:54:58 -07:00
Brian C. Lane
63a7897457 Move queue directory creation into a function
So that it can also be used during tests
2018-06-11 16:54:58 -07:00
Brian C. Lane
9603c866af Drop cancel_q from the monitor() function
Not needed, builds are canceled by writing a 'CANCEL' file in the results
directory.
2018-06-11 16:54:58 -07:00
Brian C. Lane
22c03ceba7 Fix yum config directory creation for projects and server tests 2018-06-11 16:54:58 -07:00
Brian C. Lane
3fdf3a9819 Turn on o+x permission for the queue and results directories
Otherwise it won't allow a group member to delete things.
2018-06-11 16:54:58 -07:00
Brian C. Lane
c92373ecd6 Change config and paths
With the API running as weldr the permissions on the yum directories
needs to be accessable to it, as well as the results and queue/new
directories.
2018-06-11 16:54:58 -07:00
Brian C. Lane
e2f4674fb3 Add basic composer queue handling
The queue is in /var/lib/weldr/queue/new by default. It watches the
directory for new symlinks (to /var/lib/weldr/results/<dirname>) and
handles running anaconda on the kickstart found in final-kickstart.ks
inside the symlinked directory.
2018-06-11 16:54:58 -07:00
Brian C. Lane
0ce4197a1e Change compress to use communicate instead of wait
Also move default_image_name into imgutils so it can be used in other
places.

When running from lorax-composer the wait() call wasn't waiting until
the tar was finished. I think this is due to gevent monkey-patching
something. Using communicate() solves this problem.
2018-06-11 16:54:58 -07:00
Brian C. Lane
377f5e4124 Switch the API to use a Unix Domain Socket
This drops support for the TCP port and switches to using a socket at
/var/run/weldr/api.socket

Also add the start of some docs for lorax-composer.

--host and --port argument have been removed.

--group sets the group name to use for access to the socket and its
parent directory. Defaults to 'weldr'

--socket sets the full path to the socket to create. Defaults to
'/var/run/weldr/api.socket'
2018-06-11 16:54:58 -07:00
Alexander Todorov
0af072c77e Silence pocketlint bad-preconf-access warnings 2018-06-11 16:54:58 -07:00
Brian C. Lane
3e5c0707bb Redirect yum's logging to yum.log 2018-06-11 16:54:58 -07:00
Brian C. Lane
efd15aaad7 Add --releasever option to lorax-composer
If it isn't set the releasever value will use yum and the system / to
find the releasever.
2018-06-11 16:54:58 -07:00
Brian C. Lane
164232ba4d Modify pylorax.api.config.configure so it can also be used for tests. 2018-06-11 16:54:58 -07:00
Brian C. Lane
ad5ab71551 Move ComposerConfig into pylorax.api.config module 2018-06-11 16:54:58 -07:00
Brian C. Lane
569ce1d3dc Add support for yum to lorax-composer
This includes a new configuration file at /etc/lorax/composer.conf with
built-in defaults. It also adds a YUMLOCK server config object so that
request handlers can access the yum base object without interfering
with each other.
2018-06-11 16:54:58 -07:00
Brian C. Lane
950d9df217 Fix server request logging.
And switch the other logs to append instead of overwrite on each run.
2018-06-11 16:54:57 -07:00
Brian C. Lane
453082ab9f lorax-composer: Drop unneeded parameters and create missing directories
The log directory (and parents) will be created if they are missing.
An empty recipe directory will be created if it doesn't exist.
2018-06-11 16:54:57 -07:00
Brian C. Lane
b6fb22133c Add /recipes/list route and tests
Includes adding a lock for access to the git repo from the API.
2018-06-11 16:54:57 -07:00
Alexander Todorov
bf9ed50d51 Fix dangerous-default value warnings
when default value is list or dict the default arguments are
instantiated as objects at the time of definition. This is significant
(exposing visible semantics) when the object is mutable. There’s no way
of re-binding that default argument name in the function’s closure. When
function is executed multiple times with its default value the value
will change between executions, possibly leading to strange side effects.

For more information see:
http://satran.in/2012/01/12/python-dangerous-default-value-as-argument.html
2018-06-11 16:54:57 -07:00
Alexander Todorov
a0766b1d70 Don't redefine variables from outer scope 2018-06-11 16:54:57 -07:00
Alexander Todorov
663b0f8378 Fix logging formatting 2018-06-11 16:54:57 -07:00
Alexander Todorov
15de3e1687 pylint fix: remove unused imports 2018-06-11 16:54:57 -07:00
Brian C. Lane
2008a413b6 lorax-composer initial commit
The lorax-composer program will launch a BDCS compatible API server
using Flask and Gevent. Currently this is a skeleton application with
only one active route (/api/v0/status).

The API code lives in ./src/pylorax/api/v0.py with related code in other
pylorax/api/* modules.
2018-06-11 16:54:57 -07:00
Brian C. Lane
bf8be43c90 livemedia-creator: Move core functions into pylorax modules
This reduces the amount of code in livemedia-creator to the cmdline
parsing and calling of the installer functions. Moving them into other
modules will allow them to be used by other projects, like the
lorax-composer API server.
2018-06-11 16:54:57 -07:00
Brian C. Lane
4788d4eb4d Increase default ram used with lmc and virt to 2048
1024 is a bit small in some situations, double it.

Resolves: rhbz#1538747
2018-06-11 13:49:28 -07:00
Brian C. Lane
c8cee96b1c Add --virt-uefi to boot the VM using OVMF
This requires OVMF to be setup on the system, and for the kickstart to
create a /boot/efi/ partition. You can then use it to create UEFI
bootable partitioned disk images.

The UEFI firmware needs to be installed manually on the system, either
in the default location of /usr/share/OVMF/ or use --ovmf-path to point
to the location.

Resolves: rhbz#1546715
Resolves: rhbz#1544805
2018-06-11 13:45:32 -07:00
Brian C. Lane
22392b64fc Add --dracut-arg support to lorax
Use it to override the default dracut arguments (displayed as part of
the --help output). If you want to extend the default arguments they
all need to be passed in on the cmdline as well. eg.

--dracut-arg='--xz' --dracut-arg='--install /.buildstamp' ...

Resolves: rhbz#1452220
2018-06-06 17:38:55 -07:00
Brian C. Lane
6f2494f9d5 livemedia-creator: Search for kernel/initrd under /images/pxeboot (#1522629)
On aarch64 the kernel and initrd are under /images/pxeboot, add these
paths to the search list.

Resolves: rhbz#1522629
2018-05-17 10:30:34 -07:00
Brian C. Lane
1639df6862 Set the releasever and install gpg keys when using --repo
This uses the --release value as the yum releasever so that $releasever
in a --repo will work.

It also turns on assumeyes so that any gpgkey entries in the .repo file
will be installed and used automatically if gpgcheck is enabled for the
repo.

Related: rhbz#1430479
2017-09-26 15:48:37 -07:00
Brian C. Lane
3cfd532382 IsoMountpoint: Add ppc64le kernel to search (#1373358)
Resolves: rhbz#1373358
2017-08-11 17:38:58 -07:00
Brian C. Lane
d30143d146 livemedia-creator: Report correct results dir (#1374609)
Resolves: rhbz#1374609
2017-08-11 17:38:58 -07:00
Brian C. Lane
1b29041089 Add the version to the log (#1335456)
Log the version number as INFO and add -V version display to
livemedia-creator.

Resolves: rhbz#1335456
2017-08-11 15:42:16 -07:00
Brian C. Lane
487618314c Add support for --repo to read yum .repo files directly (#1430479)
This makes it easier to specify existing repos with extra args, eg.
/etc/yum.repos.d/redhat.repo generated by subscription-manager.

Resolves: rhbz#1430479
2017-08-11 15:11:30 -07:00
Brian C. Lane
d4437ce5ee Add --noverifyssl to lorax (#1430483)
Previously lorax had no way to use repos with self-signed certificates.
This adds the --noverifyssl cmdline option which will ignore certificate
errors.

Resolves: rhbz#1430483
2017-03-17 09:07:58 -07:00
Brian C. Lane
c2be4bdfc8 livemedia-creator: Use imgutils.copytree for results (#1362157)
commit 5da9f5c179 cleaned up the resultdir
handling, but didn't take into account that on master shutil.copytree
was switched to imgutil.copytree which doesn't care if the empty
directory exists or not.

Resolves: rhbz#1362157
2016-08-01 14:22:29 -07:00
Brian C. Lane
492d39933d livemedia-creator: Fix logging (#1361031)
The cherry-pick for commit 0d28b9e09c
dropped the patch into the middle of setup_logging and added a (unused)
main() function so things would still run, but logging wasn't being
setup properly.

This removes the main() and moves the default_image_name() function
after all of the logging setup code.

Resolves: rhbz#1361031
2016-07-28 14:11:37 -07:00
Brian C. Lane
3093ff7bac livemedia-creator: Use hd:LABEL for stage2 iso (#1355882)
The correct way with the current dracut (and it should be backwards
compatible) is to reference it as stage2=hd:LABEL=...

(cherry picked from commit 4bc4b4c5a5)
Resolves: rhbz#1355882
2016-07-25 08:09:12 -07:00