pylorax package

Submodules

pylorax.base module

class pylorax.base.BaseLoraxClass

Bases: object

pcritical(msg, fobj=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

pdebug(msg, fobj=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

perror(msg, fobj=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

pinfo(msg, fobj=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

pwarning(msg, fobj=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

class pylorax.base.DataHolder(**kwargs)

Bases: dict

copy() → a shallow copy of D

pylorax.buildstamp module

class pylorax.buildstamp.BuildStamp(product, version, bugurl, isfinal, buildarch, variant='')

Bases: object

write(outfile)

pylorax.cmdline module

pylorax.cmdline.lmc_parser(dracut_default='')

Return a ArgumentParser object for live-media-creator.

pylorax.cmdline.lorax_parser(dracut_default='')

Return the ArgumentParser for lorax

pylorax.creator module

class pylorax.creator.FakeDNF(conf)

Bases: object

A minimal DNF object suitable for passing to RuntimeBuilder

lmc uses RuntimeBuilder to run the arch specific iso creation templates, so the the installroot config value is the important part of this. Everything else should be a nop.

reset()

pylorax.creator.calculate_disk_size(opts, ks)

Calculate the disk size from the kickstart

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• ks (str) -- Path to the kickstart to use for the installation

Returns

Disk size in MiB

Return type

int

Also takes into account the use of reqpart or reqpart --add-boot

pylorax.creator.check_kickstart(ks, opts)

Check the parsed kickstart object for errors

Parameters

• ks (pykickstart.parser.KickstartParser) -- Parsed Kickstart object

• opts (Either a DataHolder or ArgumentParser) -- Commandline options to control the process

Returns

List of error strings or empty list

Return type

list

pylorax.creator.create_pxe_config(template, images_dir, live_image_name, add_args=None)

Create template for pxe to live configuration

Parameters

• images_dir (str) -- Path of directory with images to be used

• live_image_name (str) -- Name of live rootfs image file

• add_args (list) -- Arguments to be added to initrd= pxe config

pylorax.creator.dracut_args(opts)

Return a list of the args to pass to dracut

Return the default argument list unless one of the dracut cmdline arguments has been used.

pylorax.creator.find_ostree_root(phys_root)

Find root of ostree deployment

Parameters

phys_root (str) -- Path to physical root

Returns

Relative path of ostree deployment root

Return type

str

Raises

Exception -- More than one deployment roots were found

pylorax.creator.get_arch(mount_dir)

Get the kernel arch

Returns

Arch of first kernel found at mount_dir/boot/ or i386

Return type

str

pylorax.creator.is_image_mounted(disk_img)

Check to see if the disk_img is mounted

Returns

True if disk_img is in /proc/mounts

Return type

bool

pylorax.creator.make_appliance(disk_img, name, template, outfile, networks=None, ram=1024, vcpus=1, arch=None, title='Linux', project='Linux', releasever='34')

Generate an appliance description file

Parameters

• disk_img (str) -- Full path of the disk image

• name (str) -- Name of the appliance, passed to the template

• template (str) -- Full path of Mako template

• outfile (str) -- Full path of file to write, using template

• networks (list) -- List of networks(str) from the kickstart

• ram (int) -- Ram, in MiB, passed to template. Default is 1024

• vcpus (int) -- CPUs, passed to template. Default is 1

• arch (str) -- CPU architecture. Default is 'x86_64'

• title (str) -- Title, passed to template. Default is 'Linux'

• project (str) -- Project, passed to template. Default is 'Linux'

• releasever (str) -- Release version, passed to template. Default is 34

pylorax.creator.make_image(opts, ks, cancel_func=None)

Install to a disk image

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• ks (str) -- Path to the kickstart to use for the installation

• cancel_func (function) -- Function that returns True to cancel build

Returns

Path of the image created

Return type

str

Use qemu+boot.iso or anaconda to install to a disk image.

pylorax.creator.make_live_images(opts, work_dir, disk_img)

Create live images from direcory or rootfs image

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• work_dir (str) -- Directory for storing results

• disk_img (str) -- Path to disk image (fsimage or partitioned)

Returns

Path of directory with created images or None

Return type

str

fsck.ext4 is run on the rootfs_image to make sure there are no errors and to zero out any deleted blocks to make it compress better. If this fails for any reason it will return None and log the error.

pylorax.creator.make_livecd(opts, mount_dir, work_dir)

Take the content from the disk image and make a livecd out of it

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• mount_dir (str) -- Directory tree to compress

• work_dir (str) -- Output compressed image to work_dir+images/install.img

This uses wwood's squashfs live initramfs method:

• put the real / into LiveOS/rootfs.img

• make a squashfs of the LiveOS/rootfs.img tree

• This is loaded by dracut when the cmdline is passed to the kernel: root=live:CDLABEL=<volid> rd.live.image

pylorax.creator.make_runtime(opts, mount_dir, work_dir, size=None)

Make the squashfs image from a directory

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• mount_dir (str) -- Directory tree to compress

• work_dir (str) -- Output compressed image to work_dir+images/install.img

• size (int) -- Size of disk image, in GiB

Returns

rc of squashfs creation

Return type

int

pylorax.creator.mount_boot_part_over_root(img_mount)

Mount boot partition to /boot of root fs mounted in img_mount

Used for OSTree so it finds deployment configurations on live rootfs

param img_mount: object with mounted disk image root partition type img_mount: imgutils.PartitionMount

pylorax.creator.rebuild_initrds_for_live(opts, sys_root_dir, results_dir)

Rebuild intrds for pxe live image (root=live:http://)

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• sys_root_dir (str) -- Path to root of the system

• results_dir (str) -- Path of directory for storing results

pylorax.creator.run_creator(opts, cancel_func=None)

Run the image creator process

Parameters

• opts (Either a DataHolder or ArgumentParser) -- Commandline options to control the process

• cancel_func (function) -- Function that returns True to cancel build

Returns

The result directory and the disk image path.

Return type

Tuple of str

This function takes the opts arguments and creates the selected output image. See the cmdline --help for livemedia-creator for the possible options

(Yes, this is not ideal, but we can fix that later)

pylorax.creator.squashfs_args(opts)

Returns the compression type and args to use when making squashfs

Parameters

opts -- ArgumentParser object with compression and compressopts

Returns

tuple of compression type and args

Return type

tuple

pylorax.decorators module

pylorax.decorators.singleton(cls)

pylorax.discinfo module

class pylorax.discinfo.DiscInfo(release, basearch)

Bases: object

write(outfile)

pylorax.dnfbase module

pylorax.dnfbase.get_dnf_base_object(installroot, sources, mirrorlists=None, repos=None, enablerepos=None, disablerepos=None, tempdir='/var/tmp', proxy=None, releasever='34', cachedir=None, logdir=None, sslverify=True, dnfplugins=None)

Create a dnf Base object and setup the repositories and installroot

Parameters

• installroot (string) -- Full path to the installroot

• sources (list) -- List of source repo urls to use for the installation

• enablerepos (list) -- List of repo names to enable

• disablerepos (list) -- List of repo names to disable

• mirrorlist (list) -- List of mirrors to use

• tempdir (string) -- Path of temporary directory

• proxy (string) -- http proxy to use when fetching packages

• releasever (string) -- Release version to pass to dnf

• cachedir (string) -- Directory to use for caching packages

• noverifyssl (bool) -- Set to True to ignore the CA of ssl certs. eg. use self-signed ssl for https repos.

If tempdir is not set /var/tmp is used. If cachedir is None a dnf.cache directory is created inside tmpdir

pylorax.dnfhelper module

class pylorax.dnfhelper.LoraxDownloadCallback

Bases: DownloadProgress

end(payload, status, msg)

progress(payload, done)

start(total_files, total_size, total_drpms=0)

class pylorax.dnfhelper.LoraxRpmCallback

Bases: TransactionProgress

error(message)

progress(package, action, ti_done, ti_total, ts_done, ts_total)

pylorax.executils module

class pylorax.executils.ExecProduct(rc, stdout, stderr)

Bases: object

pylorax.executils.augmentEnv()

pylorax.executils.execReadlines(command, argv, stdin=None, root='/', env_prune=None, filter_stderr=False, callback=<function <lambda>>, env_add=None, reset_handlers=True, reset_lang=True)

Execute an external command and return the line output of the command in real-time.

This method assumes that there is a reasonably low delay between the end of output and the process exiting. If the child process closes stdout and then keeps on truckin' there will be problems.

NOTE/WARNING: UnicodeDecodeError will be raised if the output of the

external command can't be decoded as UTF-8.

Parameters

• command -- The command to run

• argv -- The argument list

• stdin -- The file object to read stdin from.

• stdout -- Optional file object to redirect stdout and stderr to.

• root -- The directory to chroot to before running command.

• env_prune -- environment variable to remove before execution

• filter_stderr -- Whether stderr should be excluded from the returned output

• callback -- method to call while waiting for process to finish, passed Popen object

• env_add -- environment variables to add before execution

• reset_handlers -- whether to reset to SIG_DFL any signal handlers set to SIG_IGN

• reset_lang -- whether to set the locale of the child process to C

Returns

Iterator of the lines from the command

Output from the file is not logged to program.log This returns an iterator with the lines from the command until it has finished

pylorax.executils.execWithCapture(command, argv, stdin=None, root='/', log_output=True, filter_stderr=False, raise_err=False, callback=None, env_add=None, reset_handlers=True, reset_lang=True)

Run an external program and capture standard out and err.

Parameters

• command -- The command to run

• argv -- The argument list

• stdin -- The file object to read stdin from.

• root -- The directory to chroot to before running command.

• log_output -- Whether to log the output of command

• filter_stderr -- Whether stderr should be excluded from the returned output

• callback -- method to call while waiting for process to finish, passed Popen object

• env_add -- environment variables to add before execution

• reset_handlers -- whether to reset to SIG_DFL any signal handlers set to SIG_IGN

• reset_lang -- whether to set the locale of the child process to C

Returns

The output of the command

pylorax.executils.execWithRedirect(command, argv, stdin=None, stdout=None, root='/', env_prune=None, log_output=True, binary_output=False, raise_err=False, callback=None, env_add=None, reset_handlers=True, reset_lang=True)

Run an external program and redirect the output to a file.

Parameters

• command -- The command to run

• argv -- The argument list

• stdin -- The file object to read stdin from.

• stdout -- Optional file object to redirect stdout and stderr to.

• root -- The directory to chroot to before running command.

• env_prune -- environment variable to remove before execution

• log_output -- whether to log the output of command

• binary_output -- whether to treat the output of command as binary data

• raise_err -- whether to raise a CalledProcessError if the returncode is non-zero

• callback -- method to call while waiting for process to finish, passed Popen object

• env_add -- environment variables to add before execution

• reset_handlers -- whether to reset to SIG_DFL any signal handlers set to SIG_IGN

• reset_lang -- whether to set the locale of the child process to C

Returns

The return code of the command

pylorax.executils.runcmd(cmd, **kwargs)

run execWithRedirect with raise_err=True

pylorax.executils.runcmd_output(cmd, **kwargs)

run execWithCapture with raise_err=True

pylorax.executils.setenv(name, value)

Set an environment variable to be used by child processes.

This method does not modify os.environ for the running process, which is not thread-safe. If setenv has already been called for a particular variable name, the old value is overwritten.

Parameters

• name (str) -- The name of the environment variable

• value (str) -- The value of the environment variable

pylorax.executils.startProgram(argv, root='/', stdin=None, stdout=- 1, stderr=- 2, env_prune=None, env_add=None, reset_handlers=True, reset_lang=True, **kwargs)

Start an external program and return the Popen object.

The root and reset_handlers arguments are handled by passing a preexec_fn argument to subprocess.Popen, but an additional preexec_fn can still be specified and will be run. The user preexec_fn will be run last.

Parameters

• argv -- The command to run and argument

• root -- The directory to chroot to before running command.

• stdin -- The file object to read stdin from.

• stdout -- The file object to write stdout to.

• stderr -- The file object to write stderr to.

• env_prune -- environment variables to remove before execution

• env_add -- environment variables to add before execution

• reset_handlers -- whether to reset to SIG_DFL any signal handlers set to SIG_IGN

• reset_lang -- whether to set the locale of the child process to C

• kwargs -- Additional parameters to pass to subprocess.Popen

• preexec_fn -- A function to run before execution starts.

Returns

A Popen object for the running command.

pylorax.imgutils module

class pylorax.imgutils.DMDev(dev, size, name=None)

Bases: object

class pylorax.imgutils.LoopDev(filename, size=None)

Bases: object

class pylorax.imgutils.Mount(dev, opts='', mnt=None)

Bases: object

class pylorax.imgutils.PartitionMount(disk_img, mount_ok=None, submount=None)

Bases: object

Mount a partitioned image file using kpartx

pylorax.imgutils.compress(command, root, outfile, compression='xz', compressargs=None)

Make a compressed archive of the given rootdir or file. command is a list of the archiver commands to run compression should be "xz", "gzip", "lzma", "bzip2", or None. compressargs will be used on the compression commandline.

pylorax.imgutils.copytree(src, dest, preserve=True)

Copy a tree of files using cp -a, thus preserving modes, timestamps, links, acls, sparse files, xattrs, selinux contexts, etc. If preserve is False, uses cp -R (useful for modeless filesystems) raises CalledProcessError if copy fails.

pylorax.imgutils.default_image_name(compression, basename)

Return a default image name with the correct suffix for the compression type.

Parameters

• compression (str) -- Compression type

• basename (str) -- Base filename

Returns

basename with compression suffix

If the compression is unknown it defaults to xz

pylorax.imgutils.dm_attach(dev, size, name=None)

Attach a devicemapper device to the given device, with the given size. If name is None, a random name will be chosen. Returns the device name. raises CalledProcessError if dmsetup fails.

pylorax.imgutils.dm_detach(dev)

Detach the named devicemapper device. Returns False if dmsetup fails.

pylorax.imgutils.do_grafts(grafts, dest, preserve=True)

Copy each of the items listed in grafts into dest. If the key ends with '/' it's assumed to be a directory which should be created, otherwise just the leading directories will be created.

pylorax.imgutils.estimate_size(rootdir, graft=None, fstype=None, blocksize=4096, overhead=256)

pylorax.imgutils.get_loop_name(path)

Return the loop device associated with the path. Raises RuntimeError if more than one loop is associated

pylorax.imgutils.kpartx_disk_img(disk_img)

Attach a disk image's partitions to /dev/loopX using kpartx

Parameters

disk_img (str) -- The full path to a partitioned disk image

Returns

list of (loopXpN, size)

Return type

list of tuples

pylorax.imgutils.loop_attach(outfile)

Attach a loop device to the given file. Return the loop device name.

On rare occasions it appears that the device never shows up, some experiments seem to indicate that it may be a race with another process using /dev/loop* devices.

So we now try 3 times before actually failing.

Raises CalledProcessError if losetup fails.

pylorax.imgutils.loop_detach(loopdev)

Detach the given loop device. Return False on failure.

pylorax.imgutils.loop_waitfor(loop_dev, outfile)

Make sure the loop device is attached to the outfile.

It seems that on rare occasions losetup can return before the /dev/loopX is ready for use, causing problems with mkfs. This tries to make sure that the loop device really is associated with the backing file before continuing.

Raise RuntimeError if it isn't setup after 5 tries.

pylorax.imgutils.mkbtrfsimg(rootdir, outfile, size=None, label='', mountargs='', graft=None)

pylorax.imgutils.mkcpio(root, outfile, compression='xz', compressargs=None)

pylorax.imgutils.mkdosimg(rootdir, outfile, size=None, label='', mountargs='shortname=winnt,umask=0077', graft=None)

pylorax.imgutils.mkext4img(rootdir, outfile, size=None, label='', mountargs='', graft=None)

pylorax.imgutils.mkfsimage(fstype, rootdir, outfile, size=None, mkfsargs=None, mountargs='', graft=None)

Generic filesystem image creation function. fstype should be a filesystem type - "mkfs.${fstype}" must exist. graft should be a dict: {"some/path/in/image": "local/file/or/dir"}; if the path ends with a '/' it's assumed to be a directory. Will raise CalledProcessError if something goes wrong.

pylorax.imgutils.mkfsimage_from_disk(diskimage, fsimage, img_size=None, label='Anaconda')

Copy the / partition of a partitioned disk image to an un-partitioned disk image.

Parameters

• diskimage (str) -- The full path to partitioned disk image with a /

• fsimage (str) -- The full path of the output fs image file

• img_size (int) -- Optional size of the fsimage in MiB or None to make it as small as possible

• label (str) -- The label to apply to the image. Defaults to "Anaconda"

pylorax.imgutils.mkhfsimg(rootdir, outfile, size=None, label='', mountargs='', graft=None)

pylorax.imgutils.mkqcow2(outfile, size, options=None)

use qemu-img to create a file of the given size. options is a list of options passed to qemu-img

Default format is qcow2, override by passing "-f", fmt in options.

pylorax.imgutils.mkqemu_img(outfile, size, options=None)

use qemu-img to create a file of the given size. options is a list of options passed to qemu-img

Default format is qcow2, override by passing "-f", fmt in options.

pylorax.imgutils.mkrootfsimg(rootdir, outfile, label, size=2, sysroot='')

Make rootfs image from a directory

Parameters

• rootdir (str) -- Root directory

• outfile (str) -- Path of output image file

• label (str) -- Filesystem label

• size (int) -- Size of the image in GiB, if None computed automatically

• sysroot (str) -- path to system (deployment) root relative to physical root

pylorax.imgutils.mksparse(outfile, size)

use os.ftruncate to create a sparse file of the given size.

pylorax.imgutils.mksquashfs(rootdir, outfile, compression='default', compressargs=None)

Make a squashfs image containing the given rootdir.

pylorax.imgutils.mktar(root, outfile, compression='xz', compressargs=None, selinux=True)

pylorax.imgutils.mount(dev, opts='', mnt=None)

Mount the given device at the given mountpoint, using the given opts. opts should be a comma-separated string of mount options. if mnt is none, a temporary directory will be created and its path will be returned. raises CalledProcessError if mount fails.

pylorax.imgutils.round_to_blocks(size, blocksize)

If size isn't a multiple of blocksize, round up to the next multiple

pylorax.imgutils.umount(mnt, lazy=False, maxretry=3, retrysleep=1.0, delete=True)

Unmount the given mountpoint. If lazy is True, do a lazy umount (-l). If the mount was a temporary dir created by mount, it will be deleted. raises CalledProcessError if umount fails.

pylorax.installer module

exception pylorax.installer.InstallError

Bases: Exception

class pylorax.installer.QEMUInstall(opts, iso, ks_paths, disk_img, img_size=2048, kernel_args=None, memory=1024, vcpus=None, vnc=None, arch=None, cancel_func=None, virtio_host='127.0.0.1', virtio_port=6080, image_type=None, boot_uefi=False, ovmf_path=None)

Bases: object

Run qemu using an iso and a kickstart

QEMU_CMDS = {'aarch64': 'qemu-system-aarch64', 'arm': 'qemu-system-arm', 'i386': 'qemu-system-i386', 'ppc64le': 'qemu-system-ppc64', 'x86_64': 'qemu-system-x86_64'}

pylorax.installer.anaconda_cleanup(dirinstall_path)

Cleanup any leftover mounts from anaconda

Parameters

dirinstall_path (str) -- Path where anaconda mounts things

Returns

True if cleanups were successful. False if any of them failed.

If anaconda crashes it may leave things mounted under this path. It will typically be set to /mnt/sysimage/

Attempts to cleanup may also fail. Catch these and continue trying the other mountpoints.

Anaconda may also leave /run/anaconda.pid behind, clean that up as well.

pylorax.installer.append_initrd(initrd, files)

Append files to an initrd.

Parameters

• initrd (str) -- Path to initrd

• files (list) -- list of file paths to add

Returns

Path to a new initrd

Return type

str

The files are added to the initrd by creating a cpio image of the files (stored at /) and writing the cpio to the end of a copy of the initrd.

The initrd is not changed, a copy is made before appending the cpio archive.

pylorax.installer.create_vagrant_metadata(path, size=0)

Create a default Vagrant metadata.json file

Parameters

• path (str) -- Path to metadata.json file

• size (int) -- Disk size in MiB

pylorax.installer.find_free_port(start=5900, end=5999, host='127.0.0.1')

Return first free port in range.

Parameters

• start (int) -- Starting port number

• end (int) -- Ending port number

• host (str) -- Host IP to search

Returns

First free port or -1 if none found

Return type

int

pylorax.installer.novirt_cancel_check(cancel_funcs, proc)

Check to see if there has been an error in the logs

Parameters

• cancel_funcs (list) -- list of functions to call, True from any one cancels the build

• proc (subprocess.Popen) -- Popen object for the anaconda process

Returns

True if the process has been terminated

The cancel_funcs functions should return a True if an error has been detected. When an error is detected the process is terminated and this returns True

pylorax.installer.novirt_install(opts, disk_img, disk_size, cancel_func=None, tar_img=None)

Use Anaconda to install to a disk image

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• disk_img (str) -- The full path to the disk image to be created

• disk_size (int) -- The size of the disk_img in MiB

• cancel_func (function) -- Function that returns True to cancel build

• tar_img (str) -- For make_tar_disk, the path to final tarball to be created

This method runs anaconda to create the image and then based on the opts passed creates a qemu disk image or tarfile.

pylorax.installer.update_vagrant_metadata(path, size)

Update the Vagrant metadata.json file

Parameters

• path (str) -- Path to metadata.json file

• size (int) -- Disk size in MiB

This function makes sure that the provider, format and virtual size of the metadata file are set correctly. All other values are left untouched.

pylorax.installer.virt_install(opts, install_log, disk_img, disk_size, cancel_func=None, tar_img=None)

Use qemu to install to a disk image

Parameters

• opts (argparse options) -- options passed to livemedia-creator

• install_log (str) -- The path to write the log from qemu

• disk_img (str) -- The full path to the disk image to be created

• disk_size (int) -- The size of the disk_img in MiB

• cancel_func (function) -- Function that returns True to cancel build

• tar_img (str) -- For make_tar_disk, the path to final tarball to be created

This uses qemu with a boot.iso and a kickstart to create a disk image and then optionally, based on the opts passed, creates tarfile.

pylorax.ltmpl module

class pylorax.ltmpl.LiveTemplateRunner(dbo, fatalerrors=True, templatedir=None, defaults=None)

Bases: pylorax.ltmpl.TemplateRunner

This class parses and executes a limited Lorax template. Sample usage:

# install a bunch of packages runner = LiveTemplateRunner(dbo, templatedir, defaults) runner.run("live-install.tmpl")

It is meant to be used with the live-install.tmpl which lists the per-arch pacages needed to build the live-iso output.

installpkg(*pkgs)

installpkg [--required|--optional] [--except PKGGLOB [--except PKGGLOB ...]] PKGGLOB [PKGGLOB ...]

Request installation of all packages matching the given globs. Note that this is just a request - nothing is actually installed until the 'run_pkg_transaction' command is given.

--required is now the default. If the PKGGLOB can be missing pass --optional

class pylorax.ltmpl.LoraxTemplate(directories=None)

Bases: object

parse(template_file, variables)

class pylorax.ltmpl.LoraxTemplateRunner(inroot, outroot, dbo=None, fatalerrors=True, templatedir=None, defaults=None)

Bases: pylorax.ltmpl.TemplateRunner

This class parses and executes Lorax templates. Sample usage:

# install a bunch of packages runner = LoraxTemplateRunner(inroot=rundir, outroot=rundir, dbo=dnf_obj) runner.run("install-packages.ltmpl")

# modify a runtime dir runner = LoraxTemplateRunner(inroot=rundir, outroot=newrun) runner.run("runtime-transmogrify.ltmpl")

NOTES:

• Commands that run external programs (e.g. systemctl) currently use the host's copy of that program, which may cause problems if there's a big enough difference between the host and the image you're modifying.

• The commands are not executed under a real chroot, so absolute symlinks will point outside the inroot/outroot. Be careful with symlinks!

ADDING NEW COMMANDS:

• Each template command is just a method of the LoraxTemplateRunner object - so adding a new command is as easy as adding a new function.

• Each function gets arguments that correspond to the rest of the tokens on that line (after word splitting and brace expansion)

• Commands should raise exceptions for errors - don't use sys.exit()

append(filename, data)

append FILE STRING

Append STRING (followed by a newline character) to FILE. Python character escape sequences ('n', 't', etc.) will be converted to the appropriate characters.

Examples:

append /etc/depmod.d/dd.conf "search updates built-in" append /etc/resolv.conf ""

chmod(fileglob, mode)

chmod FILEGLOB OCTALMODE

Change the mode of all the files matching FILEGLOB to OCTALMODE.

copy(src, dest)

copy SRC DEST

Copy SRC to DEST. If DEST is a directory, SRC will be copied inside it. If DEST doesn't exist, SRC will be copied to a file with that name, if the path leading to it exists.

createaddrsize(addr, src, dest)

createaddrsize INITRD_ADDRESS INITRD ADDRSIZE

Create the initrd.addrsize file required in LPAR boot process.

Examples:

createaddrsize ${INITRD_ADDRESS} ${outroot}/${BOOTDIR}/initrd.img ${outroot}/${BOOTDIR}/initrd.addrsize

hardlink(src, dest)

hardlink SRC DEST

Create a hardlink at DEST which is linked to SRC.

install(srcglob, dest)

install SRC DEST

Copy the given file (or files, if a glob is used) from the input tree to the given destination in the output tree. The path to DEST must exist in the output tree. If DEST is a directory, SRC will be copied into that directory. If DEST doesn't exist, SRC will be copied to a file with that name, assuming the rest of the path exists. This is pretty much like how the 'cp' command works.

Examples:

install usr/share/myconfig/grub.conf /boot install /usr/share/myconfig/grub.conf.in /boot/grub.conf

installimg(*args)

installimg [--xz|--gzip|--bzip2|--lzma] [-ARG|--ARG=OPTION] SRCDIR DESTFILE

Create a compressed cpio archive of the contents of SRCDIR and place it in DESTFILE.

If SRCDIR doesn't exist or is empty nothing is created.

Examples:

installimg ${LORAXDIR}/product/ images/product.img installimg ${LORAXDIR}/updates/ images/updates.img installimg --xz -6 ${LORAXDIR}/updates/ images/updates.img installimg --xz -9 --memlimit-compress=3700MiB ${LORAXDIR}/updates/ images/updates.img

Optionally use a different compression type and override the default args passed to it. The default is xz -9

installinitrd(section, src, dest)

installinitrd SECTION SRC DEST

Same as installkernel, but for "initrd".

installkernel(section, src, dest)

installkernel SECTION SRC DEST

Install the kernel from SRC in the input tree to DEST in the output tree, and then add an item to the treeinfo data store, in the named SECTION, where "kernel" = DEST.

Equivalent to:

install SRC DEST treeinfo SECTION kernel DEST

installpkg(*pkgs)

installpkg [--required|--optional] [--except PKGGLOB [--except PKGGLOB ...]] PKGGLOB [PKGGLOB ...]

Request installation of all packages matching the given globs. Note that this is just a request - nothing is actually installed until the 'run_pkg_transaction' command is given.

--required is now the default. If the PKGGLOB can be missing pass --optional

installupgradeinitrd(section, src, dest)

installupgradeinitrd SECTION SRC DEST

Same as installkernel, but for "upgrade".

log(msg)

log MESSAGE

Emit the given log message. Be sure to put it in quotes!

Example:

log "Reticulating splines, please wait..."

mkdir(*dirs)

mkdir DIR [DIR ...]

Create the named DIR(s). Will create leading directories as needed.

Example:

mkdir /images

move(src, dest)

move SRC DEST

Move SRC to DEST.

remove(*fileglobs)

remove FILEGLOB [FILEGLOB ...]

Remove all the named files or directories. Will not raise exceptions if the file(s) are not found.

removefrom(pkg, *globs)

removefrom PKGGLOB [--allbut] FILEGLOB [FILEGLOB...]

Remove all files matching the given file globs from the package (or packages) named. If '--allbut' is used, all the files from the given package(s) will be removed except the ones which match the file globs.

Examples:

removefrom usbutils /usr/bin/* removefrom xfsprogs --allbut /sbin/*

removekmod(*globs)

removekmod GLOB [GLOB...] [--allbut] KEEPGLOB [KEEPGLOB...]

Remove all files and directories matching the given file globs from the kernel modules directory.

If '--allbut' is used, all the files from the modules will be removed except the ones which match the file globs. There must be at least one initial GLOB to search and one KEEPGLOB to keep. The KEEPGLOB is expanded to be KEEPGLOB so that it will match anywhere in the path.

This only removes files from under /lib/modules/*/kernel/

Examples:

removekmod sound drivers/media drivers/hwmon drivers/video removekmod drivers/char --allbut virtio_console hw_random

removepkg(*pkgs)

removepkg PKGGLOB [PKGGLOB...]

Delete the named package(s).

IMPLEMENTATION NOTES:

RPM scriptlets (%preun/%postun) are not run. Files are deleted, but directories are left behind.

replace(pat, repl, *fileglobs)

replace PATTERN REPLACEMENT FILEGLOB [FILEGLOB ...]

Find-and-replace the given PATTERN (Python-style regex) with the given REPLACEMENT string for each of the files listed.

Example:

replace @VERSION@ ${product.version} /boot/grub.conf /boot/isolinux.cfg

run_pkg_transaction()

Actually install all the packages requested by previous 'installpkg' commands.

runcmd(*cmdlist)

runcmd CMD [ARG ...]

Run the given command with the given arguments.

NOTE: All paths given MUST be COMPLETE, ABSOLUTE PATHS to the file or files mentioned. ${root}/${inroot}/${outroot} are good for constructing these paths.

FURTHER NOTE: Please use this command only as a last resort! Whenever possible, you should use the existing template commands. If the existing commands don't do what you need, fix them!

Examples:

(this should be replaced with a "find" function) runcmd find ${root} -name ".pyo" -type f -delete %for f in find(root, name=".pyo"): remove ${f} %endfor

symlink(target, dest)

symlink SRC DEST

Create a symlink at DEST which points to SRC.

systemctl(cmd, *units)

systemctl [enable|disable|mask] UNIT [UNIT...]

Enable, disable, or mask the given systemd units.

Examples:

systemctl disable lvm2-monitor.service systemctl mask fedora-storage-init.service fedora-configure.service

treeinfo(section, key, *valuetoks)

treeinfo SECTION KEY ARG [ARG ...]

Add an item to the treeinfo data store. The given SECTION will have a new item added where KEY = ARG ARG ...

Example:

treeinfo images-${kernel.arch} boot.iso images/boot.iso

class pylorax.ltmpl.TemplateRunner(fatalerrors=True, templatedir=None, defaults=None, builtins=None)

Bases: object

This class parses and executes Lorax templates. Sample usage:

# install a bunch of packages runner = LoraxTemplateRunner(inroot=rundir, outroot=rundir, dbo=dnf_obj) runner.run("install-packages.ltmpl")

NOTES:

• Parsing procedure is roughly: 1. Mako template expansion (on the whole file) 2. For each line of the result,

  1. Whitespace splitting (using shlex.split())

  2. Brace expansion (using brace_expand())

  3. If the first token is the name of a function, call that function with the rest of the line as arguments

• Parsing and execution are separate passes - so you can't use the result of a command in an %if statement (or any other control statements)!

run(templatefile, **variables)

pylorax.ltmpl.brace_expand(s)

pylorax.ltmpl.rexists(pathname, root='')

pylorax.ltmpl.rglob(pathname, root='/', fatal=False)

pylorax.ltmpl.split_and_expand(line)

pylorax.monitor module

class pylorax.monitor.LogMonitor(log_path=None, host='localhost', port=0, timeout=None, log_request_handler_class=<class 'pylorax.monitor.LogRequestHandler'>)

Bases: object

Setup a server to monitor the logs output by the installation

This needs to be running before the virt-install runs, it expects there to be a listener on the port used for the virtio log port.

shutdown()

Force shutdown of the monitoring thread

class pylorax.monitor.LogRequestHandler(request, client_address, server)

Bases: socketserver.BaseRequestHandler

Handle monitoring and saving the logfiles from the virtual install

Incoming data is written to self.server.log_path and each line is checked for patterns that would indicate that the installation failed. self.server.log_error is set True when this happens.

finish()

handle()

Write incoming data to a logfile and check for errors

Split incoming data into lines and check for any Tracebacks or other errors that indicate that the install failed.

Loops until self.server.kill is True

iserror(line)

Check a line to see if it contains an error indicating installation failure

Parameters

line (str) -- log line to check for failure

If the line contains IGNORED it will be skipped.

re_tests = ['packaging: base repo .* not valid', 'packaging: .* requires .*']

setup()

Start writing to self.server.log_path

simple_tests = ['Traceback (', 'traceback script(s) have been run', 'Out of memory:', 'Call Trace:', 'insufficient disk space:', 'Not enough disk space to download the packages', 'error populating transaction after', 'crashed on signal', 'packaging: Missed: NoSuchPackage', 'packaging: Installation failed', 'The following error occurred while installing.  This is a fatal error', 'Error in POSTIN scriptlet in rpm package']

class pylorax.monitor.LogServer(log_path, *args, **kwargs)

Bases: socketserver.TCPServer

A TCP Server that listens for log data

log_check()

Check to see if an error has been found in the log

Returns

True if there has been an error

Return type

bool

timeout = 60

pylorax.mount module

class pylorax.mount.IsoMountpoint(iso_path, initrd_path=None)

Bases: object

Mount the iso and check to make sure the vmlinuz and initrd.img files exist

Also check the iso for a a stage2 image and set a flag and extract the iso's label.

stage2 can be either LiveOS/squashfs.img or images/install.img

get_iso_label()

Get the iso's label using isoinfo

Sets self.label if one is found

umount()

Unmount the iso

pylorax.output module

pylorax.sysutils module

pylorax.sysutils.chmod_(path, mode, recursive=False)

pylorax.sysutils.chown_(path, user=None, group=None, recursive=False)

pylorax.sysutils.joinpaths(*args, **kwargs)

pylorax.sysutils.linktree(src, dst)

pylorax.sysutils.remove(target)

pylorax.sysutils.replace(fname, find, sub)

pylorax.sysutils.touch(fname)

pylorax.treebuilder module

class pylorax.treebuilder.RuntimeBuilder(product, arch, dbo, templatedir=None, installpkgs=None, excludepkgs=None, add_templates=None, add_template_vars=None, skip_branding=False)

Bases: object

Builds the anaconda runtime image.

cleanup()

Remove unneeded packages and files with runtime-cleanup.tmpl

create_ext4_runtime(outfile='/var/tmp/squashfs.img', compression='xz', compressargs=None, size=2)

Create a squashfs compressed ext4 runtime

create_squashfs_runtime(outfile='/var/tmp/squashfs.img', compression='xz', compressargs=None, size=2)

Create a plain squashfs runtime

finished()

Done using RuntimeBuilder

Close the dnf base object

generate_module_data()

install()

Install packages and do initial setup with runtime-install.tmpl

postinstall()

Do some post-install setup work with runtime-postinstall.tmpl

verify()

Ensure that contents of the installroot can run

writepkglists(pkglistdir)

debugging data: write out lists of package contents

writepkgsizes(pkgsizefile)

debugging data: write a big list of pkg sizes

class pylorax.treebuilder.TreeBuilder(product, arch, inroot, outroot, runtime, isolabel, domacboot=True, doupgrade=True, templatedir=None, add_templates=None, add_template_vars=None, workdir=None, extra_boot_args='')

Bases: object

Builds the arch-specific boot images. inroot should be the installtree root (the newly-built runtime dir)

build()

copy_dracut_hooks(hooks)

Copy the hook scripts in hooks into the installroot's /tmp/ and return a list of commands to pass to dracut when creating the initramfs

hooks is a list of tuples with the name of the hook script and the target dracut hook directory (eg. [("99anaconda-copy-ks.sh", "/lib/dracut/hooks/pre-pivot")])

property dracut_hooks_path

Return the path to the lorax dracut hooks scripts

Use the configured share dir if it is setup, otherwise default to /usr/share/lorax/dracut_hooks

implantisomd5()

property kernels

rebuild_initrds(add_args=None, backup='', prefix='')

Rebuild all the initrds in the tree. If backup is specified, each initrd will be renamed with backup as a suffix before rebuilding. If backup is empty, the existing initrd files will be overwritten. If suffix is specified, the existing initrd is untouched and a new image is built with the filename "${prefix}-${kernel.version}.img"

If the initrd doesn't exist its name will be created based on the name of the kernel.

pylorax.treebuilder.findkernels(root='/', kdir='boot')

pylorax.treebuilder.generate_module_info(moddir, outfile=None)

pylorax.treebuilder.string_lower(string)

Return a lowercase string.

Parameters

string -- String to lowercase

This is used as a filter in the templates.

pylorax.treebuilder.udev_escape(label)

pylorax.treeinfo module

class pylorax.treeinfo.TreeInfo(product, version, variant, basearch, packagedir='')

Bases: object

add_section(section, data)

write(outfile)

Module contents

class pylorax.ArchData(buildarch)

Bases: pylorax.base.DataHolder

bcj_arch = {'arm': 'arm', 'armhfp': 'arm', 'i386': 'x86', 'ppc64le': 'powerpc', 'x86_64': 'x86'}

lib64_arches = ('x86_64', 'ppc64le', 's390x', 'ia64', 'aarch64')

class pylorax.Lorax

Bases: pylorax.base.BaseLoraxClass

configure(conf_file='/etc/lorax/lorax.conf')

init_file_logging(logdir, logname='pylorax.log')

init_stream_logging()

run(dbo, product, version, release, variant='', bugurl='', isfinal=False, workdir=None, outputdir=None, buildarch=None, volid=None, domacboot=True, doupgrade=True, remove_temp=False, installpkgs=None, excludepkgs=None, size=2, add_templates=None, add_template_vars=None, add_arch_templates=None, add_arch_template_vars=None, verify=True, user_dracut_args=None, squashfs_only=False, skip_branding=False)

property templatedir

Find the template directory.

Pick the first directory under sharedir/templates.d/ if it exists. Otherwise use the sharedir

pylorax.find_templates(templatedir='/usr/share/lorax')

Find the templates to use.

Parameters

templatedir (str) -- Top directory to search for templates

Returns

Path to templates

Return type

str

If there is a templates.d directory under templatedir the lowest numbered directory entry is returned.

eg. /usr/share/lorax/templates.d/99-generic/

pylorax.get_buildarch(dbo)

pylorax.log_selinux_state()

Log the current state of selinux

pylorax.setup_logging(logfile, theLogger)

Setup the various logs

Parameters

• logfile (string) -- filename to write the log to

• theLogger (logging.Logger) -- top-level logger