pylorax.api package¶
Submodules¶
pylorax.api.cmdline module¶
pylorax.api.compose module¶
Setup for composing an image
Adding New Output Types¶
The new output type must add a kickstart template to ./share/composer/ where the name of the kickstart (without the trailing .ks) matches the entry in compose_args.
The kickstart should not have any url or repo entries, these will be added at build time. The %packages section should be the last thing, and while it can contain mandatory packages required by the output type, it should not have the trailing %end because the package NEVRAs will be appended to it at build time.
compose_args should have a name matching the kickstart, and it should set the novirt_install parameters needed to generate the desired output. Other types should be set to False.
-
pylorax.api.compose.
add_customizations
(f, recipe)[source]¶ Add customizations to the kickstart file
Parameters: - f (open file object) – kickstart file object
- recipe (Recipe object) –
Returns: None
Raises: RuntimeError if there was a problem writing to the kickstart
-
pylorax.api.compose.
compose_args
(compose_type)[source]¶ Returns the settings to pass to novirt_install for the compose type
Parameters: compose_type (str) – The type of compose to create, from compose_types() This will return a dict of options that match the ArgumentParser options for livemedia-creator. These are the ones the define the type of output, it’s filename, etc. Other options will be filled in by make_compose()
-
pylorax.api.compose.
compose_types
(share_dir)[source]¶ Returns a list of the supported output types
The output types come from the kickstart names in /usr/share/lorax/composer/*ks
-
pylorax.api.compose.
move_compose_results
(cfg, results_dir)[source]¶ Move the final image to the results_dir and cleanup the unneeded compose files
Parameters: - cfg (DataHolder) – Build configuration
- results_dir (str) – Directory to put the results into
-
pylorax.api.compose.
repo_to_ks
(r, url='url')[source]¶ Return a kickstart line with the correct args. :param r: DNF repository information :type r: dnf.Repo :param url: “url” or “baseurl” to use for the baseurl parameter :type url: str :returns: kickstart command arguments for url/repo command :rtype: str
Set url to “baseurl” if it is a repo, leave it as “url” for the installation url.
-
pylorax.api.compose.
start_build
(cfg, dnflock, gitlock, branch, recipe_name, compose_type, test_mode=0)[source]¶ Start the build
Parameters: - cfg (ComposerConfig) – Configuration object
- dnflock (YumLock) – Lock and YumBase for depsolving
- recipe (str) – The recipe to build
- compose_type (str) – The type of output to create from the recipe
Returns: Unique ID for the build that can be used to track its status
Return type:
-
pylorax.api.compose.
test_templates
(dbo, share_dir)[source]¶ Try depsolving each of the the templates and report any errors
Parameters: dbo (dnf.Base) – dnf base object Returns: List of template types and errors Return type: List of errors Return a list of templates and errors encountered or an empty list
-
pylorax.api.compose.
write_ks_group
(f, group)[source]¶ Write kickstart group entry
Parameters: - f (open file object) – kickstart file object
- group – A blueprint group dictionary
gid is optional
-
pylorax.api.compose.
write_ks_user
(f, user)[source]¶ Write kickstart user and sshkey entry
Parameters: - f (open file object) – kickstart file object
- user (dict) – A blueprint user dictionary
If the entry contains a ssh key, use sshkey to write it All of the user fields are optional, except name, write out a kickstart user entry with whatever options are relevant.
pylorax.api.config module¶
-
class
pylorax.api.config.
ComposerConfig
(*args, **kwargs)[source]¶ Bases:
configparser.SafeConfigParser
-
pylorax.api.config.
configure
(conf_file='/etc/lorax/composer.conf', root_dir='/', test_config=False)[source]¶ lorax-composer configuration
Parameters:
-
pylorax.api.config.
make_dnf_dirs
(conf)[source]¶ Make any missing dnf directories
Parameters: conf (ComposerConfig) – The configuration to use Returns: None
-
pylorax.api.config.
make_queue_dirs
(conf, gid)[source]¶ Make any missing queue directories
Parameters: - conf (ComposerConfig) – The configuration to use
- gid (int) – Group ID that has access to the queue directories
Returns: list of errors
Return type: list of str
pylorax.api.crossdomain module¶
pylorax.api.dnfbase module¶
pylorax.api.projects module¶
-
pylorax.api.projects.
api_changelog
(changelog)[source]¶ Convert the changelog to a string
Parameters: changelog (tuple) – A list of time, author, string tuples. Returns: The most recent changelog text or “” Return type: str This returns only the most recent changelog entry.
-
pylorax.api.projects.
api_time
(t)[source]¶ Convert time since epoch to a string
Parameters: t (int) – Seconds since epoch Returns: Time string Return type: str
-
pylorax.api.projects.
delete_repo_source
(source_glob, source_name)[source]¶ Delete a source from a repo file
Parameters: source_glob (str) – A glob of the repo sources to search Returns: None Raises: ProjectsError if there was a problem A repo file may have multiple sources in it, delete only the selected source. If it is the last one in the file, delete the file.
WARNING: This will delete ANY source, the caller needs to ensure that a system source_name isn’t passed to it.
-
pylorax.api.projects.
dep_evra
(dep)[source]¶ Return the epoch:version-release.arch for the dep
Parameters: dep (dict) – dependency dict Returns: epoch:version-release.arch Return type: str
-
pylorax.api.projects.
dnf_repo_to_file_repo
(repo)[source]¶ Return a string representation of a DNF Repo object suitable for writing to a .repo file
Parameters: repo (dnf.RepoDict) – DNF Repository Returns: A string Return type: str The DNF Repo.dump() function does not produce a string that can be used as a dnf .repo file, it ouputs baseurl and gpgkey as python lists which DNF cannot read. So do this manually with only the attributes we care about.
-
pylorax.api.projects.
estimate_size
(packages, block_size=6144)[source]¶ Estimate the installed size of a package list
Parameters: - packages (list of hawkey.Package objects) – The packages to be installed
- block_size (int) – The block size to use for rounding up file sizes.
Returns: The estimated size of installed packages
Return type: Estimating actual requirements is difficult without the actual file sizes, which dnf doesn’t provide access to. So use the file count and block size to estimate a minimum size for each package.
-
pylorax.api.projects.
get_repo_sources
(source_glob)[source]¶ Return a list of sources from a directory of yum repositories
Parameters: source_glob (str) – A glob to use to match the source files, including full path Returns: A list of the source ids in all of the matching files Return type: list of str
-
pylorax.api.projects.
get_source_ids
(source_path)[source]¶ Return a list of the source ids in a file
Parameters: source_path (str) – Full path and filename of the source (yum repo) file Returns: A list of source id strings Return type: list of str
-
pylorax.api.projects.
modules_info
(dbo, module_names)[source]¶ Return details about a module, including dependencies
Parameters: - dbo (dnf.Base) – dnf base object
- module_names (str) – Names of the modules to get info about
Returns: List of dicts with module details and dependencies.
Return type: list of dicts
-
pylorax.api.projects.
modules_list
(dbo, module_names)[source]¶ Return a list of modules
Parameters: - dbo (dnf.Base) – dnf base object
- offset – Number of modules to skip
- limit (int) – Maximum number of modules to return
Returns: List of module information and total count
Return type: tuple of a list of dicts and an Int
Modules don’t exist in RHEL7 so this only returns projects and sets the type to “rpm”
-
pylorax.api.projects.
pkg_to_dep
(pkg)[source]¶ Extract the info from a hawkey.Package object
Parameters: pkg (hawkey.Package) – A hawkey.Package object Returns: A dict with name, epoch, version, release, arch Return type: dict
-
pylorax.api.projects.
pkg_to_project
(pkg)[source]¶ Extract the details from a hawkey.Package object
Parameters: pkgs (hawkey.Package) – hawkey.Package object with package details Returns: A dict with the name, summary, description, and url. Return type: dict upstream_vcs is hard-coded to UPSTREAM_VCS
-
pylorax.api.projects.
pkg_to_project_info
(pkg)[source]¶ Extract the details from a hawkey.Package object
Parameters: pkg (hawkey.Package) – hawkey.Package object with package details Returns: A dict with the project details, as well as epoch, release, arch, build_time, changelog, … Return type: dict metadata entries are hard-coded to {}
-
pylorax.api.projects.
proj_to_module
(proj)[source]¶ Extract the name from a project_info dict
Parameters: pkg (dict) – dict with package details Returns: A dict with name, and group_type Return type: dict group_type is hard-coded to “rpm”
-
pylorax.api.projects.
projects_depsolve
(dbo, projects, groups)[source]¶ Return the dependencies for a list of projects
Parameters: - dbo (dnf.Base) – dnf base object
- projects (List of Strings) – The projects to find the dependencies for
- groups (List of str) – The groups to include in dependency solving
Returns: NEVRA’s of the project and its dependencies
Return type: list of dicts
Raises: ProjectsError if there was a problem installing something
-
pylorax.api.projects.
projects_depsolve_with_size
(dbo, projects, groups, with_core=True)[source]¶ Return the dependencies and installed size for a list of projects
Parameters: - dbo (dnf.Base) – dnf base object
- project_names (List of Strings) – The projects to find the dependencies for
- groups (List of str) – The groups to include in dependency solving
Returns: installed size and a list of NEVRA’s of the project and its dependencies
Return type: tuple of (int, list of dicts)
Raises: ProjectsError if there was a problem installing something
-
pylorax.api.projects.
projects_info
(dbo, project_names)[source]¶ Return details about specific projects
Parameters: - dbo (dnf.Base) – dnf base object
- project_names (str) – List of names of projects to get info about
Returns: List of project info dicts with pkg_to_project as well as epoch, version, release, etc.
Return type: list of dicts
If project_names is None it will return the full list of available packages
-
pylorax.api.projects.
projects_list
(dbo)[source]¶ Return a list of projects
Parameters: dbo (dnf.Base) – dnf base object Returns: List of project info dicts with name, summary, description, homepage, upstream_vcs Return type: list of dicts
-
pylorax.api.projects.
repo_to_source
(repo, system_source)[source]¶ Return a Weldr Source dict created from the DNF Repository
Parameters: - repo (dnf.RepoDict) – DNF Repository
- system_source (bool) – True if this source is an immutable system source
Returns: A dict with Weldr Source fields filled in
Return type: Example:
{ "check_gpg": true, "check_ssl": true, "gpgkey_url": [ "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64" ], "name": "fedora", "proxy": "http://proxy.brianlane.com:8123", "system": true "type": "yum-metalink", "url": "https://mirrors.fedoraproject.org/metalink?repo=fedora-28&arch=x86_64" }
-
pylorax.api.projects.
source_to_repo
(source, dnf_conf)[source]¶ Return a dnf Repo object created from a source dict
Parameters: source (dict) – A Weldr source dict Returns: A dnf Repo object Return type: dnf.Repo Example:
{ "check_gpg": True, "check_ssl": True, "gpgkey_urls": [ "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64" ], "name": "fedora", "proxy": "http://proxy.brianlane.com:8123", "system": True "type": "yum-metalink", "url": "https://mirrors.fedoraproject.org/metalink?repo=fedora-28&arch=x86_64" }
pylorax.api.queue module¶
Functions to monitor compose queue and run anaconda
-
pylorax.api.queue.
build_status
(cfg, status_filter=None)[source]¶ Return the details of finished or failed builds
Parameters: - cfg (ComposerConfig) – Configuration settings
- status_filter (str) – What builds to return. None == all, “FINISHED”, or “FAILED”
Returns: A list of the build details (from compose_details)
Return type: list of dicts
This returns a list of build details for each of the matching builds on the system. It does not return the status of builds that have not been finished. Use queue_status() for those.
-
pylorax.api.queue.
compose_detail
(results_dir)[source]¶ Return details about the build.
Parameters: results_dir (str) – The directory containing the metadata and results for the build Returns: A dictionary with details about the compose Return type: dict Raises: IOError if it cannot read the directory, STATUS, or blueprint file. The following details are included in the dict:
- id - The uuid of the comoposition
- queue_status - The final status of the composition (FINISHED or FAILED)
- compose_type - The type of output generated (tar, iso, etc.)
- blueprint - Blueprint name
- version - Blueprint version
- image_size - Size of the image, if finished. 0 otherwise.
Various timestamps are also included in the dict. These are all Unix UTC timestamps. It is possible for these timestamps to not always exist, in which case they will be None in Python (or null in JSON). The following timestamps are included:
- job_created - When the user submitted the compose
- job_started - Anaconda started running
- job_finished - Job entered FINISHED or FAILED state
-
pylorax.api.queue.
get_compose_type
(results_dir)[source]¶ Return the type of composition.
Parameters: results_dir (str) – The directory containing the metadata and results for the build Returns: The type of compose (eg. ‘tar’) Return type: str Raises: RuntimeError if no kickstart template can be found.
-
pylorax.api.queue.
get_image_name
(uuid_dir)[source]¶ Return the filename and full path of the build’s image file
Parameters: uuid (str) – The UUID of the build Returns: The image filename and full path Return type: tuple of strings Raises: RuntimeError if there was a problem (eg. invalid uuid, missing config file)
-
pylorax.api.queue.
make_compose
(cfg, results_dir)[source]¶ Run anaconda with the final-kickstart.ks from results_dir
Parameters: - cfg (DataHolder) – Configuration settings
- results_dir (str) – The directory containing the metadata and results for the build
Returns: Nothing
Raises: May raise various exceptions
This takes the final-kickstart.ks, and the settings in config.toml and runs Anaconda in no-virt mode (directly on the host operating system). Exceptions should be caught at the higer level.
If there is a failure, the build artifacts will be cleaned up, and any logs will be moved into logs/anaconda/ and their ownership will be set to the user from the cfg object.
-
pylorax.api.queue.
monitor
(cfg)[source]¶ Monitor the queue for new compose requests
Parameters: cfg (DataHolder) – Configuration settings Returns: Does not return The queue has 2 subdirectories, new and run. When a compose is ready to be run a symlink to the uniquely named results directory should be placed in ./queue/new/
When the it is ready to be run (it is checked every 30 seconds or after a previous compose is finished) the symlink will be moved into ./queue/run/ and a STATUS file will be created in the results directory.
STATUS can contain one of: RUNNING, FINISHED, FAILED
If the system is restarted while a compose is running it will move any old symlinks from ./queue/run/ to ./queue/new/ and rerun them.
-
pylorax.api.queue.
queue_status
(cfg)[source]¶ Return details about what is in the queue.
Parameters: cfg (ComposerConfig) – Configuration settings Returns: A list of the new composes, and a list of the running composes Return type: dict This returns a dict with 2 lists. “new” is the list of uuids that are waiting to be built, and “run” has the uuids that are being built (currently limited to 1 at a time).
-
pylorax.api.queue.
start_queue_monitor
(cfg, uid, gid)[source]¶ Start the queue monitor as a mp process
Parameters: - cfg (ComposerConfig) – Configuration settings
- uid (int) – User ID that owns the queue
- gid (int) – Group ID that owns the queue
Returns: None
-
pylorax.api.queue.
uuid_cancel
(cfg, uuid)[source]¶ Cancel a build and delete its results
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
Returns: True if it was canceled and deleted
Return type: Only call this if the build status is WAITING or RUNNING
-
pylorax.api.queue.
uuid_delete
(cfg, uuid)[source]¶ Delete all of the results from a compose
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
Returns: True if it was deleted
Return type: Raises: This will raise an error if the delete failed
-
pylorax.api.queue.
uuid_image
(cfg, uuid)[source]¶ Return the filename and full path of the build’s image file
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
Returns: The image filename and full path
Return type: tuple of strings
Raises: RuntimeError if there was a problem (eg. invalid uuid, missing config file)
-
pylorax.api.queue.
uuid_info
(cfg, uuid)[source]¶ Return information about the composition
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
Returns: dictionary of information about the composition or None
Return type: Raises: RuntimeError if there was a problem
This will return a dict with the following fields populated:
- id - The uuid of the comoposition
- config - containing the configuration settings used to run Anaconda
- blueprint - The depsolved blueprint used to generate the kickstart
- commit - The (local) git commit hash for the blueprint used
- deps - The NEVRA of all of the dependencies used in the composition
- compose_type - The type of output generated (tar, iso, etc.)
- queue_status - The final status of the composition (FINISHED or FAILED)
-
pylorax.api.queue.
uuid_log
(cfg, uuid, size=1024)[source]¶ Return size kbytes from the end of the anaconda.log
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
- size (int) – Number of kbytes to read. Default is 1024
Returns: Up to size kbytes from the end of the log
Return type: Raises: RuntimeError if there was a problem (eg. no log file available)
This function tries to return lines from the end of the log, it will attempt to start on a line boundry, and may return less than size kbytes.
-
pylorax.api.queue.
uuid_status
(cfg, uuid)[source]¶ Return the details of a specific UUID compose
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
Returns: Details about the build
Return type: dict or None
Returns the same dict as compose_details()
-
pylorax.api.queue.
uuid_tar
(cfg, uuid, metadata=False, image=False, logs=False)[source]¶ Return a tar of the build data
Parameters: - cfg (ComposerConfig) – Configuration settings
- uuid (str) – The UUID of the build
- metadata (bool) – Set to true to include all the metadata needed to reproduce the build
- image (bool) – Set to true to include the output image
- logs (bool) – Set to true to include the logs from the build
Returns: A stream of bytes from tar
Return type: A generator
Raises: RuntimeError if there was a problem (eg. missing config file)
This yields an uncompressed tar’s data to the caller. It includes the selected data to the caller by returning the Popen stdout from the tar process.
pylorax.api.recipes module¶
-
class
pylorax.api.recipes.
CommitDetails
(commit, timestamp, message, revision=None)[source]¶ Bases:
pylorax.base.DataHolder
-
class
pylorax.api.recipes.
Recipe
(name, description, version, modules, packages, groups, customizations=None)[source]¶ Bases:
dict
A Recipe of package and modules
This is a subclass of dict that enforces the constructor arguments and adds a .filename property to return the recipe’s filename, and a .toml() function to return the recipe as a TOML string.
-
bump_version
(old_version=None)[source]¶ semver recipe version number bump
Parameters: old_version (str) – An optional old version number Returns: The new version number or None Return type: str Raises: ValueError If neither have a version, 0.0.1 is returned If there is no old version the new version is checked and returned If there is no new version, but there is a old one, bump its patch level If the old and new versions are the same, bump the patch level If they are different, check and return the new version
-
filename
¶ Return the Recipe’s filename
Replaces spaces in the name with ‘-‘ and appends .toml
-
freeze
(deps)[source]¶ Return a new Recipe with full module and package NEVRA
Parameters: deps (list() – A list of dependency NEVRA to use to fill in the modules and packages Returns: A new Recipe object Return type: Recipe
-
group_names
¶ Return the names of the groups. Groups do not have versions.
-
module_names
¶ Return the names of the modules
-
module_nver
¶ Return the names and version globs of the modules
-
package_names
¶ Return the names of the packages
-
package_nver
¶ Return the names and version globs of the packages
-
-
pylorax.api.recipes.
commit_recipe
(repo, branch, recipe)[source]¶ Commit a recipe to a branch
Parameters: Returns: OId of the new commit
Return type: Git.OId
Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
commit_recipe_directory
(repo, branch, directory)[source]¶ Commit all *.toml files from a directory, if they aren’t already in git.
Parameters: Returns: None
Raises: Can raise errors from Ggit or RecipeFileError
Files with Toml or RecipeFileErrors will be skipped, and the remainder will be tried.
-
pylorax.api.recipes.
commit_recipe_file
(repo, branch, filename)[source]¶ Commit a recipe file to a branch
Parameters: Returns: OId of the new commit
Return type: Git.OId
Raises: Can raise errors from Ggit or RecipeFileError
-
pylorax.api.recipes.
delete_file
(repo, branch, filename)[source]¶ Delete a file from a branch.
Parameters: Returns: OId of the new commit
Return type: Git.OId
Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
delete_recipe
(repo, branch, recipe_name)[source]¶ Delete a recipe from a branch.
Parameters: Returns: OId of the new commit
Return type: Git.OId
Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
diff_items
(title, old_items, new_items)[source]¶ Return the differences between two lists of dicts.
Parameters: Returns: List of diff dicts with old/new entries
Return type:
-
pylorax.api.recipes.
find_commit_tag
(repo, branch, filename, commit_id)[source]¶ Find the tag that matches the commit_id
Parameters: Returns: The tag or None if there isn’t one
Return type: str or None
There should be only 1 tag pointing to a commit, but there may not be a tag at all.
The tag will look like: ‘refs/tags/<branch>/<filename>/r<revision>’
-
pylorax.api.recipes.
find_name
(name, lst)[source]¶ Find the dict matching the name in a list and return it.
Parameters: - name (str) – Name to search for
- lst – List of dict’s with “name” field
Returns: First dict with matching name, or None
Return type: dict or None
-
pylorax.api.recipes.
get_commit_details
(commit, revision=None)[source]¶ Return the details about a specific commit.
Parameters: - commit (Git.Commit) – The commit to get details from
- revision (int) – Optional commit revision
Returns: Details about the commit
Return type: Raises: CommitTimeValError or Ggit exceptions
-
pylorax.api.recipes.
get_revision_from_tag
(tag)[source]¶ Return the revision number from a tag
Parameters: tag (str) – The tag to exract the revision from Returns: The integer revision or None Return type: int or None The revision is the part after the r in ‘branch/filename/rXXX’
-
pylorax.api.recipes.
head_commit
(repo, branch)[source]¶ Get the branch’s HEAD Commit Object
Parameters: - repo (Git.Repository) – Open repository
- branch (str) – Branch name
Returns: Branch’s head commit
Return type: Git.Commit
Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
is_commit_tag
(repo, commit_id, tag)[source]¶ Check to see if a tag points to a specific commit.
Parameters: - repo (Git.Repository) – Open repository
- commit_id (Git.OId) – The commit id to check
- tag (str) – The tag to check
Returns: True if the tag points to the commit, False otherwise
Return type:
-
pylorax.api.recipes.
is_parent_diff
(repo, filename, tree, parent)[source]¶ Check to see if the commit is different from its parents
Parameters: - repo (Git.Repository) – Open repository
- filename (str) – filename to revert
- tree (Git.Tree) – The commit’s tree
- parent (Git.Commit) – The commit’s parent commit
Retuns: True if filename in the commit is different from its parents
Return type:
-
pylorax.api.recipes.
list_branch_files
(repo, branch)[source]¶ Return a sorted list of the files on the branch HEAD
Parameters: - repo (Git.Repository) – Open repository
- branch (str) – Branch name
Returns: A sorted list of the filenames
Return type: Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
list_commit_files
(repo, commit)[source]¶ Return a sorted list of the files on a commit
Parameters: - repo (Git.Repository) – Open repository
- commit (str) – The commit hash to list
Returns: A sorted list of the filenames
Return type: Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
list_commits
(repo, branch, filename, limit=0)[source]¶ List the commit history of a file on a branch.
Parameters: Returns: A list of commit details
Return type: Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
open_or_create_repo
(path)[source]¶ Open an existing repo, or create a new one
Parameters: path (string) – path to recipe directory Returns: A repository object Return type: Git.Repository Raises: Can raise errors from Ggit A bare git repo will be created in the git directory of the specified path. If a repo already exists it will be opened and returned instead of creating a new one.
-
pylorax.api.recipes.
prepare_commit
(repo, branch, builder)[source]¶ Prepare for a commit
Parameters: - repo (Git.Repository) – Open repository
- branch (str) – Branch name
- builder (TreeBuilder) – instance of TreeBuilder
Returns: (Tree, Sig, Ref)
Return type: Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
read_commit
(repo, branch, filename, commit=None)[source]¶ Return the contents of a file on a specific branch or commit.
Parameters: Returns: The commit id, and the contents of the commit
Return type: Raises: Can raise errors from Ggit
If no commit is passed the master:filename is returned, otherwise it will be commit:filename
-
pylorax.api.recipes.
read_commit_spec
(repo, spec)[source]¶ Return the raw content of the blob specified by the spec
Parameters: - repo (Git.Repository) – Open repository
- spec (str) – Git revparse spec
Returns: Contents of the commit
Return type: Raises: Can raise errors from Ggit
eg. To read the README file from master the spec is “master:README”
-
pylorax.api.recipes.
read_recipe_and_id
(repo, branch, recipe_name, commit=None)[source]¶ Read a recipe commit and its id from git
Parameters: Returns: The commit id, and a Recipe object
Return type: Raises: Can raise errors from Ggit
If no commit is passed the master:filename is returned, otherwise it will be commit:filename
-
pylorax.api.recipes.
read_recipe_commit
(repo, branch, recipe_name, commit=None)[source]¶ Read a recipe commit from git and return a Recipe object
Parameters: Returns: A Recipe object
Return type: Raises: Can raise errors from Ggit
If no commit is passed the master:filename is returned, otherwise it will be commit:filename
-
pylorax.api.recipes.
recipe_diff
(old_recipe, new_recipe)[source]¶ Diff two versions of a recipe
Parameters: Returns: A list of diff dict entries with old/new
Return type:
-
pylorax.api.recipes.
recipe_filename
(name)[source]¶ Return the toml filename for a recipe
Replaces spaces with ‘-‘ and appends ‘.toml’
-
pylorax.api.recipes.
recipe_from_dict
(recipe_dict)[source]¶ Create a Recipe object from a plain dict.
Parameters: recipe_dict (dict) – A plain dict of the recipe Returns: A Recipe object Return type: Recipe Raises: RecipeError
-
pylorax.api.recipes.
recipe_from_file
(recipe_path)[source]¶ Return a recipe file as a Recipe object
Parameters: recipe_path (str) – Path to the recipe fila Returns: A Recipe object Return type: Recipe
-
pylorax.api.recipes.
recipe_from_toml
(recipe_str)[source]¶ Create a Recipe object from a toml string.
Parameters: recipe_str (str) – The Recipe TOML string Returns: A Recipe object Return type: Recipe Raises: TomlError
-
pylorax.api.recipes.
revert_file
(repo, branch, filename, commit)[source]¶ Revert the contents of a file to that of a previous commit
Parameters: Returns: OId of the new commit
Return type: Git.OId
Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
revert_recipe
(repo, branch, recipe_name, commit)[source]¶ Revert the contents of a recipe to that of a previous commit
Parameters: Returns: OId of the new commit
Return type: Git.OId
Raises: Can raise errors from Ggit
-
pylorax.api.recipes.
tag_file_commit
(repo, branch, filename)[source]¶ Tag a file’s most recent commit
Parameters: Returns: Tag id or None if it failed.
Return type: Git.OId
Raises: Can raise errors from Ggit
This uses git tags, of the form refs/tags/<branch>/<filename>/r<revision> Only the most recent recipe commit can be tagged to prevent out of order tagging. Revisions start at 1 and increment for each new commit that is tagged. If the commit has already been tagged it will return false.
-
pylorax.api.recipes.
tag_recipe_commit
(repo, branch, recipe_name)[source]¶ Tag a file’s most recent commit
Parameters: Returns: Tag id or None if it failed.
Return type: Git.OId
Raises: Can raise errors from Ggit
Uses tag_file_commit()
pylorax.api.server module¶
pylorax.api.v0 module¶
Setup v0 of the API server
v0_api() must be called to setup the API routes for Flask
Status Responses¶
Some requests only return a status/error response.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
Example response:
{ "status": true }Error response:
{ "errors": ["ggit-error: Failed to remove entry. File isn't in the tree - jboss.toml (-1)"] "status": false }
API Routes¶
All of the blueprints routes support the optional branch argument. If it is not used then the API will use the master branch for blueprints. If you want to create a new branch use the new or workspace routes with ?branch=<branch-name> to store the new blueprint on the new branch.
/api/v0/blueprints/list¶
List the available blueprints:
{ "limit": 20, "offset": 0, "blueprints": [ "atlas", "development", "glusterfs", "http-server", "jboss", "kubernetes" ], "total": 6 }
/api/v0/blueprints/info/<blueprint_names>¶
Return the JSON representation of the blueprint. This includes 3 top level objects. changes which lists whether or not the workspace is different from the most recent commit. blueprints which lists the JSON representation of the blueprint, and errors which will list any errors, like non-existant blueprints.
Example:
{ "changes": [ { "changed": false, "name": "glusterfs" } ], "errors": [], "blueprints": [ { "description": "An example GlusterFS server with samba", "modules": [ { "name": "glusterfs", "version": "3.7.*" }, { "name": "glusterfs-cli", "version": "3.7.*" } ], "name": "glusterfs", "packages": [ { "name": "2ping", "version": "3.2.1" }, { "name": "samba", "version": "4.2.*" } ], "version": "0.0.6" } ] }Error example:
{ "changes": [], "errors": ["ggit-error: the path 'missing.toml' does not exist in the given tree (-3)"] "blueprints": [] }
/api/v0/blueprints/changes/<blueprint_names>[?offset=0&limit=20]¶
Return the commits to a blueprint. By default it returns the first 20 commits, this can be changed by passing offset and/or limit. The response will include the commit hash, summary, timestamp, and optionally the revision number. The commit hash can be passed to /api/v0/blueprints/diff/ to retrieve the exact changes.
Example:
{ "errors": [], "limit": 20, "offset": 0, "blueprints": [ { "changes": [ { "commit": "e083921a7ed1cf2eec91ad12b9ad1e70ef3470be", "message": "blueprint glusterfs, version 0.0.6 saved.", "revision": null, "timestamp": "2017-11-23T00:18:13Z" }, { "commit": "cee5f4c20fc33ea4d54bfecf56f4ad41ad15f4f3", "message": "blueprint glusterfs, version 0.0.5 saved.", "revision": null, "timestamp": "2017-11-11T01:00:28Z" }, { "commit": "29b492f26ed35d80800b536623bafc51e2f0eff2", "message": "blueprint glusterfs, version 0.0.4 saved.", "revision": null, "timestamp": "2017-11-11T00:28:30Z" }, { "commit": "03374adbf080fe34f5c6c29f2e49cc2b86958bf2", "message": "blueprint glusterfs, version 0.0.3 saved.", "revision": null, "timestamp": "2017-11-10T23:15:52Z" }, { "commit": "0e08ecbb708675bfabc82952599a1712a843779d", "message": "blueprint glusterfs, version 0.0.2 saved.", "revision": null, "timestamp": "2017-11-10T23:14:56Z" }, { "commit": "3e11eb87a63d289662cba4b1804a0947a6843379", "message": "blueprint glusterfs, version 0.0.1 saved.", "revision": null, "timestamp": "2017-11-08T00:02:47Z" } ], "name": "glusterfs", "total": 6 } ] }
POST /api/v0/blueprints/new¶
Create a new blueprint, or update an existing blueprint. This supports both JSON and TOML for the blueprint format. The blueprint should be in the body of the request with the Content-Type header set to either application/json or text/x-toml.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
DELETE /api/v0/blueprints/delete/<blueprint_name>¶
Delete a blueprint. The blueprint is deleted from the branch, and will no longer be listed by the list route. A blueprint can be undeleted using the undo route to revert to a previous commit.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
POST /api/v0/blueprints/workspace¶
Write a blueprint to the temporary workspace. This works exactly the same as new except that it does not create a commit. JSON and TOML bodies are supported.
The workspace is meant to be used as a temporary blueprint storage for clients. It will be read by the info and diff routes if it is different from the most recent commit.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
DELETE /api/v0/blueprints/workspace/<blueprint_name>¶
Remove the temporary workspace copy of a blueprint. The info route will now return the most recent commit of the blueprint. Any changes that were in the workspace will be lost.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
POST /api/v0/blueprints/undo/<blueprint_name>/<commit>¶
This will revert the blueprint to a previous commit. The commit hash from the changes route can be used in this request.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
POST /api/v0/blueprints/tag/<blueprint_name>¶
Tag a blueprint as a new release. This uses git tags with a special format. refs/tags/<branch>/<filename>/r<revision>. Only the most recent blueprint commit can be tagged. Revisions start at 1 and increment for each new tag (per-blueprint). If the commit has already been tagged it will return false.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
/api/v0/blueprints/diff/<blueprint_name>/<from_commit>/<to_commit>¶
Return the differences between two commits, or the workspace. The commit hash from the changes response can be used here, or several special strings:
- NEWEST will select the newest git commit. This works for from_commit or to_commit
- WORKSPACE will select the workspace copy. This can only be used in to_commit
eg. /api/v0/blueprints/diff/glusterfs/NEWEST/WORKSPACE will return the differences between the most recent git commit and the contents of the workspace.
Each entry in the response’s diff object contains the old blueprint value and the new one. If old is null and new is set, then it was added. If new is null and old is set, then it was removed. If both are set, then it was changed.
The old/new entries will have the name of the blueprint field that was changed. This can be one of: Name, Description, Version, Module, or Package. The contents for these will be the old/new values for them.
In the example below the version was changed and the ping package was added.
Example:
{ "diff": [ { "new": { "Version": "0.0.6" }, "old": { "Version": "0.0.5" } }, { "new": { "Package": { "name": "ping", "version": "3.2.1" } }, "old": null } ] }
/api/v0/blueprints/freeze/<blueprint_names>¶
Return a JSON representation of the blueprint with the package and module versions set to the exact versions chosen by depsolving the blueprint.
Example:
{ "errors": [], "blueprints": [ { "blueprint": { "description": "An example GlusterFS server with samba", "modules": [ { "name": "glusterfs", "version": "3.8.4-18.4.el7.x86_64" }, { "name": "glusterfs-cli", "version": "3.8.4-18.4.el7.x86_64" } ], "name": "glusterfs", "packages": [ { "name": "ping", "version": "2:3.2.1-2.el7.noarch" }, { "name": "samba", "version": "4.6.2-8.el7.x86_64" } ], "version": "0.0.6" } } ] }
/api/v0/blueprints/depsolve/<blueprint_names>¶
Depsolve the blueprint using yum, return the blueprint used, and the NEVRAs of the packages chosen to satisfy the blueprint’s requirements. The response will include a list of results, with the full dependency list in dependencies, the NEVRAs for the blueprint’s direct modules and packages in modules, and any error will be in errors.
Example:
{ "errors": [], "blueprints": [ { "dependencies": [ { "arch": "noarch", "epoch": "0", "name": "2ping", "release": "2.el7", "version": "3.2.1" }, { "arch": "x86_64", "epoch": "0", "name": "acl", "release": "12.el7", "version": "2.2.51" }, { "arch": "x86_64", "epoch": "0", "name": "audit-libs", "release": "3.el7", "version": "2.7.6" }, { "arch": "x86_64", "epoch": "0", "name": "avahi-libs", "release": "17.el7", "version": "0.6.31" }, ... ], "modules": [ { "arch": "noarch", "epoch": "0", "name": "2ping", "release": "2.el7", "version": "3.2.1" }, { "arch": "x86_64", "epoch": "0", "name": "glusterfs", "release": "18.4.el7", "version": "3.8.4" }, ... ], "blueprint": { "description": "An example GlusterFS server with samba", "modules": [ { "name": "glusterfs", "version": "3.7.*" }, ... } } ] }
/api/v0/projects/list[?offset=0&limit=20]¶
List all of the available projects. By default this returns the first 20 items, but this can be changed by setting the offset and limit arguments.
Example:
{ "limit": 20, "offset": 0, "projects": [ { "description": "0 A.D. (pronounced "zero ey-dee") is a ...", "homepage": "http://play0ad.com", "name": "0ad", "summary": "Cross-Platform RTS Game of Ancient Warfare", "upstream_vcs": "UPSTREAM_VCS" }, ... ], "total": 21770 }
/api/v0/projects/info/<project_names>¶
Return information about the comma-separated list of projects. It includes the description of the package along with the list of available builds.
Example:
{ "projects": [ { "builds": [ { "arch": "x86_64", "build_config_ref": "BUILD_CONFIG_REF", "build_env_ref": "BUILD_ENV_REF", "build_time": "2017-03-01T08:39:23", "changelog": "- restore incremental backups correctly, files ...", "epoch": "2", "metadata": {}, "release": "32.el7", "source": { "license": "GPLv3+", "metadata": {}, "source_ref": "SOURCE_REF", "version": "1.26" } } ], "description": "The GNU tar program saves many ...", "homepage": "http://www.gnu.org/software/tar/", "name": "tar", "summary": "A GNU file archiving program", "upstream_vcs": "UPSTREAM_VCS" } ] }
/api/v0/projects/depsolve/<project_names>¶
Depsolve the comma-separated list of projects and return the list of NEVRAs needed to satisfy the request.
Example:
{ "projects": [ { "arch": "noarch", "epoch": "0", "name": "basesystem", "release": "7.el7", "version": "10.0" }, { "arch": "x86_64", "epoch": "0", "name": "bash", "release": "28.el7", "version": "4.2.46" }, { "arch": "x86_64", "epoch": "0", "name": "filesystem", "release": "21.el7", "version": "3.2" }, ... ] }
/api/v0/projects/source/list¶
Return the list of repositories used for depsolving and installing packages.
Example:
{ "sources": [ "fedora", "fedora-cisco-openh264", "fedora-updates-testing", "fedora-updates" ] }
/api/v0/projects/source/info/<source-names>¶
Return information about the comma-separated list of source names. Or all of the sources if ‘*’ is passed. Note that general globbing is not supported, only ‘*’.
immutable system sources will have the “system” field set to true. User added sources will have it set to false. System sources cannot be changed or deleted.
Example:
{ "errors": [], "sources": { "fedora": { "check_gpg": true, "check_ssl": true, "gpgkey_urls": [ "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64" ], "name": "fedora", "proxy": "http://proxy.brianlane.com:8123", "system": true, "type": "yum-metalink", "url": "https://mirrors.fedoraproject.org/metalink?repo=fedora-28&arch=x86_64" } } }
POST /api/v0/projects/source/new¶
Add (or change) a source for use when depsolving blueprints and composing images.
The
proxy
andgpgkey_urls
entries are optional. All of the others are required. The supported types for the urls are:
yum-baseurl
is a URL to a yum repository.yum-mirrorlist
is a URL for a mirrorlist.yum-metalink
is a URL for a metalink.If
check_ssl
is true the https certificates must be valid. If they are self-signed you can either set this to false, or add your Certificate Authority to the host system.If
check_gpg
is true the GPG key must either be installed on the host system, orgpgkey_urls
should point to it.You can edit an existing source (other than system sources), by doing a POST of the new version of the source. It will overwrite the previous one.
Example:
{ "name": "custom-source-1", "url": "https://url/path/to/repository/", "type": "yum-baseurl", "check_ssl": true, "check_gpg": true, "gpgkey_urls": [ "https://url/path/to/gpg-key" ] }
DELETE /api/v0/projects/source/delete/<source-name>¶
Delete a user added source. This will fail if a system source is passed to it.
The response will be a status response with status set to true, or an error response with it set to false and an error message included.
/api/v0/modules/list[?offset=0&limit=20]¶
Return a list of all of the available modules. This includes the name and the group_type, which is always “rpm” for lorax-composer. By default this returns the first 20 items. This can be changed by setting the offset and limit arguments.
Example:
{ "limit": 20, "modules": [ { "group_type": "rpm", "name": "0ad" }, { "group_type": "rpm", "name": "0ad-data" }, { "group_type": "rpm", "name": "0install" }, { "group_type": "rpm", "name": "2048-cli" }, ... ] "total": 21770 }
/api/v0/modules/list/<module_names>[?offset=0&limit=20]¶
Return the list of comma-separated modules. Output is the same as /modules/list
Example:
{ "limit": 20, "modules": [ { "group_type": "rpm", "name": "tar" } ], "offset": 0, "total": 1 }
/api/v0/modules/info/<module_names>¶
Return the module’s dependencies, and the information about the module.
Example:
{ "modules": [ { "dependencies": [ { "arch": "noarch", "epoch": "0", "name": "basesystem", "release": "7.el7", "version": "10.0" }, { "arch": "x86_64", "epoch": "0", "name": "bash", "release": "28.el7", "version": "4.2.46" }, ... ], "description": "The GNU tar program saves ...", "homepage": "http://www.gnu.org/software/tar/", "name": "tar", "summary": "A GNU file archiving program", "upstream_vcs": "UPSTREAM_VCS" } ] }
POST /api/v0/compose¶
Start a compose. The content type should be ‘application/json’ and the body of the POST should look like this:
{ "blueprint_name": "http-server", "compose_type": "tar", "branch": "master" }Pass it the name of the blueprint, the type of output (from ‘/api/v0/compose/types’), and the blueprint branch to use. ‘branch’ is optional and will default to master. It will create a new build and add it to the queue. It returns the build uuid and a status if it succeeds:
{ "build_id": "e6fa6db4-9c81-4b70-870f-a697ca405cdf", "status": true }
/api/v0/compose/types¶
Returns the list of supported output types that are valid for use with ‘POST /api/v0/compose’
- {
- “types”: [
- {
- “enabled”: true, “name”: “tar”
}
]
}
/api/v0/compose/queue¶
Return the status of the build queue. It includes information about the builds waiting, and the build that is running.
Example:
{ "new": [ { "id": "45502a6d-06e8-48a5-a215-2b4174b3614b", "blueprint": "glusterfs", "queue_status": "WAITING", "job_created": 1517362647.4570868, "version": "0.0.6" }, { "id": "6d292bd0-bec7-4825-8d7d-41ef9c3e4b73", "blueprint": "kubernetes", "queue_status": "WAITING", "job_created": 1517362659.0034983, "version": "0.0.1" } ], "run": [ { "id": "745712b2-96db-44c0-8014-fe925c35e795", "blueprint": "glusterfs", "queue_status": "RUNNING", "job_created": 1517362633.7965999, "job_started": 1517362633.8001345, "version": "0.0.6" } ] }
/api/v0/compose/finished¶
Return the details on all of the finished composes on the system.
Example:
{ "finished": [ { "id": "70b84195-9817-4b8a-af92-45e380f39894", "blueprint": "glusterfs", "queue_status": "FINISHED", "job_created": 1517351003.8210032, "job_started": 1517351003.8230415, "job_finished": 1517359234.1003145, "version": "0.0.6" }, { "id": "e695affd-397f-4af9-9022-add2636e7459", "blueprint": "glusterfs", "queue_status": "FINISHED", "job_created": 1517362289.7193348, "job_started": 1517362289.9751132, "job_finished": 1517363500.1234567, "version": "0.0.6" } ] }
/api/v0/compose/failed¶
Return the details on all of the failed composes on the system.
Example:
{ "failed": [ { "id": "8c8435ef-d6bd-4c68-9bf1-a2ef832e6b1a", "blueprint": "http-server", "queue_status": "FAILED", "job_created": 1517523249.9301329, "job_started": 1517523249.9314211, "job_finished": 1517523255.5623411, "version": "0.0.2" } ] }
/api/v0/compose/status/<uuids>¶
Return the details for each of the comma-separated list of uuids.
Example:
{ "uuids": [ { "id": "8c8435ef-d6bd-4c68-9bf1-a2ef832e6b1a", "blueprint": "http-server", "queue_status": "FINISHED", "job_created": 1517523644.2384307, "job_started": 1517523644.2551234, "job_finished": 1517523689.9864314, "version": "0.0.2" }, { "id": "45502a6d-06e8-48a5-a215-2b4174b3614b", "blueprint": "glusterfs", "queue_status": "FINISHED", "job_created": 1517363442.188399, "job_started": 1517363442.325324, "job_finished": 1517363451.653621, "version": "0.0.6" } ] }
DELETE /api/v0/blueprints/cancel/<uuid>¶
Cancel the build, if it is not finished, and delete the results. It will return a status of True if it is successful.
Example:
{ "status": true, "uuid": "03397f8d-acff-4cdb-bd31-f629b7a948f5" }
DELETE /api/v0/compose/delete/<uuids>¶
Delete the list of comma-separated uuids from the compose results.
Example:
{ "errors": [], "uuids": [ { "status": true, "uuid": "ae1bf7e3-7f16-4c9f-b36e-3726a1093fd0" } ] }
/api/v0/compose/info/<uuid>¶
Get detailed information about the compose. The returned JSON string will contain the following information:
- id - The uuid of the comoposition
- config - containing the configuration settings used to run Anaconda
- blueprint - The depsolved blueprint used to generate the kickstart
- commit - The (local) git commit hash for the blueprint used
- deps - The NEVRA of all of the dependencies used in the composition
- compose_type - The type of output generated (tar, iso, etc.)
- queue_status - The final status of the composition (FINISHED or FAILED)
Example:
{ "commit": "7078e521a54b12eae31c3fd028680da7a0815a4d", "compose_type": "tar", "config": { "anaconda_args": "", "armplatform": "", "compress_args": [], "compression": "xz", "image_name": "root.tar.xz", ... }, "deps": { "packages": [ { "arch": "x86_64", "epoch": "0", "name": "acl", "release": "14.el7", "version": "2.2.51" } ] }, "id": "c30b7d80-523b-4a23-ad52-61b799739ce8", "queue_status": "FINISHED", "blueprint": { "description": "An example kubernetes master", ... } }
/api/v0/compose/metadata/<uuid>¶
Returns a .tar of the metadata used for the build. This includes all the information needed to reproduce the build, including the final kickstart populated with repository and package NEVRA.
The mime type is set to ‘application/x-tar’ and the filename is set to UUID-metadata.tar
The .tar is uncompressed, but is not large.
/api/v0/compose/results/<uuid>¶
Returns a .tar of the metadata, logs, and output image of the build. This includes all the information needed to reproduce the build, including the final kickstart populated with repository and package NEVRA. The output image is already in compressed form so the returned tar is not compressed.
The mime type is set to ‘application/x-tar’ and the filename is set to UUID.tar
/api/v0/compose/logs/<uuid>¶
Returns a .tar of the anaconda build logs. The tar is not compressed, but is not large.
The mime type is set to ‘application/x-tar’ and the filename is set to UUID-logs.tar
/api/v0/compose/image/<uuid>¶
Returns the output image from the build. The filename is set to the filename from the build with the UUID as a prefix. eg. UUID-root.tar.xz or UUID-boot.iso.
/api/v0/compose/log/<uuid>[?size=kbytes]¶
Returns the end of the anaconda.log. The size parameter is optional and defaults to 1Mbytes if it is not included. The returned data is raw text from the end of the logfile, starting on a line boundry.
Example:
12:59:24,222 INFO anaconda: Running Thread: AnaConfigurationThread (140629395244800) 12:59:24,223 INFO anaconda: Configuring installed system 12:59:24,912 INFO anaconda: Configuring installed system 12:59:24,912 INFO anaconda: Creating users 12:59:24,913 INFO anaconda: Clearing libuser.conf at /tmp/libuser.Dyy8Gj 12:59:25,154 INFO anaconda: Creating users 12:59:25,155 INFO anaconda: Configuring addons 12:59:25,155 INFO anaconda: Configuring addons 12:59:25,155 INFO anaconda: Generating initramfs 12:59:49,467 INFO anaconda: Generating initramfs 12:59:49,467 INFO anaconda: Running post-installation scripts 12:59:49,467 INFO anaconda: Running kickstart %%post script(s) 12:59:50,782 INFO anaconda: All kickstart %%post script(s) have been run 12:59:50,782 INFO anaconda: Running post-installation scripts 12:59:50,784 INFO anaconda: Thread Done: AnaConfigurationThread (140629395244800)
pylorax.api.workspace module¶
-
pylorax.api.workspace.
workspace_delete
(repo, branch, recipe_name)[source]¶ Delete the recipe from the workspace
Parameters: Returns: None
Raises: IO related errors
-
pylorax.api.workspace.
workspace_dir
(repo, branch)[source]¶ Create the workspace’s path from a Repository and branch
Parameters: - repo (Git.Repository) – Open repository
- branch (str) – Branch name
Returns: The path to the branch’s workspace directory
Return type: