Commit Graph

1498 Commits

Author SHA1 Message Date
Alexander Todorov
5500ec8c06 Use sudo to run the tests
in 9ed1964 the tests start using makq_queue_dirs() which does
chown with guid/id 0!
2018-06-11 16:54:58 -07:00
Alexander Todorov
98f3b7e7eb Add tests for api.crossdomain.py 2018-06-11 16:54:58 -07:00
Alexander Todorov
1f90455ee6 Add required_methods for decorator
otherwise we can't issue OPTIONS requests to the server, see
first comment at http://flask.pocoo.org/snippets/56/
2018-06-11 16:54:58 -07:00
Alexander Todorov
7f7295e3b0 Convert max_age to int b/c timedelta.total_seconds() is a float 2018-06-11 16:54:58 -07:00
Alexander Todorov
3dd5612732 Fix syntax error caused by conflict resolution 2018-06-11 16:54:58 -07:00
Brian C. Lane
64ce465285 Automatic commit of package [lorax] release [19.7.8-1].
Created by command:

/usr/bin/tito tag
2018-06-11 16:54:58 -07:00
Brian C. Lane
c399447b81 Fix a problem with using a mirror as the primary url
Because of how Anaconda is run it needs to be passed a baseurl (using
--repo on the anaconda cmdline), not a mirrorlist url. This fixes it so
that the first mirror is used if the main repository is using a
mirrorlist.
2018-06-11 16:54:58 -07:00
Brian C. Lane
a1427b1a31 Set the HOME variable to a directory the uid can access
libgit2 will look for $HOME/.gitconfig after dropping root privileges so
this needs to be accessible to the user.
2018-06-11 16:54:58 -07:00
Brian C. Lane
442dc43206 Open the git repo after dropping root privileges
Also make sure the recipe directory and its contents have correct
ownership, and change the default recipe path when using the systemd
service to /var/lib/lorax/composer/recipes/
2018-06-11 16:54:58 -07:00
Brian C. Lane
43fe00c1c2 Create the weldr user in lorax.spec
Only created when lorax-composer is installed, not lorax by itself.
2018-06-11 16:54:58 -07:00
Brian C. Lane
4ae2e1450d Exit on uid/gid errors before checking directory permissions
If the uid/gid check fails there is no sense in trying to examine the
ownership or permissions.
2018-06-11 16:54:58 -07:00
Brian C. Lane
5e075f6102 lorax-composer now requires anaconda-tui 2018-06-11 16:54:58 -07:00
Brian C. Lane
fdecf13299 Add tests for /compose API
Because you cannot share data between test methods these have to all be
in one big function. This adds one series to test the failed compose
results, and a 2nd function to test for the successful compose.
2018-06-11 16:54:58 -07:00
Brian C. Lane
daa5ed2aaf Add documentation for /compose and /compose/types 2018-06-11 16:54:58 -07:00
Brian C. Lane
eb2a7208c8 Move queue monitor startup into a function
This allows it to be used by the test framework.
2018-06-11 16:54:58 -07:00
Brian C. Lane
63a7897457 Move queue directory creation into a function
So that it can also be used during tests
2018-06-11 16:54:58 -07:00
Brian C. Lane
8da6214e10 Add a test mode to /compose
This will allow testing without having a full system setup with
anaconda, if ?test=1 is passed to the POST /compose command it will wait
10 seconds instead of running Anaconda, and then raise an error to
generate a failed build.

Passing ?test=2 will also wait 10 seconds instead of running Anaconda,
but will finish successfully.
2018-06-11 16:54:58 -07:00
Brian C. Lane
2e1e8c20c7 Cleanup docstrings for queue.py 2018-06-11 16:54:58 -07:00
Brian C. Lane
9603c866af Drop cancel_q from the monitor() function
Not needed, builds are canceled by writing a 'CANCEL' file in the results
directory.
2018-06-11 16:54:58 -07:00
Brian C. Lane
166350fcb6 Fix the jsonify calls to use kwargs
No need to pass it a dict, just set the args.
2018-06-11 16:54:58 -07:00
Brian C. Lane
ff7d36bcbc Add /compose/log/ API to retrieve the end of the build log
This allows the client to request the end of the anaconda.log during and
after a build. The amount of data returned can be set by adding
?size=<kbytes>

Output is raw bytes, starting on the next available line boundry.
2018-06-11 16:54:58 -07:00
Brian C. Lane
caee8e7cdf Return a status of false if the uuid isn't valid 2018-06-11 16:54:58 -07:00
Brian C. Lane
072aa720ff Add /compose/cancel API to cancel a running build
If the build hasn't started yet (state is WAITING) try removing the
symlink to it. If this succeeds, delete the partial results directory.

If the build makes it to RUNNING then it writes a CANCEL file in the
results directory. The callback that is passed to execWithRedirect
catches this, causing a SIGTERM to be sent to anaconda. It then exits
and cleanup happens normally. The partial results directory is then
removed.
2018-06-11 16:54:58 -07:00
Brian C. Lane
8f0bca00c0 Pass the callback_func through novirt_install to execWithRedirect
The default is no callback.
2018-06-11 16:54:58 -07:00
Brian C. Lane
f1d8b10867 Add a callback to execWithRedirect
If the callback returns true send the process a SIGTERM to terminate it.
2018-06-11 16:54:58 -07:00
Alexander Todorov
dfe8774616 Update how we pass the source to docker so it includes docs/ dir
since the last commit the tests require to have the docs built
before hand but the docs/ dir was missing from the docker image.
2018-06-11 16:54:58 -07:00
Alexander Todorov
d3d92d13b1 Add tests for functions in api/projects 2018-06-11 16:54:58 -07:00
Alexander Todorov
1c977b79ef Add tests for api/server.py
some of the new tests require that we first build the documentation
2018-06-11 16:54:58 -07:00
Alexander Todorov
32db249538 Add tests for yumbase and update how we inspect boolean options
.get_default() returns string so make sure we're actually parsing
the value as boolean and not evaluating a non-empty string in a
boolean context (which will always return True)
2018-06-11 16:54:58 -07:00
Alexander Todorov
a954cde36a Add new tests for workspace_read() and workspace_delete() 2018-06-11 16:54:58 -07:00
Alexander Todorov
e830d9766b Add new tests for configure() 2018-06-11 16:54:58 -07:00
Alexander Todorov
395c4a4b7f Add more tests for api.recipes
- test against already existing git repository
- test commit_recipe_file() handling of IOError
- update tests for commit_recipe_directory()
  - add asserts on the existing test. Not raising an exception
    isn't enough!
  - add test which exercises the method under test while it handles
    exceptions raised by other methods
- test for list_commits() when the underlying calls fail to convert
  timestamp
- test for find_name() when `name' is not on the list
- tests for get_revision_from_tag()
2018-06-11 16:54:58 -07:00
Brian C. Lane
6a95a314e9 Add API routes for downloading build results
Also fix a bug with the name of the queue status in the status results
(it is now 'queue_status' not 'status' which is used for error
responses).

This adds the following routes:
 - /compose/metadata/<uuid> to retrieve a .tar of the build metadata
 - /compose/results/<uuid> to retrieve .tar of all of the build results
 - /compose/logs/<uuid> to retrieve a .tar of just the logs from the build
 - /compose/image/<uuid> to retrieve the output image from the build
2018-06-11 16:54:58 -07:00
Brian C. Lane
ed03ac7524 Add /compose/info route to retrieve details about a compose
The results is a JSON string with the following information:

* id - The uuid of the comoposition
* config - containing the configuration settings used to run Anaconda
* recipe - The depsolved recipe used to generate the kickstart
* commit - The (local) git commit hash for the recipe used
* deps - The NEVRA of all of the dependencies used in the composition
* compose_type - The type of output generated (tar, iso, etc.)
* queue_status - The final status of the composition (FINISHED or FAILED)
2018-06-11 16:54:58 -07:00
Brian C. Lane
d3a9ec3002 Return the commit id for the recipe being read
This adds returning the commit id from read_commit, and a new function
read_recipe_and_id() that returns the commit id and the recipe in a
tuple.

If the commit is passed in, it is used as is. If no commit is passed in
it finds the most recent commit for the file on the selected branch and
returns that.

Missing recipes now raise a RecipeError with an informative message.
eg. "No commits for missing-recipe.toml on the master branch."
2018-06-11 16:54:58 -07:00
Brian C. Lane
22c03ceba7 Fix yum config directory creation for projects and server tests 2018-06-11 16:54:58 -07:00
Brian C. Lane
b99a7756de Add DELETE /compose/delete/<uuids> API route
This will delete all of the build's results and cannot be undone.
2018-06-11 16:54:58 -07:00
Brian C. Lane
3fdf3a9819 Turn on o+x permission for the queue and results directories
Otherwise it won't allow a group member to delete things.
2018-06-11 16:54:58 -07:00
Brian C. Lane
fdcf839753 Add /compose/status/<uuids> to retrieve details of a specific build 2018-06-11 16:54:58 -07:00
Brian C. Lane
91c6d1525d Add compose status routes /compose/finished and /compose/failed
These will return a list of details about the finished or failed builds.
2018-06-11 16:54:58 -07:00
Brian C. Lane
fd2aa7c79b Automatic commit of package [lorax] release [19.7.7-1].
Created by command:

/usr/bin/tito tag
2018-06-11 16:54:58 -07:00
Brian C. Lane
520177afca Add /compose/queue to get the status of the build queue
This will return a JSON result that includes the waiting builds,
and the running builds.
2018-06-11 16:54:58 -07:00
Brian C. Lane
47f21af7fb Add reading a recipe directly from a file
Adds helper function recipe_from_file to recipes.py
2018-06-11 16:54:58 -07:00
Brian C. Lane
0065cadf2d Include the recipe in the results of a build
Write original as recipe.toml and the depsolved version as frozen.toml

Also write 'WAITING' to the STATUS file as its first state.
The STATUS states are now WAITING -> RUNNING -> FINISHED|FAILED
2018-06-11 16:54:58 -07:00
Brian C. Lane
9fc973366f Move creating a frozen recipe into recipes.py
Also adds .package_names and .module_names properties. Call
recipe.freeze with a list of NEVRA dependencies and it will return a new
Recipe object with all of the packages and modules set to the depsolved
version.
2018-06-11 16:54:58 -07:00
Brian C. Lane
67da4d6971 Add building an image, and the /compose route to start it
This adds the ability to build a tar output image. The /compose and
/compose/types API routes are now available.

To start a build POST a JSON body to /compose, like this:

{"recipe_name":"glusterfs", "compose_type":"tar", "branch":"master"}

This will return a unique build id:

{
  "build_id": "4d13abb6-aa4e-4c80-a671-0b867e6e77f6",
  "status": true
}

which will be used to keep track of the build status (routes for this
do not exist yet).
2018-06-11 16:54:58 -07:00
Brian C. Lane
de9f5b0456 Remove test configuration and read it from the build directory 2018-06-11 16:54:58 -07:00
Brian C. Lane
7372faa969 Add function to return full NEVRA of a dependency 2018-06-11 16:54:58 -07:00
Brian C. Lane
c92373ecd6 Change config and paths
With the API running as weldr the permissions on the yum directories
needs to be accessable to it, as well as the results and queue/new
directories.
2018-06-11 16:54:58 -07:00
Brian C. Lane
e2f4674fb3 Add basic composer queue handling
The queue is in /var/lib/weldr/queue/new by default. It watches the
directory for new symlinks (to /var/lib/weldr/results/<dirname>) and
handles running anaconda on the kickstart found in final-kickstart.ks
inside the symlinked directory.
2018-06-11 16:54:58 -07:00