184 lines
6.3 KiB
Plaintext
184 lines
6.3 KiB
Plaintext
Design
|
|
========
|
|
|
|
After cgroup system has taken shape, its time to have some basic tools
|
|
in user space which can enable a user to use the resource management
|
|
functionality effictively.
|
|
|
|
One of the needed functionality is rule based placement of a task. In general,
|
|
there can be either uid or gid or exec based rules. Admin/root will
|
|
control/enforce uid/gid based rules and exec based rules can be defined by
|
|
user in a config file residing in user's home dir and fully controlled by user.
|
|
|
|
uid/gid based rules will be defined in /etc/cgrules.conf config file and
|
|
this file will be managed by root.
|
|
|
|
Basic idea is that to begin with provide facility to implement rules
|
|
based on uid and gid. So a hierarchy might look like as follows.
|
|
|
|
/mnt/cgroup
|
|
| |
|
|
gid1 gid2
|
|
| |
|
|
uid1 uid2
|
|
| |
|
|
proj1 proj2
|
|
|
|
|
|
Admin will write rules to control the resources among users. Then users
|
|
can manage their own cgroup at their own (proj1 and proj2) and control
|
|
the resources as they want.
|
|
|
|
Following are the few methods using which tasks can be placed in right
|
|
cgroups.
|
|
|
|
- Use pam_cgroup PAM plugin which will make sure users are placed in right
|
|
cgroup at login time and any tasks launch after login, will continue to run
|
|
in user's cgroup.
|
|
|
|
- Use command line tool "cgexec" to launch the task in right cgroup.
|
|
|
|
- Modify the program and use libcgroup provided APIs for placing a task
|
|
in right cgroup before doing exec().
|
|
|
|
- Use "cgclassify" tool to classify a already running task.
|
|
|
|
- May be, a user space daemon can also be implemented which will listen to
|
|
kernel events and place the task in right group based on the rules.
|
|
This method involves few concerns.
|
|
|
|
- Reliability of netlink socket. Messages can be dropped.
|
|
- Change the netlink with a cgroup controller which exports the
|
|
events.
|
|
|
|
- Delay incurred since the event took place and task was actually placed
|
|
in right cgroup.
|
|
|
|
- daemon will interfere with container's tasks which is not desired.
|
|
|
|
HOWTO
|
|
=====
|
|
|
|
Section 1:
|
|
----------
|
|
To use "cgexec" to place the task in right cgroup.
|
|
|
|
- make cgexec
|
|
- Run a task using cgexec. Following is the cgexec syntax.
|
|
|
|
cgexec [-g <list of controllers>:<relative path to cgroup>] command [arguments]
|
|
|
|
Note: Cgroup controllers and path are optional. If user does not know the
|
|
right cgroup, cgexec will automatically place the task in right
|
|
cgroup based on /etc/cgrules.conf
|
|
|
|
Example:
|
|
cgexec -g *:test1 ls
|
|
cgexec -g cpu,memory:test1 ls -l
|
|
cgexec -g cpu,memory:test1 -g swap:test2 ls -l
|
|
|
|
Section 2
|
|
---------
|
|
To use "cgclassify" to place task in right cgroup.
|
|
|
|
- make cgclassify
|
|
- Pick a task's pid to be classified, and run
|
|
cgclassify <list of pids>
|
|
|
|
Example:
|
|
--------
|
|
cgclassify 2140 4325
|
|
|
|
Note: After classification check out whether tasks 2140 and 4325
|
|
are in the right cgroup or not (Based on rules in /etc/cgrules.conf)
|
|
|
|
Section 3:
|
|
----------
|
|
To use a pam plugin which will automatically place the task in right
|
|
cgroup upon login.
|
|
|
|
- Build pam_cgroup.so
|
|
make pam_cgroup.so
|
|
- Copy pam_cgroup.so to /lib/security/
|
|
- Edit /etc/pam.d/su to make use of pam_cgroup.so session module upon
|
|
execution of su.
|
|
|
|
example:
|
|
Add following line at the end of /etc/pam.d/su file
|
|
|
|
session optional pam_cgroup.so
|
|
|
|
- Now launch a shell for a user "xyz" using su and the resulting shell
|
|
should be running in the cgroup designated for the user as specified
|
|
by cgrules.conf
|
|
|
|
ex. "su test1"
|
|
|
|
Try similar things with other services like sshd.
|
|
|
|
Note: pam_cgroup.so moves the service providing process in the right cgroup
|
|
and not the process which will be launched later. Due to parent child
|
|
relationship, yet to be forked/execed process will launch in right
|
|
group.
|
|
|
|
Ex. Lets say user root does "su test1". In this case process "su" is the
|
|
one providing service (launching a shell) for user "test1". pam_cgroup.so
|
|
will move process "su" to the user "test1"'s cgroup (Decided by the uid
|
|
and gid of "test1"). Now once su forks/execs a shell for user test1,
|
|
final shell is effectively running in the cgroup it should have been
|
|
running based on /etc/cgrules.conf for user test1.
|
|
|
|
|
|
Section 4:
|
|
----------
|
|
To use cgrulesengd which will move a task to right cgroup based on
|
|
rules in /etc/cgrules.conf do following.
|
|
|
|
- build and install latest libcgroup.so
|
|
- build cgrulesengd
|
|
make cgrulesengd
|
|
- specify some uid/gid based rules in /etc/cgrules.conf
|
|
- Mount some controllers and create an hierarchy of cgroups (matching
|
|
your rules).
|
|
- Run cgrulesengd.
|
|
- ./cgrulesengd
|
|
- Launch some task or login as a user to the sytem. Daemon should automatically
|
|
place the task in right cgroup.
|
|
|
|
FAQ
|
|
===
|
|
Q. Why admin should not control "exec" based rules.
|
|
A. Unix file system provides access control only based on uid/gid. So
|
|
even if admin starts putting tasks based on uid/gid, it can't enforce
|
|
it. For example, consider following scenario.
|
|
|
|
Lets say an admin creates following cgroup hierarchy.
|
|
|
|
/container
|
|
| |
|
|
database browser
|
|
| | | |
|
|
user1 user2 user1 user2
|
|
|
|
Now admin wants to run all the database jobs under /container/database/
|
|
and all the firefox jobs under /container/browser/. Based on which user
|
|
launched it, jobs should go in either user1 or user2 dir.
|
|
|
|
Now assume that database subdir has to more cpu resources as compared to
|
|
browser subdir. Then a user, say user2, can always move his browser job
|
|
also to /container/database/user2 dir to get more cpu resources and
|
|
admin will not be able to control that.
|
|
|
|
[Note: user2 will control what tasks can be added in /container/database/user2
|
|
and will contol what further subdirs can be created under user2 dir. Root
|
|
should not restrict the control to root only for practical purposes. Its
|
|
something like that till /container/databse, admin controls the resources
|
|
and below that how resources are further subdivided among various projects
|
|
should be controlled by respective user].
|
|
|
|
In the light of above, it seems to make more sense that admin should enforce
|
|
rules only based on uid and gid. Probably later we can have a per user exec
|
|
based rules config file (~/.cgrules.conf), which can be parsed by cgrulesd
|
|
and then jobs launched by user will be placed in right cgroup based on
|
|
combination of rules in /etc/cgrules.conf and ~/cgrules.conf.
|