2021.06.17 - Using systemd-run to Manage Long-Running Process Groups¶

In my last blog entry 2021.06.04 - Using Other Window Managers with Plasma 5.21 and systemd Startup I provided and described how you can use a simple script to launch applications in their own transient systemd scope.

After using that script in my desktop workflow to launch desktop applications I realized I could, with some minor modifications to the script, also use it to launch large multi-process programs on the shell; and so have them also be organized in a transient systemd scope. This allows for some nice ways to simply address the whole group of processes in systemd, and manage them with the commands that offers.

You can find the script I’m referring to in section Starting Child Porcesses in app.slice with i3. In the blog I suggest using the name dmenu_run_systemd for it, but that’s just in the context of using it as replacement for dmenu_run, so name it as whatever you like, or create a shell alias for it.

Since writing the initial version, I extended the script with two command line options when run in its second form - with an explicit command given, so that dmenu is not called:

-f, --forground: Start <command> as foreground task (default: no);

-p, --pwd: Use the current ${PWD} as working directory (default: ${HOME}).

I added these, rather than changing the default, so I can use the script in the same way as before, but also easily create an alias to use it in the way I want for easily starting programs in transient systemd scopes. Simply add something along the lines of this to your shell rc (the example is for bash):

alias srun="dmenu_run_systemd -f -p --".

Now I have a simple command: srun; that I can use to start a foreground process in its own slice in my user-sessions. Neat!

Example: Using `srun` to Control Gentoo’s `emerge`¶

Lets look at an example. On my Gentoo system, I want to install a new version off GCC. In Gentoo this means it is compile, and that takes a while, and spawns multiple parallel sub-processes (making use of my multi-core CPU). Here is the command line I use:

srun emerge -aD sys-devel/gcc

Now I can use systemctl to work with the created scope: systemctl --user status xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope.

● xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope - dmenu selection 'emerge' '-aD' 'sys-devel/gcc'
     Loaded: loaded (/run/user/0/systemd/transient/xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope; transient)
  Transient: yes
     Active: active (running) since Thu 2021-06-17 21:26:11 CEST; 2min 52s ago
      Tasks: 1 (limit: 18470)
     Memory: 387.2M
        CPU: 30.586s
     CGroup: /user.slice/user-0.slice/user@0.service/app.slice/xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope
             └─360019 /usr/bin/pypy3 -b /usr/lib/python-exec/pypy3/emerge -aD sys-devel/gcc

Aaa 00 00:00:00 aaaa systemd[3115]: Started dmenu selection 'emerge' '-aD' 'sys-devel/gcc'.

Note that systemctl usually offers TAB completion, so you don’t need to search all that much for the scope, try just starting to type the unit with xrun-, and the use TAB completion for easily finding the scope you want (assuming you don’t crazy amounts).

The output above is from the very beginning of the process. When I take a look later I can see a lot more going on.

● xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope - dmenu selection 'emerge' '-aD' 'sys-devel/gcc'
     Loaded: loaded (/run/user/0/systemd/transient/xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope; transient)
  Transient: yes
     Active: active (running) since Thu 2021-06-17 21:26:11 CEST; 12min ago
      Tasks: 15 (limit: 18470)
     Memory: 1.6G
        CPU: 2min 34.447s
     CGroup: /user.slice/user-0.slice/user@0.service/app.slice/xrun-emerge_-aD_sys-devel_gcc-E4SRSGQSHIBUTQ5QRQWLGAGJIUDVYBZ3.scope
             ├─360019 /usr/bin/pypy3 -b /usr/lib/python-exec/pypy3/emerge -aD sys-devel/gcc
             ├─364476 /usr/lib/pypy3.7/pypy3-c-7.3.4 /usr/lib/portage/pypy3/pid-ns-init 364477
             ├─364477 /usr/lib/pypy3.7/pypy3-c-7.3.4 /usr/lib/portage/pypy3/pid-ns-init 250 250 250 18 0,1,2 /usr/bin/sandbox [sys-devel/gcc-11.1.0-r1] sandbox /usr/lib/portage/pypy3/ebuild.sh compile
             ├─364480 [sys-devel/gcc-11.1.0-r1] sandbox /usr/lib/portage/pypy3/ebuild.sh compile
             ├─364481 /bin/bash /usr/lib/portage/pypy3/ebuild.sh compile
             ├─364497 /bin/bash /usr/lib/portage/pypy3/ebuild.sh compile
             ├─364514 /bin/bash /usr/lib/portage/pypy3/ebuild-helpers/emake LDFLAGS=-Wl,-O1 -Wl,--as-needed STAGE1_CFLAGS= LIBPATH=/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0 BOOT_CFLAGS=-m64 -march=broadwell -mtune=broadwell -pipe -O2 boots ...
             ├─364516 make -j9 -l5 LDFLAGS=-Wl,-O1 -Wl,--as-needed STAGE1_CFLAGS= LIBPATH=/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0 BOOT_CFLAGS=-m64 -march=broadwell -mtune=broadwell -pipe -O2 bootstrap-lean
             ├─364526 make DESTDIR= RPATH_ENVVAR=LD_LIBRARY_PATH TARGET_SUBDIR=x86_64-pc-linux-gnu bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/11.1.0 datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/11.1.0 exec_prefix=/usr includedir=/usr/lib ...
             ...
             ├─364576 make DESTDIR= RPATH_ENVVAR=LD_LIBRARY_PATH TARGET_SUBDIR=x86_64-pc-linux-gnu bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/11.1.0 datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/11.1.0 exec_prefix=/usr includedir=/usr/lib ...
             ├─384649 /bin/bash /var/tmp/portage/sys-devel/gcc-11.1.0-r1/work/gcc-11.1.0/libcpp/configure --srcdir=/var/tmp/portage/sys-devel/gcc-11.1.0-r1/work/gcc-11.1.0/libcpp --cache-file=./config.cache --with-system-zlib --prefix=/u ...
             ├─387245 /bin/bash /var/tmp/portage/sys-devel/gcc-11.1.0-r1/work/gcc-11.1.0/libcpp/configure --srcdir=/var/tmp/portage/sys-devel/gcc-11.1.0-r1/work/gcc-11.1.0/libcpp --cache-file=./config.cache --with-system-zlib --prefix=/u ...
             ├─387247 x86_64-pc-linux-gnu-gcc -c conftest.c
             └─387248 /usr/libexec/gcc/x86_64-pc-linux-gnu/10.3.0/cc1 -quiet conftest.c -quiet -dumpbase conftest.c -mtune=generic -march=x86-64 -auxbase conftest -o /var/tmp/portage/sys-devel/gcc-11.1.0-r1/temp/cc1sNTPA.s

Aaa 00 00:00:00 aaaa systemd[3115]: Started dmenu selection 'emerge' '-aD' 'sys-devel/gcc'.

And now I can also use other operations that systemd offers. For example, I can freeze the scope:

systemctl --user freeze xrun-emerge_-aD_sys-devel_gcc-<...>.scope

All the processes in the scope are stopped, using Linux process control. If I later want to restart the compilation, I can thaw the scope again:

systemctl --user thaw xrun-emerge_-aD_sys-devel_gcc-<...>.scope

I can also dynamically change properties, like those provided via systemd.resource-control(5); CPU weight, IO device weight, … :

systemctl --user set-property xrun-emerge_-aD_sys-devel_gcc-<...>.scope CPUWeight=10

For long running programs with many possible sub-processes this is great, because you get one predictable handle - the name of the transient scope -, and plenty of tools to manage them via the handle - systemctl and such.

Like I said, pretty neat!

mageta.org

Navigation

Related Topics

2021.06.17 - Using systemd-run to Manage Long-Running Process Groups¶

Example: Using `srun` to Control Gentoo’s `emerge`¶

2021.06.17 - Using systemd-run to Manage Long-Running Process Groups¶

Example: Using srun to Control Gentoo’s emerge¶

Example: Using `srun` to Control Gentoo’s `emerge`¶