Logging for runit on Void Linux with svlogger
Posted by Dave Eddy on Sep 26 2018 - tags: techUpdate 10/01/2018
svlogger
can be easily replicated with vlogger(8), which is shipped
with void-runit. This comment from
Duncaen on reddit explains it:
I wrote vlogger(8) which works similar, by default it logs to syslog and if
/etc/vlogger
is an executable file it will exec into/etc/vlogger
with the service name as argument.The script would be something like this:
#!/bin/sh exec svlogd /var/log/$1
The main idea is to make it simple to switch from logging to syslog to svlogd by just creating
/etc/vlogger
. Its part of therunit-void
package and services could be shipped with this as default at some point.
I created /etc/vlogger
as an executable with the following contents:
#!/bin/sh
logdir=/tmp/sv/log/$1
mkdir -p "$logdir"
exec svlogd -tt "$logdir"
Updated my existing log services with:
cd /var/service
for f in */log/run; do sudo ln -svf /usr/bin/vlogger "$f"; done
And restarted them with:
sudo sv restart /var/service/*/log
The rest of the post is left here for historical purposes.
I’ve been using Void Linux on my laptop for the last month or
so and it has just been fantastic; using runit to manage services has
been so incredibly easy. I wrote vsv to make the output a little more
colorful and the usage slightly easier, but even just using sv
directly has
just been great.
Coming from illumos, I’m used to having SMF (the Service Management
Facility) to manage services and daemons. On Void, runit
is way more
bare bones and lightweight. One thing I really wanted that I had easily with
SMF was logging. I didn’t need anything crazy or complicated, I just wanted to
be able to quickly see what a daemon was doing and if it was experiencing any
issues.
By default, most services under runit have their stdout
discarded and their
stderr
sent to the process title of the runsvdir
process controlling them.
In my opinion, having logs sent to the process title of a running program is a
really cool way of doing it, but I wanted something slightly more persistent
and more consistent than parsing ps
output.
I looked into running a syslogging daemon with runit, and it was surprisingly
easy using socklog, but the socklog
daemon didn’t solve
the actual problem that I wanted to solve - having each service log to their
own log file.
With that, svlogger was born.
svlogger
GitHub Repo: https://github.com/bahamas10/svlogger
svlogger
is a generic svlogd
wrapper for runit services that is meant to be
symlinked to /.../<service-name>/log/run
. This script will automatically
figure out the service name under which it is running and write out logs to a
file under a directory with that service name using svlogd
.
svlogger
is a standalone bash script that can be put anywhere on your system.
I personally use this script by installing it to /etc/sv
and symlinking it to
each service’s log directory as the run
script. You can move the script into
place with:
$ sudo make install
cp svlogger /etc/sv
And then, to enable logs for a single service:
# mkdir -p /var/service/my-service/log
# ln -s /etc/sv/svlogger /var/service/my-service/log/run
Within 5 seconds, runit
will start the logger service and svlogd
will write
logs to /tmp/sv/log/my-service
:
$ tail /tmp/sv/log/my-service/current
2018-09-19_21:52:51.30654 starting...
2018-09-19_21:52:52.30754 tick
2018-09-19_21:52:53.30926 tick
2018-09-19_21:52:54.31068 tick
To enable logs for all active services that don’t currently have a logger:
cd /var/service
for d in */; do
sudo mkdir -p "$d/log"
[[ -e $d/log/run ]] || sudo ln -s /etc/sv/svlogger "$d/log/run"
done
You can then view all of the logs generated with:
# tail -f /tmp/sv/log/*/current
...
I personally choose to write the logs to /tmp/sv
since /tmp
is a
filesystem that lives in ram. This way, all logs will be wiped on a system
reboot and they will not be unnecessarily stressing my system drive.
To log to a directory other than /tmp/sv
, you can edit the script or pass a
directory name as the first argument. If you choose to pass a directory name as
an argument, you cannot symlink svlogger
and instead will need to create a
small wrapper script for svlogger
. For example:
/etc/sv/my-service/log/run
#!/bin/sh
exec /etc/sv/svlogger /var/log/sv
This will cause my-service
to output its logs to the newly created
/var/log/sv/my-service
directory.
Finally, with runit
, stdout
will go to the logger program (if it is set) and
stderr
will show up in the command line of the runsvdir
program. To log
stderr
to the log file as well, you need to redirect stderr
to stdout
in
your service’s run
script. You can add an exec
line like this to the top of
your service’s run
script:
#!/bin/sh
exec 2>&1
...
vsv Integration
I added support for log services to vsv
with version v1.2.0
.
-l
can be specified to show log services alongside the services themselves:
-t
can be specified with -l
to see the process tree as well:
Conclusion
vsv
makes it easy to manage the log services, but, in my experience, the log
services just work on their own for the most part and don’t need to be modified
very often, if at all. svlogger
execs svlogd
, and that process handles log
rotation automatically, restricting each service to 10mb max for all of the log
files generated.
All logs for the services controlled by runit
can be viewed with:
$ tail -f /tmp/sv/log/*/current
...