init

In Unix-based computer operating systems, the init (short for initialization) process is the first process started during the booting of the computer system. It is started by the kernel. The init process is a daemon process that continues running until the system is shut down. It is the direct or indirect ancestor of all other processes and automatically adopts all orphaned processes. The init process is started by the kernel using a hard-coded filename. If the kernel is unable to start the init process, a kernel panic will be issued. The init is typically assigned a process identifier of 1.

systemd

systemd is an init process used by some Linux distributions to run the user space and manage all processes subsequently. There are other processes that are considered to be init processes, like UNIX System V or Berkeley Software Distribution (BSD) init systems.

The name systemd adheres to the Unix convention of naming daemons by appending the letter d.

Useful links:

• Wiki
• freedesktop.org

Goals

The systemd daemon was developed with several goals in mind. This includes:

• Unify basic Linux configurations and service behaviors across all distributions.
• Improve the software framework for expressing dependencies
• Allow more processing to be done concurrently or in parallel during system booting
• To reduce the computational overhead of the shell.

systemd is not just the name of the init daemon, but also refers to the entire software bundle around it, which, in addition to the systemd init daimon, includes the daemons journald, logind and networkd, and many other low-level components. It is a large software suite that includes over 69 individual binaries. As an integrated software suite, systemd replaces the startup sequences and runlevels controlled by the traditional init daemon, along with the shell scripts executed under its control. systemd also integrates many other services that are common on Linux systems by handling user logins, the system console, device hotplugging, scheduled execution (replacing cron), logging, hostnames and locales.

Like the init daemon, systemd is a daemon that manages other daemons, which, including systemd itself, are background processes. systemd is the first daemon to start during booting and the last daemon to terminate during shutdown. The systemd daemon serves as the root of the user space's process tree; the first process (pid 1) has a special role on Unix systems, as it receives a SIGCHLD signal when a daemon process (which has detached from its parent) terminates. Therefore, the first process is particularly well suited for the purpose of monitoring daemons; systemd attempts to improve in that particular area over the traditional approach, which would usually not restart daemons automatically but only launch them once without further monitoring.

systemd executes elements of its startup sequence in parallel, which is faster than the traditional startup sequence's sequential approach. For inter-process communication (IPC), systemd makes Unix domain sockets and D-Bus available to the running daemons. The state of systemd itself can also be preserved in a snapshot for future recall.

Unit Files

systemd records initialization instructions for each daemon in a configuration file (referred to as a "unit file") that uses a declarative language, replacing the traditionally used per-daemon startup shell scripts. Unit file types include service, socket, device, mount, automount, swap, target, path, timer (which can be used as a cron-like job scheduler), snapshot, slice and scope.

Core Components and Libraries

Following its integrated approach, systemd also provides replacements for various daemons and utilities, including the startup shell scripts, pm-utils, inetd, acpid, syslog, watchdog, cron and atd. systemd's core components include the following:

• systemd is a system and service manager for Linux operating systems.
• systemctl may be used to introspect and control the state of the systemd system and service manager.
• systemd-analyze may be used to determine system boot-up performance statistics and retrieve other state and tracing information from the system and service manager.

systemd tracks processes using the Linux kernel's cgroups subsystem instead of using process identifiers (PIDs); thus, daemons cannot "escape" systemd, not even by double-forking. systemd not only uses cgroups, but also augments them with systemd-nspawn and machinectl, two utility programs that facilitate the creation and management of software containers. Since version 205, systemd also offers ControlGroupInterface, which is an API to the Linux kernel cgroups. The Linux kernel cgroups are adapted to support kernfs, and are being modified to support a unified hierarchy.

Ancillary Components

Beside its primary purpose of providing a replacement Linux init system, systemd suite provides additional functionality, including its following components:

systemctl

In systemd, the target of most actions are units, which are resources that systemd knows how to manage. Units are categorized by the type of resource they represent (i.e. service, socket, device, mount, automount, swap, target, path, timer, snapshot, slice and scope) and they are defined with files known as unit files. The type of each unit can be inferred from the suffix on the end of the file.

For service management tasks, the target unit will be service units, which have unit files with a suffix of .service. However, for most service management commands, you can actually leave off the .service suffix, as systemd is smart enough to know that you probably want to operate on a service when using service management commands.

Starting and Stopping Services

To start a systemd service, executing instructions in the service's unit file, use the start command. If you are running as a non-root user, you will have to use sudo since this will affect the state of the operating system:

sudo systemctl start application[.service]

As it was mentioned above, systemd knows to look for *.service files for service management commands, so the command could just as easily be typed like this:

sudo systemctl start application

To stop a currently running service, you can use the stop command:

sudo systemctl stop application[.service]

Restarting and Reloading Services

To restart a running service, you can use the restart command:

sudo systemctl restart application[.service]

If the application in question is able to reload its configuration files (without restarting), you can issue the reload command to initiate that process:

sudo systemctl reload application[.service]

Enabling and Disabling Services

The above commands are useful for starting or stopping commands during the current session. To tell systemd to start services automatically at boot, you must enable them. To start a service at boot, use the enable command:

sudo systemctl enable application[.service]

[root@hdcentos ~]# systemctl enable bluetooth.service
Created symlink from /etc/systemd/system/dbus-org.bluez.service to /usr/lib/syst
emd/system/bluetooth.service.
Created symlink from /etc/systemd/system/bluetooth.target.wants/bluetooth.servic
e to /usr/lib/systemd/system/bluetooth.service.

This will create a symbolic link from the system's copy of the service file (usually in /lib/systemd/system or /etc/systemd/system) into the location on disk where systemd looks for autostart files (usually /etc/systemd/system/some_target.target.wants.

To disable the service from starting automatically, use:

sudo systemctl disable application[.service]

[root@hdcentos ~]# systemctl disable bluetooth.service
Removed symlink /etc/systemd/system/dbus-org.bluez.service.
Removed symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service.

This will remove the symbolic link that indicated that the service should be started automatically.

Keep in mind that enabling a service does not start it in the current session. If you wish to start the service and enable it at boot, you will have to issue both the start and enable commands.

Checking the Status of Services

To check the status of a service on your system, you can use the status command:

sudo systemctl status application[.service]

This will provide you with the service state, the cgroup hierarchy, and the first few log lines.

[root@hdcentos ~]# systemctl status bluetooth.service
● bluetooth.service - Bluetooth service
   Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2016-05-08 19:11:56 PDT; 9min ago
     Docs: man:bluetoothd(8)
 Main PID: 7164 (bluetoothd)
   Status: "Running"
   CGroup: /system.slice/bluetooth.service
           └─7164 /usr/libexec/bluetooth/bluetoothd

May 08 19:11:56 hdcentos systemd[1]: Starting Bluetooth service...
May 08 19:11:56 hdcentos bluetoothd[7164]: Bluetooth daemon 5.23
May 08 19:11:56 hdcentos bluetoothd[7164]: Starting SDP server
May 08 19:11:56 hdcentos bluetoothd[7164]: Bluetooth management interface 1....d
May 08 19:11:56 hdcentos systemd[1]: Started Bluetooth service.
Hint: Some lines were ellipsized, use -l to show in full.

This is a good overview of the current status of the application, notifying you of any problems and any actions that may be required.

There are also methods for checking for specific states. For instance, to check to see if a unit is currently active (running), you can use the is-active command:

systemctl is-active application[.service]

This will return the current unit state, which is usually active or inactive. The exit code will be 0 if it is active, making the result simpler to parse programatically.

To see if the unit is enabled, you can use the is-enabled command:

systemctl is-enabled application[.service]

This will output whether the service is enabled or disabled and will again set the exit code to 0 or 1 depending on the answer to the command question.

A third check is whether the unit is in a failed state. This indicates that there was a problem starting the unit in question:

systemctl is-failed application[.service]

This will return active if it is running properly or failed if an error occurred. If the unit was intentionally stopped, it may return unknown or inactive. An exit status of 0 indicates that a failure occurred and an exit status of 1 indicates any other status.

System State Overview

The commands so far have been useful for managing single services, but they are not very helpful for exploring the current state of the system. There are a number of systemctl commands that provide this information.

Listing Current Units

To see a list of all of the active units that systemd knows about, we can use the list-units command:

systemctl list-units

This will show you a list of all of the units that systemd currently has active on the system. On my system, this listed 147 loaded units. The output will look something like this:

UNIT                                      LOAD   ACTIVE SUB     DESCRIPTION
atd.service                               loaded active running ATD daemon
avahi-daemon.service                      loaded active running Avahi mDNS/DNS-SD Stack
dbus.service                              loaded active running D-Bus System Message Bus
dcron.service                             loaded active running Periodic Command Scheduler
dkms.service                              loaded active exited  Dynamic Kernel Modules System
[email protected]                        loaded active running Getty on tty1
...

The output has the following columns:

Since the list-units command shows only active units by default, all of the entries above will show loaded in the LOAD column and active in the ACTIVE column. This display is actually the default behavior of systemctl when called without additional commands, so you will see the same thing if you call systemctl with no arguments:

systemctl

We can tell systemctl to output different information by adding additional flags. For instance, to see all of the units that systemd has loaded (or attempted to load), regardless of whether they are currently active, you can use the --all flag, like this:

systemctl list-units --all

This will show any unit that systemd loaded or attempted to load, regardless of its current state on the system. Some units become inactive after running, and some units that systemd attempted to load may have not been found on disk.

You can use other flags to filter these results. For example, we can use the --state= flag to indicate the LOAD, ACTIVE, or SUB states that we wish to see. You will have to keep the --all flag so that systemctl allows non-active units to be displayed:

systemctl list-units --all --state=inactive

Another common filter is the --type= filter. We can tell systemctl to only display units of the type we are interested in. For example, to see only active service units, we can use:

systemctl list-units --type=service

Listing All Unit Files

The list-units command only displays units that systemd has attempted to parse and load into memory. Since systemd will only read units that it thinks it needs, this will not necessarily include all of the available units on the system. To see every available unit file within the systemd paths, including those that systemd has not attempted to load, you can use the list-unit-files command instead:

systemctl list-unit-files

Units are representations of resources that systemd knows about. Since systemd has not necessarily read all of the unit definitions in this view, it only presents information about the files themselves. The output has two columns: the unit file and the state.

[root@hdcentos ~]# systemctl list-unit-files
UNIT FILE                                   STATE   
proc-sys-fs-binfmt_misc.automount           static  
dev-hugepages.mount                         static  
dev-mqueue.mount                            static  
proc-fs-nfsd.mount                          static  
proc-sys-fs-binfmt_misc.mount               static  
sys-fs-fuse-connections.mount               static  
sys-kernel-config.mount                     static  
sys-kernel-debug.mount                      static  
tmp.mount                                   disabled
var-lib-nfs-rpc_pipefs.mount                static  
brandbot.path                               disabled
cups.path                                   enabled
...

The state will usually be "enabled", "disabled", "static", or "masked". In this context, static means that the unit file does not contain an "install" section, which is used to enable a unit. As such, these units cannot be enabled. Usually, this means that the unit performs a one-off action or is used only as a dependency of another unit and should not be run by itself.

Unit Management

So far, we have been working with services and displaying information about the unit and unit files that systemd knows about. However, we can find out more specific information about units using some additional commands.

Displaying a Unit File

To display the unit file that systemd has loaded into its system, you can use the cat command. For instance, to see the unit file of the bluetooth daemon, we could type:

systemctl cat bluetooth.service

[root@hdcentos ~]# systemctl cat bluetooth.service
# /usr/lib/systemd/system/bluetooth.service
[Unit]
Description=Bluetooth service
Documentation=man:bluetoothd(8)

[Service]
Type=dbus
BusName=org.bluez
ExecStart=/usr/libexec/bluetooth/bluetoothd
NotifyAccess=main
#WatchdogSec=10
#Restart=on-failure
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
LimitNPROC=1

[Install]
WantedBy=bluetooth.target
Alias=dbus-org.bluez.service

The output is the unit file as known to the currently running systemd process. This can be important if you have modified unit files recently or if you are overriding certain options in a unit file fragment.

Displaying Dependencies

To see a unit's dependency tree, you can use the list-dependencies command:

systemctl list-dependencies bluetooth.service

This will display a hierarchy mapping the dependencies that must be dealt with in order to start the unit in question. Dependencies, in this context, include those units that are either required by or wanted by the units above it.

[root@hdcentos ~]# systemctl list-dependencies bluetooth.service
bluetooth.service
● ├─system.slice  
● └─basic.target
●   ├─alsa-restore.service
●   ├─alsa-state.service
...

The recursive dependencies are only displayed for .target units, which indicate system states. To recursively list all dependencies, include the --all flag.

To show reverse dependencies (units that depend on the specified unit), you can add the --reverse flag to the command. Other flags that are useful are the --before and --after flags, which can be used to show units that depend on the specified unit starting before and after themselves, respectively.

Checking Unit Properties

To see the low-level properties of a unit, you can use the show command. This will display a list of properties that are set for the specified unit using a key=value format:

systemctl show bluetooth.service

[root@hdcentos ~]# systemctl show bluetooth.service
Type=dbus
Restart=no
NotifyAccess=main
RestartUSec=100ms
TimeoutStartUSec=1min 30s
TimeoutStopUSec=1min 30s
...

If you want to display a single property, you can pass the -p flag with the property name. For instance, to see the conflicts that the sshd.service unit has, you can type:

systemctl show bluetooth.service -p Conflicts

[root@hdcentos ~]# systemctl show bluetooth.service -p Conflicts
Conflicts=shutdown.target

Masking and Unmasking Units

We saw in the service management section how to stop or disable a service, but systemd also has the ability to mark a unit as completely unstartable, automatically or manually, by linking it to /dev/null. This is called masking the unit, and is possible with the mask command:

sudo systemctl mask bluetooth.service

This will prevent the Nginx service from being started, automatically or manually, for as long as it is masked.

If you check the list-unit-files, you will see the service is now listed as masked:

[root@hdcentos ~]# systemctl list-unit-files
...
[email protected]                             disabled
avahi-daemon.service                        enabled
blk-availability.service                    disabled
bluetooth.service                           masked  
brandbot.service                            static  
brltty.service                              disabled
...

Editing Unit Files

systemctl provides builtin mechanisms for editing and modifying unit files if you need to make adjustments.

The edit command, by default, will open a unit file snippet for the unit in question:

sudo systemctl edit nginx.service

This will be a blank file that can be used to override or add directives to the unit definition. A directory will be created within the /etc/systemd/system directory which contains the name of the unit with .d appended. For instance, for the nginx.service, a directory called nginx.service.d will be created.

Within this directory, a snippet will be created called override.conf. When the unit is loaded, systemd will, in memory, merge the override snippet with the full unit file. The snippet's directives will take precedence over those found in the original unit file.

If you wish to edit the full unit file instead of creating a snippet, you can pass the --full flag:

sudo systemctl edit --full nginx.service

This will load the current unit file into the editor, where it can be modified. When the editor exits, the changed file will be written to /etc/systemd/system, which will take precedence over the system's unit definition (usually found somewhere in /lib/systemd/system).

To remove any additions you have made, either delete the unit's .d configuration directory or the modified service file from /etc/systemd/system. For instance, to remove a snippet, we could type:

sudo rm -r /etc/systemd/system/nginx.service.d

To remove a full modified unit file, we would type:

sudo rm /etc/systemd/system/nginx.service

After deleting the file or directory, you should reload the systemd process so that it no longer attempts to reference these files and reverts back to using the system copies. You can do this by typing:

sudo systemctl daemon-reload

Adjusting the System State (Runlevel) with Targets

Targets are special unit files that describe a system state or synchronization point. Like other units, the files that define targets can be identified by their suffix, which in this case is .target. Targets do not do much themselves, but are instead used to group other units together.

This can be used in order to bring the system to certain states, much like other init systems use runlevels. They are used as a reference for when certain functions are available, allowing you to specify the desired state instead of the individual units needed to produce that state.

For instance, there is a swap.target that is used to indicate that swap is ready for use. Units that are part of this process can sync with this target by indicating in their configuration that they are WantedBy= or RequiredBy= the swap.target. Units that require swap to be available can specify this condition using the Wants=, Requires=, and After= specifications to indicate the nature of their relationship.

Getting and Setting the Default Target

The systemd process has a default target that it uses when booting the system. Satisfying the cascade of dependencies from that single target will bring the system into the desired state. To find the default target for your system, type:

systemctl get-default

multi-user.target

If you wish to set a different default target, you can use the set-default. For instance, if you have a graphical desktop installed and you wish for the system to boot into that by default, you can change your default target accordingly:

sudo systemctl set-default graphical.target

Listing Available Targets

You can get a list of the available targets on your system by typing:

systemctl list-unit-files --type=target

[root@hdcentos ~]# systemctl list-unit-files --type=target
UNIT FILE                 STATE   
anaconda.target           static  
basic.target              static  
bluetooth.target          static  
cryptsetup-pre.target     static  
cryptsetup.target         static  
ctrl-alt-del.target       disabled
default.target            enabled
...

Unlike runlevels, multiple targets can be active at one time. An active target indicates that systemd has attempted to start all of the units tied to the target and has not tried to tear them down again. To see all of the active targets, type:

systemctl list-units --type=target

[root@hdcentos ~]# systemctl list-units --type=target
UNIT                   LOAD   ACTIVE SUB    DESCRIPTION
basic.target           loaded active active Basic System
cryptsetup.target      loaded active active Encrypted Volumes
getty.target           loaded active active Login Prompts
...

Isolating Targets

It is possible to start all of the units associated with a target and stop all units that are not part of the dependency tree. The command that we need to do this is called, appropriately, isolate. This is similar to changing the runlevel in other init systems.

For instance, if you are operating in a graphical environment with graphical.target active, you can shut down the graphical system and put the system into a multi-user command line state by isolating the multi-user.target. Since graphical.target depends on multi-user.target but not the other way around, all of the graphical units will be stopped.

You may wish to take a look at the dependencies of the target you are isolating before performing this procedure to ensure that you are not stopping vital services:

systemctl list-dependencies multi-user.target

When you are satisfied with the units that will be kept alive, you can isolate the target by typing:

sudo systemctl isolate multi-user.target

Using Shortcuts for Important Events

There are targets defined for important events like powering off or rebooting. However, systemctl also has some shortcuts that add a bit of additional functionality.

For instance, to put the system into rescue (single-user) mode, you can just use the rescue command instead of isolate rescue.target:

This will provide the additional functionality of alerting all logged in users about the event.

To halt the system, you can use the halt command:

sudo systemctl halt

To initiate a full shutdown, you can use the poweroff command:

sudo systemctl poweroff

A restart can be started with the reboot command:

sudo systemctl reboot

These all alert logged in users that the event is occurring, something that simply running or isolating the target will not do. Note that most machines will link the shorter, more conventional commands for these operations so that they work properly with systemd.

For example, to reboot the system, you can usually type:

sudo reboot

Backward Compatibility

systemd is somewhat backward compatible. In the past, in other Init processes, the /etc/rc.d/rc.local script is executed by the init command at boot time or when changing runlevels. Adding commands to the bottom of this script was an easy way to perform necessary tasks like starting special services or initialize devices without writing complex initialization scripts in the /etc/rc.d/init.d/ directory and creating symbolic links. In systemd, after all of the units have been executed, the /etc/rc.d/rc.local script is executed.

The /etc/rc.serial script is used if serial ports must be setup at boot time. This script runs setserial commands to configure the system's serial ports. Refer to the setserial man page for more information.

When the script /etc/rc.d/rc.local is opened for editing, there is a note in the file:

#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local

After editing the systemd, the access and modification times of the file must be updated and the file must be made executable:

touch /etc/rc.d/rc.local
chmod +x /etc/rc.d/rc.local

Next, in order for systemd to recognise and use this file, the systemd rc-local.service must be enabled.

sudo systemctl enable rc-local.service

.service

[Unit]
Description=Daemon to detect crashing apps
After=syslog.target

[Service]
ExecStart=/usr/sbin/abrtd
Type=forking

[Install]
WantedBy=multi-user.target

[Unit]
Description=Daemon to start Hadoop
After=hadoop.conf

[Service]
ExecStart=/usr/hdeco/hadoop/sbin/hadoop-daemon.sh start namenode
Type=dbus

[Install]
WantedBy=multi-user.target

[Unit]

The [Unit] section contains generic information about the service. systemd not only manages system services, but also devices, mount points, timer, and other components of the system. The generic term for all these objects in systemd is a unit, and the [Unit] section encodes information about it that might be applicable not only to services but also in to the other unit types systemd maintains.

Description=

This is a free-form string describing the unit. This is intended for use in UIs to show descriptive information along with the unit name. The description should contain a name that means something to the end user. "Apache2 Web Server" is a good example. Bad examples are "high-performance light-weight HTTP server" (too generic) or "Apache2" (too specific and meaningless for people who do not know Apache).

Documentation=

A space-separated list of URIs referencing documentation for this unit or its configuration. Accepted are only URIs of the types "http://", "https://", "file:", "info:", "man:". For more information about the syntax of these URIs, see uri(7). The URIs should be listed in order of relevance, starting with the most relevant. It is a good idea to first reference documentation that explains what the unit's purpose is, followed by how it is configured, followed by any other related documentation. This option may be specified more than once, in which case the specified list of URIs is merged. If the empty string is assigned to this option, the list is reset and all prior assignments will have no effect.

Before=, After=

This is a space-separated list of unit names. It configures ordering dependencies between units. If a unit foo.service contains a setting Before=bar.service and both units are being started, bar.service's start-up is delayed until foo.service is started up. Note that this setting is independent of and orthogonal to the requirement dependencies as configured by Requires=. It is a common pattern to include a unit name in both the After= and Requires= option, in which case the unit listed will be started before the unit that is configured with these options. This option may be specified more than once, in which case ordering dependencies for all listed names are created. After= is the inverse of Before=, i.e. while After= ensures that the configured unit is started after the listed unit finished starting up, Before= ensures the opposite, i.e. that the configured unit is fully started up before the listed unit is started. Note that when two units with an ordering dependency between them are shut down, the inverse of the start-up order is applied. i.e. if a unit is configured with After= on another unit, the former is stopped before the latter if both are shut down. If one unit with an ordering dependency on another unit is shut down while the latter is started up, the shut down is ordered before the start-up regardless of whether the ordering dependency is actually of type After= or Before=. If two units have no ordering dependencies between them, they are shut down or started up simultaneously, and no ordering takes place.

For more information concerning the [Unit] section, please refer to the Free Desktop reference.

[Service]

The next section is [Service] which encodes information about the service itself. It contains all those settings that apply only to services, and not the other kinds of units systemd maintains (mount points, devices, timers, ...). Two settings are used here: ExecStart= takes the path to the binary to execute when the service shall be started up. And with Type= we configure how the service notifies the init system that it finished starting up. Since traditional Unix daemons do this by returning to the parent process after having forked off and initialized the background daemon we set the type to forking here. That tells systemd to wait until the start-up binary returns and then consider the processes still running afterwards the daemon processes.

[Install]

The final section is [Install]. It encodes information about how the suggested installation should look like, i.e. under which circumstances and by which triggers the service shall be started. In this case we simply say that this service shall be started when the multi-user.target unit is activated. The setting WantedBy= has little effect on the daemon during runtime. It is only read by the systemctl enable command, which is the recommended way to enable a service in systemd. This command will simply ensure that our little service gets automatically activated as soon as multi-user.target is requested, which it is on all normal boots.

Installing

Notes from Class

systemd units are daemons.

journalctl

journalctl -b

journalctl -u rngd.service