Skip to content

Commit

Permalink
systemd: activate disks via systemd service instead of udev
Browse files Browse the repository at this point in the history
The udev(7) man page states:
  RUN
  ...
  This can only be used for very short-running foreground tasks. Running
  an event process for a long period of time may block all further
  events for this or a dependent device.

  Starting daemons or other long-running processes is not appropriate
  for udev; the forked processes, detached or not, will be
  unconditionally killed after the event handling has finished.

ceph-disk activate is far from a short-running task:
- check whether path is a block dev, for dirs call through to
  activate_dir()
- call blkid to obtain the filesystem type for the block dev
- pull mount options from hard-coded ceph.conf file
- mount the OSD dev at a temporary path
- check the ceph magic for mounted filesystem
- read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf
  path
- read or generate (if missing) the OSD uuid
- create a file indicating init system usage (systemd)
- mount the device at a second (final) location
- umount (lazy) the temporary mount path
- enable the systemd ceph-osd@{osd_id} service
- start the systemd ceph-osd@{osd_id} service

This logic is therefore best left in a systemd service for execution. As
it is less limited in terms of execution time, and also allows for
improved event handling in future (fsck, dmcrypt mapping etc.).

This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or
ceph-disk activate-journal via new [email protected],
[email protected] and [email protected]
systemd service files.

[email protected] makes use of the newly added
--dmcrypt parameter for ceph-disk activate.

Signed-off-by: David Disseldorp <[email protected]>
  • Loading branch information
ddiss authored and liewegas committed Aug 1, 2015
1 parent ac16d83 commit 85a8946
Show file tree
Hide file tree
Showing 5 changed files with 56 additions and 1 deletion.
5 changes: 4 additions & 1 deletion systemd/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ unitfiles = \
[email protected] \
[email protected] \
[email protected] \
[email protected]
[email protected] \
[email protected] \
[email protected] \
[email protected]

unitdir = $(systemd_unit_dir)

Expand Down
8 changes: 8 additions & 0 deletions systemd/[email protected]
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[Unit]
Description=Ceph disk journal activation: %f

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/ceph-disk --verbose --log-stdout activate-journal --mark-init systemd %f
TimeoutSec=0
8 changes: 8 additions & 0 deletions systemd/[email protected]
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[Unit]
Description=Ceph disk activation: %f

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/ceph-disk --verbose --log-stdout activate --mark-init systemd %f
TimeoutSec=0
8 changes: 8 additions & 0 deletions systemd/[email protected]
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[Unit]
Description=Ceph disk dmcrypt activation: %f

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/ceph-disk --verbose --log-stdout activate --dmcrypt --mark-init systemd %f
TimeoutSec=0
28 changes: 28 additions & 0 deletions udev/95-ceph-osd.rules.systemd
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# activate ceph-tagged partitions
ACTION=="add", SUBSYSTEM=="block", \
ENV{DEVTYPE}=="partition", \
ENV{ID_PART_ENTRY_TYPE}=="4fbd7e29-9d25-41b8-afd0-062c0ceff05d", \
TAG+="systemd", \
ENV{SYSTEMD_WANTS}+="ceph-disk-activate@/dev/$name.service"

# activate ceph-tagged partitions
ACTION=="add", SUBSYSTEM=="block", \
ENV{DEVTYPE}=="partition", \
ENV{ID_PART_ENTRY_TYPE}=="45b0969e-9b03-4f30-b4c6-b4b80ceff106", \
TAG+="systemd", \
ENV{SYSTEMD_WANTS}+="ceph-disk-activate-journal@/dev/$name.service"

# Map journal if using dm-crypt
ACTION=="add" SUBSYSTEM=="block", \
ENV{DEVTYPE}=="partition", \
ENV{ID_PART_ENTRY_TYPE}=="45b0969e-9b03-4f30-b4c6-5ec00ceff106", \
RUN+="/sbin/cryptsetup --key-file /etc/ceph/dmcrypt-keys/$env{ID_PART_ENTRY_UUID} --key-size 256 create $env{ID_PART_ENTRY_UUID} /dev/$name"

# Map data device and
# activate ceph-tagged partitions
# for dm-crypted data devices
ACTION=="add" SUBSYSTEM=="block", \
ENV{DEVTYPE}=="partition", \
ENV{ID_PART_ENTRY_TYPE}=="4fbd7e29-9d25-41b8-afd0-5ec00ceff05d", \
TAG+="systemd", \
ENV{SYSTEMD_WANTS}+="ceph-disk-dmcrypt-activate@/dev/$name.service"

0 comments on commit 85a8946

Please sign in to comment.