Skip to content

Commit

Permalink
SRE-18741: improve shutdown handling
Browse files Browse the repository at this point in the history
Don't release the lock before cancelling pending locks. There might or
might not have been a race condition there, but either way, it's best to
have signal handlers be minimal. We can release any existing locks
later.

We do still need to cancel the lock in the handler, in order to raise an
exception to break out of synchronous code.

Also, don't try self.zoo.close if self.zoo is None.
  • Loading branch information
bugfood committed Jan 22, 2024
1 parent e685ad2 commit 4427225
Showing 1 changed file with 9 additions and 5 deletions.
14 changes: 9 additions & 5 deletions tds/scripts/daemon.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,13 @@ def main(self):
if self.end_callback:
self.end_callback()

self.zoo.close()
if self.zoo is not None:
if self.lock is not None:
self.release_lock()

self.zoo.stop()
self.zoo.close()

log.info("Stopped.")

def release_lock(self):
Expand Down Expand Up @@ -255,8 +261,6 @@ def shutdown_handler(self, signum, frame):

self._should_stop = True
if self.lock is not None:
self.release_lock()
# Cancel any pending lock acquisition. This will cause kazoo to
# raise a CancelledError, which we catch.
self.cancel_lock()

if self.zoo is not None:
self.zoo.stop()

0 comments on commit 4427225

Please sign in to comment.