Merge pull request #11304 from saubyk/fix/shutdown-exit-codes
What changed, and why it matters
This change makes the LND Bitcoin Lightning node return a non-zero exit code when it shuts itself down because of an internal failure, such as a failed health check. Previously, LND often exited with code 0 even on failure, which made process managers like systemd think the stop was intentional and not restart the node. The patch also assigns specific exit codes to different health-check failures (chain backend, disk space, TLS, Tor, remote signer, leader status). This is a reliability/operational fix, not a vulnerability that an attacker can exploit.
No immediate security action required. Node operators using systemd or other process managers should review the updated lnd.service comments and consider whether to configure RestartPreventExitStatus for non-recoverable conditions such as low disk space (exit code 11).
Security signals we found
Operational security improvement: failed self-shutdowns are now distinguishable from graceful stops by exit code
Non-exploitable: no untrusted input parsing, no memory corruption, no privilege change, no cryptographic weakness
Process-manager integration: systemd Restart=on-failure will now restart LND after health-check failures
Regression test coverage added for exit codes in signal, log, server, and etcd failover integration tests
Evidence from the diff
The commit introduces a signal.ExitCode type and an atomic exitCode field in the signal.Interceptor. Components that detect a failure call interceptor.SetExitCode() before requesting shutdown; the main() wrapper in cmd/lnd/main.go now calls os.Exit(int(shutdownInterceptor.ExitCode())). Health checks in server.go register a failure callback that records a specific code (10-15) before the generic critical-error logger records code 1, and SetExitCode keeps only the first non-zero code. OS signals and lncli stop still exit 0. The change is defensive and improves observability and auto-recovery behavior for node operators.
Changed components
cmd/lnd/main.golnd.golog.goserver.gosignal/exit_codes.gosignal/signal.gocontrib/init/lnd.servicesample-lnd.confInspect captured patch +588 / −12
### cmd/lnd/main.go
@@ -41,4 +41,9 @@ func main() {
_, _ = fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
+
+ // A graceful shutdown may still have been forced by a failure, such as
+ // a health check giving up on the chain backend. Report it so process
+ // managers can tell it apart from a deliberate stop.
+ os.Exit(int(shutdownInterceptor.ExitCode()))
}
### contrib/init/lnd.service
@@ -15,9 +15,18 @@ ExecStop=/usr/local/bin/lncli stop
User=bitcoin
Group=bitcoin
-# Try restarting lnd if it stops due to a failure
+# Try restarting lnd if it stops due to a failure. lnd exits with 0 after an
+# OS signal or `lncli stop`, with 1 on a critical error, and with a distinct
+# code when a health check gives up: 10 chain backend, 11 disk space, 12 TLS
+# certificate, 13 tor connection, 14 remote signer, 15 leader status. A health
+# check code always names the real cause, but a failure can surface elsewhere
+# before the check gives up and then lnd exits with 1 instead. A restart cannot
+# fix some of these, such as low disk space. Whether lnd should stay down in
+# that case is an operator decision, so uncomment and adjust the
+# RestartPreventExitStatus line below to make it.
Restart=on-failure
RestartSec=60
+#RestartPreventExitStatus=11
# Type=notify is required for lnd to notify systemd when it is ready
Type=notify
### docs/release-notes/unreleased/11304.md
@@ -0,0 +1,15 @@
+# Bug Fixes
+
+* lnd [now exits with a non-zero
+ code](https://github.com/lightningnetwork/lnd/pull/11304) when it shuts
+ itself down because of a
+ [failure](https://github.com/lightningnetwork/lnd/issues/5625), so process
+ managers such as systemd restart it instead of treating the stop as
+ deliberate. A critical error exits with 1, and a failed health check exits
+ with a code that identifies it: 10 chain backend, 11 disk space, 12 TLS
+ certificate, 13 tor connection, 14 remote signer, 15 leader status. A
+ health check code always names the real cause, but a failure can surface
+ elsewhere before the check gives up, and then lnd exits with 1 instead. The
+ code is logged when the shutdown starts. An OS signal or `lncli stop` exits
+ with 0, including a signal received while lnd is still waiting for the
+ wallet to be unlocked, which previously exited with 1.
### itest/lnd_etcd_failover_test.go
@@ -323,8 +323,11 @@ func testLeaderHealthCheck(ht *lntest.HarnessTest) {
proxy.Stop(ht.T)
// Wait for Carol-1 to stop. If the health check wouldn't properly work
- // this call would timeout and trigger a test failure.
- require.NoError(ht.T, carol.WaitForProcessExit())
+ // this call would timeout and trigger a test failure. A failed leader
+ // health check exits with signal.ExitCodeLeaderStatus.
+ require.ErrorContains(
+ ht.T, carol.WaitForProcessExit(), "exit status 15",
+ )
// Now that Carol-1 is shut down we should fail over to Carol-2.
failoverTimeout := time.Duration(2*leaderSessionTTL) * time.Second
### lnd.go
@@ -563,9 +563,19 @@ func Main(cfg *Config, lisCfg ListenerCfg, implCfg *ImplementationCfg,
// clean up functions can be executed. That will also shut down
// the wallet.
// We can't continue to execute the code below as we can't
- // do any operations which requires private keys.
- return mkErr("Shutting down", errors.New("shutdown signal "+
- "received while waiting for wallet to be ready"))
+ // do any operations which requires private keys. A stop forced
+ // by a failure is still reported as an error for callers of
+ // Main that do not read the exit code.
+ if interceptor.ExitCode() != signal.ExitCodeSuccess {
+ return mkErr("Shutting down", errors.New("shutdown "+
+ "forced by a failure while waiting for wallet "+
+ "to be ready"))
+ }
+
+ ltndLog.Info("Shutdown signal received while waiting for " +
+ "wallet to be ready")
+
+ return nil
}
// TODO(roasbeef): add rotation
@@ -699,7 +709,7 @@ func Main(cfg *Config, lisCfg ListenerCfg, implCfg *ImplementationCfg,
ctx, cfg, cfg.Listeners, dbs, activeChainControl, &idKeyDesc,
activeChainControl.Cfg.WalletUnlockParams.ChansToRestore,
multiAcceptor, torController, tlsManager, leaderElector,
- implCfg, remoteSignerClientFactory,
+ implCfg, remoteSignerClientFactory, interceptor,
)
if err != nil {
return mkErr("unable to create server", err)
### log.go
@@ -113,12 +113,15 @@ func genSubLogger(root *build.SubLoggerManager,
interceptor signal.Interceptor) func(string) btclog.Logger {
// Create a shutdown function which will request shutdown from our
- // interceptor if it is listening.
+ // interceptor if it is listening. A critical error is a failure, so
+ // the process exits with a non-zero code unless a more specific one
+ // was already recorded.
shutdown := func() {
if !interceptor.Listening() {
return
}
+ interceptor.SetExitCode(signal.ExitCodeCriticalError)
interceptor.RequestShutdown()
}
### log_test.go
@@ -0,0 +1,34 @@
+package lnd
+
+import (
+ "io"
+ "testing"
+ "time"
+
+ "github.com/btcsuite/btclog/v2"
+ "github.com/lightningnetwork/lnd/build"
+ "github.com/lightningnetwork/lnd/signal"
+ "github.com/stretchr/testify/require"
+)
+
+// TestCriticalLogExitCode asserts that a critical log entry requests a
+// shutdown that exits with the generic critical error code.
+func TestCriticalLogExitCode(t *testing.T) {
+ interceptor, err := signal.Intercept()
+ require.NoError(t, err)
+
+ logger := genSubLogger(
+ build.NewSubLoggerManager(btclog.NewDefaultHandler(io.Discard)),
+ interceptor,
+ )("TEST")
+
+ logger.Criticalf("something went wrong")
+
+ select {
+ case <-interceptor.ShutdownChannel():
+ case <-time.After(time.Second):
+ t.Fatal("expected critical log to request shutdown")
+ }
+
+ require.Equal(t, signal.ExitCodeCriticalError, interceptor.ExitCode())
+}
### sample-lnd.conf
@@ -1171,6 +1171,15 @@
[healthcheck]
+; When a health check fails for all of its attempts, lnd shuts down gracefully
+; and exits with a non-zero code that identifies the check: 10 chain backend,
+; 11 disk space, 12 tls, 13 tor connection, 14 remote signer, 15 leader
+; status. A critical error that is not a health check exits with 1, and an OS
+; signal or `lncli stop` exits with 0. A health check code always names the
+; real cause, but a failure can surface elsewhere before the check gives up,
+; for example when the chain backend goes away between two check intervals,
+; and then lnd exits with 1 instead.
+
; The number of times we should attempt to query our chain backend before
; gracefully shutting down. Set this value to 0 to disable this health check.
; healthcheck.chainbackend.attempts=3
### server.go
@@ -79,6 +79,7 @@ import (
"github.com/lightningnetwork/lnd/routing"
"github.com/lightningnetwork/lnd/routing/localchans"
"github.com/lightningnetwork/lnd/routing/route"
+ "github.com/lightningnetwork/lnd/signal"
"github.com/lightningnetwork/lnd/subscribe"
"github.com/lightningnetwork/lnd/sweep"
"github.com/lightningnetwork/lnd/ticker"
@@ -235,6 +236,10 @@ type server struct {
implCfg *ImplementationCfg
+ // interceptor is used to request a shutdown of the daemon, and to
+ // record the exit code that explains why.
+ interceptor signal.Interceptor
+
// identityECDH is an ECDH capable wrapper for the private key used
// to authenticate any incoming connections.
identityECDH keychain.SingleKeyECDH
@@ -677,7 +682,7 @@ func newServer(ctx context.Context, cfg *Config, listenAddrs []net.Addr,
leaderElector cluster.LeaderElector,
implCfg *ImplementationCfg,
remoteSignerClientFactory func() (rpcwallet.RemoteSignerClient,
- error)) (*server, error) {
+ error), interceptor signal.Interceptor) (*server, error) {
var (
err error
@@ -802,6 +807,7 @@ func newServer(ctx context.Context, cfg *Config, listenAddrs []net.Addr,
s := &server{
cfg: cfg,
implCfg: implCfg,
+ interceptor: interceptor,
graphDB: dbs.GraphDB,
v1Graph: v1Graph,
chanStateDB: chanStateDB,
@@ -2130,6 +2136,9 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
cfg.HealthChecks.ChainCheck.Timeout,
cfg.HealthChecks.ChainCheck.Backoff,
chainBackendAttempts,
+ healthcheck.WithFailureCallback(exitCodeOnFailure(
+ s.interceptor, signal.ExitCodeChainBackend,
+ )),
)
diskCheck := healthcheck.NewObservation(
@@ -2156,6 +2165,9 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
cfg.HealthChecks.DiskCheck.Timeout,
cfg.HealthChecks.DiskCheck.Backoff,
cfg.HealthChecks.DiskCheck.Attempts,
+ healthcheck.WithFailureCallback(exitCodeOnFailure(
+ s.interceptor, signal.ExitCodeDiskSpace,
+ )),
)
tlsHealthCheck := healthcheck.NewObservation(
@@ -2180,6 +2192,9 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
cfg.HealthChecks.TLSCheck.Timeout,
cfg.HealthChecks.TLSCheck.Backoff,
cfg.HealthChecks.TLSCheck.Attempts,
+ healthcheck.WithFailureCallback(exitCodeOnFailure(
+ s.interceptor, signal.ExitCodeTLSCert,
+ )),
)
checks := []*healthcheck.Observation{
@@ -2204,6 +2219,9 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
cfg.HealthChecks.TorConnection.Timeout,
cfg.HealthChecks.TorConnection.Backoff,
cfg.HealthChecks.TorConnection.Attempts,
+ healthcheck.WithFailureCallback(exitCodeOnFailure(
+ s.interceptor, signal.ExitCodeTorConnection,
+ )),
)
checks = append(checks, torConnectionCheck)
}
@@ -2248,6 +2266,9 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
outerTimeout,
cfg.HealthChecks.RemoteSigner.Backoff,
cfg.HealthChecks.RemoteSigner.Attempts,
+ healthcheck.WithFailureCallback(exitCodeOnFailure(
+ s.interceptor, signal.ExitCodeRemoteSigner,
+ )),
)
checks = append(checks, rsConnectionCheck)
}
@@ -2290,6 +2311,9 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
cfg.HealthChecks.LeaderCheck.Timeout,
cfg.HealthChecks.LeaderCheck.Backoff,
cfg.HealthChecks.LeaderCheck.Attempts,
+ healthcheck.WithFailureCallback(exitCodeOnFailure(
+ s.interceptor, signal.ExitCodeLeaderStatus,
+ )),
)
checks = append(checks, leaderCheck)
@@ -2307,6 +2331,18 @@ func (s *server) createLivenessMonitor(ctx context.Context, cfg *Config,
return nil
}
+// exitCodeOnFailure returns a health check failure callback that records code
+// as the exit code of the process. The monitor fires it right before it
+// requests shutdown through the critical logger, which records the generic
+// critical error code, so the code for the failed check is the one kept.
+func exitCodeOnFailure(interceptor signal.Interceptor,
+ code signal.ExitCode) func() {
+
+ return func() {
+ interceptor.SetExitCode(code)
+ }
+}
+
// Started returns true if the server has been started, and false otherwise.
// NOTE: This function is safe for concurrent access.
func (s *server) Started() bool {
### server_test.go
@@ -1,10 +1,17 @@
package lnd
import (
+ "context"
+ "errors"
"net"
+ "path/filepath"
"testing"
"time"
+ "github.com/lightningnetwork/lnd/chainreg"
+ "github.com/lightningnetwork/lnd/lncfg"
+ "github.com/lightningnetwork/lnd/lntest/mock"
+ "github.com/lightningnetwork/lnd/signal"
"github.com/lightningnetwork/lnd/tor"
"github.com/stretchr/testify/require"
)
@@ -238,3 +245,192 @@ func TestWithoutV2Onion(t *testing.T) {
// fetchNodeAdvertisedAddrs treat this as "no advertised address".
require.Empty(t, withoutV2Onion([]net.Addr{v2, v2}))
}
+
+// TestExitCodeOnFailure asserts that the failure callback attached to a
+// health check records that check's exit code, and that a later generic code
+// does not overwrite it.
+func TestExitCodeOnFailure(t *testing.T) {
+ interceptor, err := signal.Intercept()
+ require.NoError(t, err)
+ t.Cleanup(func() {
+ interceptor.RequestShutdown()
+ <-interceptor.ShutdownChannel()
+ })
+
+ exitCodeOnFailure(interceptor, signal.ExitCodeChainBackend)()
+ interceptor.SetExitCode(signal.ExitCodeCriticalError)
+
+ require.Equal(t, signal.ExitCodeChainBackend, interceptor.ExitCode())
+}
+
+// failingPinger is a wallet controller whose remote signer ping always fails.
+type failingPinger struct {
+ *mock.WalletController
+}
+
+// Ping fails so that the remote signer health check gives up.
+func (p *failingPinger) Ping(context.Context, time.Duration) error {
+ return errors.New("remote signer unreachable")
+}
+
+// notLeader is a leader elector that reports it is never the leader.
+type notLeader struct{}
+
+func (notLeader) Campaign(context.Context) error { return nil }
+func (notLeader) Resign(context.Context) error { return nil }
+func (notLeader) Leader(context.Context) (string, error) { return "", nil }
+func (notLeader) IsLeader(context.Context) (bool, error) { return false, nil }
+
+// TestLivenessMonitorExitCodes drives the server's liveness monitor with one
+// failing health check at a time and asserts that each check records its own
+// exit code. The tor check is not covered as it needs a live tor control
+// connection.
+func TestLivenessMonitorExitCodes(t *testing.T) {
+ // disabled keeps a check out of the monitor, enabled fails it on the
+ // first attempt shortly after the monitor starts.
+ disabled := func() *lncfg.CheckConfig {
+ return &lncfg.CheckConfig{Attempts: 0}
+ }
+ enabled := func() *lncfg.CheckConfig {
+ return &lncfg.CheckConfig{
+ Interval: 10 * time.Millisecond,
+ Attempts: 1,
+ Timeout: time.Second,
+ Backoff: 0,
+ }
+ }
+
+ tests := []struct {
+ name string
+
+ // setup enables the check under test and makes it fail.
+ setup func(t *testing.T, s *server, cfg *Config,
+ cc *chainreg.ChainControl)
+
+ leader bool
+ exitCode signal.ExitCode
+ }{{
+ name: "chain backend",
+ setup: func(t *testing.T, s *server, cfg *Config,
+ cc *chainreg.ChainControl) {
+
+ cfg.HealthChecks.ChainCheck = enabled()
+ cc.HealthCheck = func() error {
+ return errors.New("chain backend down")
+ }
+ },
+ exitCode: signal.ExitCodeChainBackend,
+ }, {
+ name: "disk space",
+ setup: func(t *testing.T, s *server, cfg *Config,
+ cc *chainreg.ChainControl) {
+
+ // A missing dir fails the check regardless of the
+ // host's actual free space.
+ cfg.LndDir = filepath.Join(cfg.LndDir, "missing")
+ cfg.HealthChecks.DiskCheck = &lncfg.DiskCheckConfig{
+ RequiredRemaining: 0.5,
+ CheckConfig: enabled(),
+ }
+ },
+ exitCode: signal.ExitCodeDiskSpace,
+ }, {
+ name: "tls certificate",
+ setup: func(t *testing.T, s *server, cfg *Config,
+ cc *chainreg.ChainControl) {
+
+ cfg.HealthChecks.TLSCheck = enabled()
+ s.tlsManager = NewTLSManager(&TLSManagerCfg{
+ TLSCertPath: filepath.Join(
+ cfg.LndDir, "missing.cert",
+ ),
+ TLSKeyPath: filepath.Join(
+ cfg.LndDir, "missing.key",
+ ),
+ })
+ },
+ exitCode: signal.ExitCodeTLSCert,
+ }, {
+ name: "remote signer",
+ setup: func(t *testing.T, s *server, cfg *Config,
+ cc *chainreg.ChainControl) {
+
+ cfg.HealthChecks.RemoteSigner = enabled()
+ cfg.RemoteSigner.Enable = true
+ cc.Wc = &failingPinger{
+ WalletController: &mock.WalletController{},
+ }
+ },
+ exitCode: signal.ExitCodeRemoteSigner,
+ }, {
+ name: "leader status",
+ setup: func(t *testing.T, s *server, cfg *Config,
+ cc *chainreg.ChainControl) {
+
+ cfg.HealthChecks.LeaderCheck = enabled()
+ },
+ leader: true,
+ exitCode: signal.ExitCodeLeaderStatus,
+ }}
+
+ for _, test := range tests {
+ t.Run(test.name, func(t *testing.T) {
+ interceptor, err := signal.Intercept()
+ require.NoError(t, err)
+ t.Cleanup(func() {
+ interceptor.RequestShutdown()
+ <-interceptor.ShutdownChannel()
+ })
+
+ cfg := &Config{
+ LndDir: t.TempDir(),
+ Bitcoin: &lncfg.Chain{Node: "bitcoind"},
+ RemoteSigner: &lncfg.RemoteSigner{},
+ HealthChecks: &lncfg.HealthCheckConfig{
+ ChainCheck: disabled(),
+ DiskCheck: &lncfg.DiskCheckConfig{
+ CheckConfig: disabled(),
+ },
+ TLSCheck: disabled(),
+ TorConnection: disabled(),
+ RemoteSigner: disabled(),
+ LeaderCheck: disabled(),
+ },
+ }
+ partial := &chainreg.PartialChainControl{
+ HealthCheck: func() error { return nil },
+ }
+ cc := &chainreg.ChainControl{
+ PartialChainControl: partial,
+ }
+ s := &server{
+ cfg: cfg,
+ interceptor: interceptor,
+ cc: cc,
+ }
+
+ test.setup(t, s, cfg, cc)
+
+ var elector notLeader
+ if test.leader {
+ err = s.createLivenessMonitor(
+ t.Context(), cfg, cc, elector,
+ )
+ } else {
+ err = s.createLivenessMonitor(
+ t.Context(), cfg, cc, nil,
+ )
+ }
+ require.NoError(t, err)
+
+ require.NoError(t, s.livenessMonitor.Start())
+ t.Cleanup(func() {
+ require.NoError(t, s.livenessMonitor.Stop())
+ })
+
+ require.Eventually(t, func() bool {
+ return interceptor.ExitCode() == test.exitCode
+ }, 5*time.Second, 10*time.Millisecond)
+ })
+ }
+}
### signal/exit_codes.go
@@ -0,0 +1,89 @@
+package signal
+
+import "fmt"
+
+// The exit codes below are reported to the parent process when lnd stops. They
+// let process managers such as systemd tell a deliberate stop apart from one
+// forced by a failure, and identify which failure it was, without parsing the
+// logs. A unit configured with Restart=on-failure only restarts lnd for a
+// non-zero code, and RestartPreventExitStatus can exclude specific codes.
+//
+// Health check codes start at 10 to stay clear of the codes used by the Go
+// runtime and by shells: 2 is an unrecovered panic or fatal runtime error.
+//
+// A health check code is only reported when that check gave up, so it always
+// names the real cause. The reverse does not hold: the failure a check exists
+// to catch can surface elsewhere first, for instance when the chain backend
+// goes away between two check intervals and a component that depends on it
+// raises a critical error. lnd then exits with ExitCodeCriticalError, so that
+// code does not rule out a cause that has its own code.
+
+// ExitCode is the status lnd reports to the parent process when it stops.
+type ExitCode int
+
+const (
+ // ExitCodeSuccess is used when lnd was stopped deliberately, either by
+ // an OS signal or by a StopDaemon RPC call (lncli stop).
+ ExitCodeSuccess ExitCode = 0
+
+ // ExitCodeCriticalError is used when lnd stopped because of a critical
+ // error that is not covered by a more specific code below. Startup and
+ // configuration failures also exit with this code.
+ ExitCodeCriticalError ExitCode = 1
+
+ // ExitCodeChainBackend is used when the chain backend health check
+ // failed.
+ ExitCodeChainBackend ExitCode = 10
+
+ // ExitCodeDiskSpace is used when the disk space health check failed.
+ ExitCodeDiskSpace ExitCode = 11
+
+ // ExitCodeTLSCert is used when the TLS certificate health check
+ // failed.
+ ExitCodeTLSCert ExitCode = 12
+
+ // ExitCodeTorConnection is used when the tor connection health check
+ // failed.
+ ExitCodeTorConnection ExitCode = 13
+
+ // ExitCodeRemoteSigner is used when the remote signer connection
+ // health check failed.
+ ExitCodeRemoteSigner ExitCode = 14
+
+ // ExitCodeLeaderStatus is used when the leader status health check
+ // failed.
+ ExitCodeLeaderStatus ExitCode = 15
+)
+
+// String returns the reason an exit code stands for, so log lines and error
+// messages can name it without the reader looking up the number.
+func (c ExitCode) String() string {
+ switch c {
+ case ExitCodeSuccess:
+ return "stop requested"
+
+ case ExitCodeCriticalError:
+ return "critical error or startup failure"
+
+ case ExitCodeChainBackend:
+ return "chain backend health check failed"
+
+ case ExitCodeDiskSpace:
+ return "disk space health check failed"
+
+ case ExitCodeTLSCert:
+ return "TLS certificate health check failed"
+
+ case ExitCodeTorConnection:
+ return "tor connection health check failed"
+
+ case ExitCodeRemoteSigner:
+ return "remote signer health check failed"
+
+ case ExitCodeLeaderStatus:
+ return "leader status health check failed"
+
+ default:
+ return fmt.Sprintf("unknown exit code %d", int(c))
+ }
+}
### signal/signal.go
@@ -111,6 +111,11 @@ type Interceptor struct {
// close this channel.
quit chan struct{}
+ // exitCode is the code the process should exit with once shutdown
+ // completes. It is a pointer so that every copy of the interceptor
+ // shares it, as the interceptor is passed around by value.
+ exitCode *atomic.Int32
+
// Notifier handles sending shutdown notifications.
Notifier Notifier
}
@@ -127,6 +132,7 @@ func Intercept() (Interceptor, error) {
shutdownChannel: make(chan struct{}),
shutdownRequestChannel: make(chan struct{}),
quit: make(chan struct{}),
+ exitCode: &atomic.Int32{},
}
signalsToCatch := []os.Signal{
@@ -147,7 +153,6 @@ func Intercept() (Interceptor, error) {
// callback registration.
// It must be run as a goroutine.
func (c *Interceptor) mainInterruptHandler() {
- defer atomic.StoreInt32(&started, 0)
// isShutdown is a flag which is used to indicate whether or not
// the shutdown signal has already been received and hence any future
// attempts to add a new interrupt handler should invoke them
@@ -182,7 +187,16 @@ func (c *Interceptor) mainInterruptHandler() {
shutdown()
case <-c.quit:
- log.Infof("Gracefully shutting down.")
+ // The exit code can no longer change once quit is
+ // closed, so this is the code the process exits with.
+ code := c.ExitCode()
+ log.Infof("Gracefully shutting down with exit code "+
+ "%d (%v).", code, code)
+
+ // Release the started flag before signaling the exit
+ // so a new interceptor can be started as soon as the
+ // shutdown channel is closed.
+ atomic.StoreInt32(&started, 0)
close(c.shutdownChannel)
signal.Stop(c.interruptChannel)
return
@@ -214,14 +228,42 @@ func (c *Interceptor) Alive() bool {
}
}
-// RequestShutdown initiates a graceful shutdown from the application.
+// RequestShutdown initiates a graceful shutdown from the application. Unless
+// an exit code was recorded with SetExitCode, the process exits with
+// ExitCodeSuccess.
func (c *Interceptor) RequestShutdown() {
select {
case c.shutdownRequestChannel <- struct{}{}:
case <-c.quit:
}
}
+// SetExitCode records the code the process should exit with once shutdown
+// completes. Only the first non-zero code is kept, so that the specific
+// reason recorded by the component that detected a failure is not overwritten
+// by the generic critical error raised while logging it. A code recorded
+// after shutdown has already begun is ignored, so a failure noticed during
+// the teardown of a deliberate stop does not turn it into a failed exit. That
+// guard is best-effort: the liveness check and the record are separate steps,
+// so a code recorded in the instant shutdown begins may still be kept.
+func (c *Interceptor) SetExitCode(code ExitCode) {
+ if c.exitCode == nil || !c.Alive() {
+ return
+ }
+
+ c.exitCode.CompareAndSwap(int32(ExitCodeSuccess), int32(code))
+}
+
+// ExitCode returns the code the process should exit with. It is
+// ExitCodeSuccess unless a failure was recorded with SetExitCode.
+func (c *Interceptor) ExitCode() ExitCode {
+ if c.exitCode == nil {
+ return ExitCodeSuccess
+ }
+
+ return ExitCode(c.exitCode.Load())
+}
+
// ShutdownChannel returns the channel that will be closed once the main
// interrupt handler has exited.
func (c *Interceptor) ShutdownChannel() <-chan struct{} {
### signal/signal_test.go
@@ -0,0 +1,125 @@
+package signal
+
+import (
+ "os"
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+)
+
+// newTestInterceptor starts an interceptor and registers a cleanup that waits
+// for its main handler to exit, so that the next test can start a fresh one.
+func newTestInterceptor(t *testing.T) Interceptor {
+ t.Helper()
+
+ interceptor, err := Intercept()
+ require.NoError(t, err)
+
+ t.Cleanup(func() {
+ interceptor.RequestShutdown()
+ waitForStopped(t, interceptor)
+ })
+
+ return interceptor
+}
+
+// waitForStopped waits for the interceptor's main handler to exit.
+func waitForStopped(t *testing.T, interceptor Interceptor) {
+ t.Helper()
+
+ select {
+ case <-interceptor.ShutdownChannel():
+ case <-time.After(time.Second):
+ t.Fatal("interceptor did not shut down")
+ }
+}
+
+// TestExitCodeDefault asserts that a shutdown requested without a code, as done
+// by lncli stop, exits successfully.
+func TestExitCodeDefault(t *testing.T) {
+ interceptor := newTestInterceptor(t)
+
+ interceptor.RequestShutdown()
+ waitForStopped(t, interceptor)
+
+ require.Equal(t, ExitCodeSuccess, interceptor.ExitCode())
+}
+
+// TestExitCodeSignal asserts that a shutdown triggered by an OS signal exits
+// successfully.
+func TestExitCodeSignal(t *testing.T) {
+ interceptor := newTestInterceptor(t)
+
+ interceptor.interruptChannel <- os.Interrupt
+ waitForStopped(t, interceptor)
+
+ require.Equal(t, ExitCodeSuccess, interceptor.ExitCode())
+}
+
+// TestExitCodeFirstWins asserts that the first non-zero exit code recorded is
+// the one reported, so a specific reason is not overwritten by the generic
+// critical error raised while logging it, and that the code is visible through
+// every copy of the interceptor.
+func TestExitCodeFirstWins(t *testing.T) {
+ interceptor := newTestInterceptor(t)
+
+ // The interceptor is passed around by value, so record the code on a
+ // copy and read it back through the original.
+ cp := interceptor
+ cp.SetExitCode(ExitCodeDiskSpace)
+ cp.SetExitCode(ExitCodeCriticalError)
+
+ interceptor.RequestShutdown()
+ waitForStopped(t, interceptor)
+
+ require.Equal(t, ExitCodeDiskSpace, interceptor.ExitCode())
+}
+
+// TestExitCodeAfterShutdown asserts that a failure recorded once a shutdown
+// is already underway, for example a health check giving up during the
+// teardown of lncli stop, does not change the exit code.
+func TestExitCodeAfterShutdown(t *testing.T) {
+ interceptor := newTestInterceptor(t)
+
+ interceptor.RequestShutdown()
+ waitForStopped(t, interceptor)
+
+ interceptor.SetExitCode(ExitCodeChainBackend)
+
+ require.Equal(t, ExitCodeSuccess, interceptor.ExitCode())
+}
+
+// TestExitCodeZeroValue asserts that a zero-value interceptor, as returned by
+// Intercept on error or used by embedders that never start one, reports a
+// successful exit instead of panicking.
+func TestExitCodeZeroValue(t *testing.T) {
+ var interceptor Interceptor
+
+ require.NotPanics(t, func() {
+ interceptor.SetExitCode(ExitCodeChainBackend)
+ })
+ require.Equal(t, ExitCodeSuccess, interceptor.ExitCode())
+}
+
+// TestExitCodeString checks that every documented exit code has a reason, and
+// that an unknown code still prints its number.
+func TestExitCodeString(t *testing.T) {
+ t.Parallel()
+
+ codes := map[ExitCode]string{
+ ExitCodeSuccess: "stop requested",
+ ExitCodeCriticalError: "critical error or startup failure",
+ ExitCodeChainBackend: "chain backend health check failed",
+ ExitCodeDiskSpace: "disk space health check failed",
+ ExitCodeTLSCert: "TLS certificate health check failed",
+ ExitCodeTorConnection: "tor connection health check failed",
+ ExitCodeRemoteSigner: "remote signer health check failed",
+ ExitCodeLeaderStatus: "leader status health check failed",
+ }
+ for code, want := range codes {
+ require.Equal(t, want, code.String())
+ }
+
+ require.Equal(t, "unknown exit code 99", ExitCode(99).String())
+}Why this scored 34/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.