Shut the API down gracefully on deploys
Go · Go · intermediate · modification
Deploys currently cut off in-flight requests: on SIGTERM the process dies mid-response and the load balancer sees 502s. `run` now owns the lifecycle. It pings the database before listening, serves in a goroutine, waits for a signal or a serve error, and then calls `Shutdown` with a 25-second budget. SIGTERM arrives after the 5-second `preStop` sleep, which counts against the 35-second grace period, so the budget leaves 5 seconds to log and exit before SIGKILL. After the first signal, signal handling is reset, so a second Ctrl-C still kills a stuck process. The server also gets real timeouts instead of none. Tested locally: SIGTERM during a slow request lets it finish and exits 0; a taken port and an unreachable database both exit 1.
Go 1.22 JSON API in a Kubernetes Deployment. The container runs the binary under `tini` as init, and the pod spec has a `preStop` hook (`sleep 5`), which delays SIGTERM until the pod's removal from the Service endpoints has propagated; the grace period is 35 seconds. No streaming, WebSocket or long-poll endpoints; request bodies are small JSON (files go straight to object storage through presigned URLs). Every handler finishes within 20 seconds (the database has a 15-second statement timeout) and uses the request context. `config.Load` (existing) reads and validates the environment; `api.NewRouter(db)` (existing) builds the handler tree. The pgx stdlib driver registers itself as `pgx` through the blank import, as before.
Requirements
- Once the server is listening, SIGTERM (Kubernetes) or SIGINT (Ctrl-C locally) makes the API stop accepting new connections, let in-flight requests finish for up to 25 seconds, and exit 0. The pod's `terminationGracePeriodSeconds` is 35, and the 5-second `preStop` sleep counts against it. If requests are still running at 25 seconds, it logs the error and exits 1 without waiting further. A second signal during the drain kills the process immediately.
- Startup fails fast, with a logged error and exit code 1, when the config does not load, the database does not answer a ping within 5 seconds, or the listener cannot start (for example, the port is taken).
- After a completed drain, the database pool is closed only after the HTTP server has stopped serving.
- The server gets explicit timeouts (the old `http.ListenAndServe` had none): `ReadHeaderTimeout` 5 s, `ReadTimeout` and `WriteTimeout` 30 s, `IdleTimeout` 120 s.
- Logging goes through `log/slog`'s default logger.
Files touched
- cmd/api/main.go
--- cmd/api/main.go
import (
+ "context"
"database/sql"
- "log"
+ "errors"
+ "fmt"
+ "log/slog"
"net/http"
+ "os"
+ "os/signal"
+ "syscall"
+ "time"
_ "github.com/jackc/pgx/v5/stdlib"
)
+const (
+ startupPingTimeout = 5 * time.Second
+ shutdownTimeout = 25 * time.Second
+)
+
func main() {
+ if err := run(); err != nil {
+ slog.Error("api exited", "err", err)
+ os.Exit(1)
+ }
+ slog.Info("api stopped")
+}