Add binary checkpoint serialization for the usage-metering counters
Python · Python · intermediate · greenfield
Adds a compact binary checkpoint codec for the usage-metering tier so a restarting node reloads per-tenant lifetime call totals straight from disk instead of replaying the whole event log. Records are fixed-layout big-endian (raw 16-byte tenant UUID + the lifetime call count), so a checkpoint is a single contiguous buffer with no per-record allocation; serialize and deserialize are exact inverses and the per-checkpoint tenant cap is enforced. Verified round-trip against the metering fixtures.
Runs on the usage-metering tier of a multi-tenant API platform; these lifetime call counters are the billing system's source of truth and feed each tenant's monthly invoice. The busiest tenants on this shard have been live for years and their lifetime call counts have already passed 2 billion.
Requirements
- serialize_checkpoint(counters) encodes a {tenant_uuid: lifetime_call_count} mapping to a compact binary checkpoint, and deserialize_checkpoint(blob) is its exact inverse — every tenant and every count must round-trip unchanged.
- Lifetime call counts only ever increase and must serialize exactly across the service's full operating range: the busiest tenants on this shard have already passed 2,000,000,000 lifetime calls and keep climbing into the tens of billions. Counts are non-negative integers that fit in a signed 64-bit integer (always far below 9.2e18), validated upstream.
- A single checkpoint holds at most 65,535 tenant records (the node's tenant assignment stays well under that cap); exceeding the cap must fail loudly. tenant_uuid is a uuid.UUID.
Files touched
- app/metering/checkpoint.py
--- app/metering/checkpoint.py
+"""Binary checkpoint codec for the usage-metering tier.
+
+Each metering node keeps a lifetime API-call counter per tenant in memory as
+a plain int. This module snapshots those counters to a compact binary
+checkpoint on disk, so a node that restarts reloads exact totals instead of
+replaying the whole event log.
+
+Checkpoint layout (big-endian):
+ header : magic b'UMC1' + uint16 record count
+ record : 16-byte tenant UUID + int32 lifetime call count, repeated
+"""
+
+import struct
+import uuid
+
+MAGIC = b"UMC1"
+
+_HEADER = struct.Struct(">4sH")
+# One record per tenant: the raw 16-byte tenant UUID followed by that
+# tenant's lifetime API-call count.
+_RECORD = struct.Struct(">16si")
+
+
+def serialize_checkpoint(counters):
+ """Serialize {tenant_uuid: lifetime_call_count} to a checkpoint blob.
+
+ tenant_uuid is a uuid.UUID; lifetime_call_count is a non-negative int.
+ """
+ if len(counters) > 0xFFFF: