Bill stored data in the daily usage export
Go · Go · intermediate · modification
Storage is now billed alongside API calls. The exporter gets a `StorageMeter` dependency (the usage database already implements `BytesByTenant` for the dashboard), and `Day` appends one `storage_bytes` line per tenant with data, after the API call lines. Tenants with nothing stored are skipped, the same as tenants with no calls. The provider does the GB-month conversion, so we send raw bytes. `main` wires the same database handle as the storage meter.
Go 1.22 batch job (`cmd/exporter`), run once a day shortly after midnight UTC by a scheduler that reruns it when it exits non-zero. `usagedb.DB` (existing, unchanged) implements `CallsByTenant(ctx, day time.Time) ([]usagedb.TenantCalls, error)` (`TenantID string`, `Calls int32`) and `BytesByTenant(ctx, day time.Time) ([]usagedb.TenantBytes, error)` (`TenantID string`, `Bytes int64`); both return at most one row per tenant, ordered by tenant id, with non-negative counts, for the UTC day starting at `day` (always midnight UTC). `billing.Client.Upload(ctx, day string, lines []export.Line) error` (existing) JSON-encodes the lines and posts them as that day's usage; uploading the same day again replaces the earlier upload. The provider rejects the whole upload when any line has a negative `units`. `Line` is used only by this package and `billing.Client.Upload`, and `main` is the only place an `Exporter` is constructed.
Requirements
- The daily usage export for a UTC day gains a second metric, `storage_bytes`: one line per tenant whose stored data at the end of that day, as reported by `BytesByTenant`, is greater than zero. A tenant holds between 0 bytes and 10 TB (10,000,000,000,000 bytes).
- `units` on a `storage_bytes` line is the tenant's exact stored byte count. The billing provider accepts `units` as a non-negative integer up to 2^63 - 1 and converts bytes to GB-months on its side, so the export does no rounding or unit conversion.
- The existing `api_calls` lines are unchanged: one line per tenant with at least one call that day. A tenant's daily call counter is a 32-bit column, and the API gateway caps a tenant at 50 million calls a day.
- The `api_calls` lines come first, then the `storage_bytes` lines, each group in the order its query returns it. If either query fails, `Day` returns an error and nothing is uploaded; the scheduler reruns the job.
Files touched
- internal/export/usage.go
- cmd/exporter/main.go
--- internal/export/usage.go
}
-const MetricAPICalls = "api_calls"
+const (
+ MetricAPICalls = "api_calls"
+ MetricStorageBytes = "storage_bytes"
+)
type CallCounter interface {
}
-type Exporter struct {
- Calls CallCounter
+type StorageMeter interface {
+ BytesByTenant(ctx context.Context, day time.Time) ([]usagedb.TenantBytes, error)
}
-// Day returns the usage lines for the UTC day that starts at day.
+type Exporter struct {
+ Calls CallCounter
+ Storage StorageMeter
+}
+
+// Day returns the usage lines for the UTC day that starts at day:
+// API calls first, then stored bytes at the end of the day.
func (e *Exporter) Day(ctx context.Context, day time.Time) ([]Line, error) {
date := day.Format(time.DateOnly)
return nil, fmt.Errorf("count api calls for %s: %w", date, err)
}