Add the export delivery poller
Node · Node · advanced · greenfield
Adds the export delivery poller behind a `startExportPoller()` entry point, wired on an interval with per-job error isolation so one bad delivery target can't stall the queue. Failed jobs stay pending and get retried on a later tick. Returns a disposer so tests and shutdown hooks can stop it cleanly.
Back-office worker process. Delivery targets are customer SFTP/webhook endpoints; duplicate deliveries are customer-visible incidents.
Requirements
- Add `startExportPoller()`: every 2 seconds, pick up pending export jobs and deliver them. It returns a stop function that halts future polling.
- The store API (all async): `fetchPendingExports()` resolves with every job whose status is `pending`; `deliverExport(job)` uploads the export to the customer's delivery target; `markDelivered(id)` flips the job to `delivered`. A job stays `pending` until `markDelivered` resolves.
- Each pending export must be delivered exactly once. A delivery can take well over the 2-second poll interval — large exports routinely run 10+ seconds — and this poller is the only consumer of the queue, so exactly-once is this file's job, not the store's.
- One failed delivery must not block the others: log the error with the job id and let the job be retried on a later poll (it is still `pending`).
- No job may be processed concurrently with itself.
Files touched
- src/jobs/exportPoller.js
--- src/jobs/exportPoller.js
+const {
+ fetchPendingExports,
+ deliverExport,
+ markDelivered,
+} = require('../stores/exportStore');
+const logger = require('../lib/logger');
+
+const POLL_INTERVAL_MS = 2000;
+
+function startExportPoller() {
+ const timer = setInterval(async () => {
+ const pending = await fetchPendingExports();
+ for (const job of pending) {
+ try {
+ await deliverExport(job);
+ await markDelivered(job.id);
+ } catch (err) {
+ logger.error({ jobId: job.id, err }, 'export delivery failed; job stays pending');
+ }
+ }
+ }, POLL_INTERVAL_MS);
+
+ return function stopExportPoller() {
+ clearInterval(timer);
+ };
+}
+
+module.exports = { startExportPoller };
+