Ingest

The ingest is the server-side brick of Dotyc, mounted on your infrastructure: your domain, your endpoint, your secrets. Everything the emitter is forbidden to know lives here — validation, identity verification, wiring, adapters, and the durable outbox.

Because the endpoint is first-party (same origin as your app), ingestion is invisible to ad-blockers with no workarounds.

Two layers: pure core + fetch sugar

The product is a pure, typed core with zero framework dependency and no imposed I/O:

import { createDispatcher, toHandler } from 'dotyc/ingest'
import { clerkIdentity } from '@dotyc/clerk/server'

const dispatcher = createDispatcher({
  catalog,
  wiring,
  identity: clerkIdentity(),     // resolves session/JWT → verified identity
  store: redisStore(env.URL),    // required with durable events; also backs identify/replay dedupe
  strict: false,                 // true = unmatched event is an error (default: dev warning)
  onError: (err, ctx) => { /* report: log, Sentry, … */ },
})

// Mountable anywhere: custom route, queue worker, cron, tests
await dispatcher.ingest(envelope, { request })

The store is the pluggable brick that persists pipeline state: the durable outbox and retry bookkeeping, plus the identify-dedupe and replay-dedupe caches. memoryStore() for dev, official Redis / Postgres / KV drivers for production. It is required as soon as the catalog declares durable events. See Delivery guarantees.

On top, toHandler wraps the dispatcher in a standard (Request) => Response handler — about twenty lines of comfort. Since modern frameworks (Next, Hono, Remix, SvelteKit, Bun, Deno, Elysia, …) all speak Web Request/Response, one handler covers them all:

export const POST = toHandler(dispatcher, {
  origin: ['https://app.example.com'],   // CORS / origin check
  maxBodySize: '256kb',
  guard: async (request) => { /* rate limiting, anti-spam */ },
  waitUntil,                             // serverless: fan-out finishes after the response
})

The core is the contract; the handler is sugar, never a required dependency. If your framework is exotic or your events arrive from a queue, call dispatcher.ingest() directly.

On serverless (Vercel, Lambda), pass the platform's waitUntil so fan-out and retries can complete after the response is sent, and keep all state in the store — never in instance memory.

The pipeline

Each incoming envelope (one POST = N events) goes through, in order:

1. Parse the envelope.
2. Resolve the schema. Each event's schemaHash is matched against the committed schema history: current, legacy (a known older version), or unknown. A version mismatch never blocks an event. See Schema evolution.
3. Validate — strict. An event on the current schema is checked strictly against it; invalid means rejected, surfaced via onError, never dispatched. A legacy event is validated against its own schema and dispatched flagged. A trusted event arriving over HTTP is rejected here too — it may only come through direct().
4. Resolve identity. The provider verifies the session or JWT and overrides the emitter's claim; new anonymousId ↔ userId links are signaled to adapters via their identify hook, deduped through the store. See Identity.
5. Apply computed fields from the event definitions — enrichment happens once, here, so every consumer sees the same data.
6. Persist durables. Durable events are written to the outbox (via the store) before anything else happens to them.
7. Ack. The response reports each event's fate — accepted | rejected | legacy — so the emitter's durable queue can release (or retry) exactly what it should.
8. Match each event against the wiring's declarative criteria.
9. Fan out — parallel, isolated. See below.
10. Mark done / retry. For durable events, each consumer's success is tracked in the store; failures are retried with backoff, and exhausted retries land in dead-letter status plus onError.

Error philosophy: isolation always, retry where declared

Fan-out is pure, parallel, and isolated:

• An adapter that throws affects neither the other adapters nor the response sent to the emitter.
• All failures — adapter errors, rejected invalid events, dead-lettered durables — flow to the single onError hook, where you plug your logging or error tracking.
• Retry follows the delivery class. A best-effort event is dispatched once; a failed handle is reported, not replayed. A durable event is retried per consumer, with backoff, from the outbox until it succeeds or dead-letters. You choose per event, in the catalog — not globally.

An event that matches no consumer is a warning in dev and silent in prod; set strict: true to make it an error instead.

QoS: durables first, noise shed

The pipeline never lets high-volume best-effort traffic starve business facts:

• Durable and best-effort events arrive in separate envelopes (the emitter never mixes them), so the ingest can prioritize durable envelopes.
• Under pressure, durables are processed first and best-effort events are shed — dropping them is their contract; degrading everything is not.
• For infrastructure-level separation, a documented pattern: mount the same dispatcher on two routes (say /ingest and /ingest/critical) with different rate limits, scaling, or WAF rules — free, since toHandler is just a function.
• The most critical events never enter this pipe at all: trusted facts arrive in-process via direct().

Endpoint security

The handler ships the basics — payload size limit, configurable origin/CORS check, and a guard hook for your own rate limiting or anti-spam. The ingest also dedupes replayed events on their eventId over a TTL window. Everything beyond that (WAF, infrastructure-level throttling) belongs to your stack, deliberately.

Deployment targets

• Self-hosted (the default). Mounted in your app, your domain, your adapters — as shown above.
• Hosted by Dotyc (planned). For full-client apps with no backend: the exact same brick, operated by us. The identity model already supports this — JWT verification works without a server session.