> ## Documentation Index
> Fetch the complete documentation index at: https://internal.softcrum.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Event Ingestion and Tracked Events

> After this ships, a tenant's point of sale can send "this customer paid invoice 889" and get an answer in under 100 ms, with everything downstream happening within seconds.

## Context

This is the front door of the whole platform and the endpoint every integration touches first. Its
latency is the first thing a developer measures, and it sits inline in someone else's checkout, so
the budget is not negotiable.

ADR-013 resolves the tension between "acknowledge fast" and "do a lot of work" with fast-ack:
authenticate, validate, rate-limit, enqueue, answer 202. The heavy path — identity resolution,
segment evaluation, rules matching — happens behind the queue. The cost is that reads are eventually
consistent, and the contract says so explicitly by answering 202 rather than 200.

Retries are guaranteed, because networks fail. Idempotency is therefore not an optimisation but the
correctness condition.

## Scope *(normative)*

* `POST /v1/core/track`, `/identify` and `/batch` with fast-ack.
* `core.tracked_events`, append-only, monthly RANGE partitioned.
* Idempotency key per tenant, deduplicating at the storage fence.
* Rate limiting in Upstash, per tenant and per key.
* The processor: identity resolution, append, and hand-off to segment evaluation and rules.
* Retention by plan with export to Storage before dropping a partition.
* `core.event.tracked` on the outbox.

## Non-scope *(normative)*

* Segment evaluation itself — FS-CORE-0008.
* Rules matching — FS-LOY-0004.
* Public browser write keys — F1b, with the widget (DEC-D6). F1a is server-side only.
* Segment-style aliases. The canonical path is module-scoped (DEC-D2); a compatibility layer only
  if a real integration demands it.

## Behaviour *(normative)*

1. The endpoint does five things and nothing else: authenticate, Zod-validate, rate-limit, enqueue,
   answer **202** with the event id. p95 \<100 ms. FORBIDDEN: any business logic on this path.
2. The response is **202, never 200**. The status code is the contract that processing has not
   happened yet, and it is what stops an integrator from writing a read-after-write assumption.
3. `idempotency_key` is unique per tenant. A duplicate returns 202 with the original event id and
   creates **no** second row and no second downstream effect.
4. `occurred_at` is supplied by the caller with its timezone; `received_at` is ours. Both are
   stored — a point of sale that was offline for an hour must not have its events reordered.
5. The processor is idempotent through `core.processed_jobs` **and** the event's idempotency key.
   Two fences, because this is the path where a duplicate costs points.
6. Identity resolution runs first: an unknown identifier creates an anonymous contact rather than
   dropping the event.
7. `tracked_events` is **append-only** and monthly partitioned. Every query carries the partition
   key — a query by `contact_id` alone is rejected in review and by the CI lint.
8. Event properties are JSONB, validated against the installed taxonomy where one applies
   (FS-CORE-0012) and accepted permissively where none does.
9. Retention by plan: 13 / 25 / 37+ months. A partition is **exported to Storage as compressed
   NDJSON before being dropped**, never simply dropped.
10. Rate limits are published per plan (DEC-D4) and returned in `RateLimit-*` headers.

## Data *(normative)*

| Table                 | Key invariants                                                                                                                                                                                                                                                                                 |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `core.tracked_events` | append-only; `tenant_id`, `cell_id`, `contact_id`; `event_name`; `properties` JSONB; `occurred_at` (caller, tz-aware) and `received_at` (ours); `idempotency_key` unique per tenant; `correlation_id` mandatory; monthly RANGE partition on `occurred_at`; DEFAULT partition alerts on any row |

## API *(normative)*

| Endpoint                 | Class   | Permission          | Budget                            |
| ------------------------ | ------- | ------------------- | --------------------------------- |
| `POST /v1/core/track`    | Runtime | `core.events.write` | **p95 \<100 ms**, 202             |
| `POST /v1/core/identify` | Runtime | `core.events.write` | p95 \<100 ms, 202                 |
| `POST /v1/core/batch`    | Runtime | `core.events.write` | p95 \<200 ms, 202, max 500 events |

## Events *(normative)*

`core.event.tracked` on the outbox after the processor appends. Available as an outgoing webhook,
which is how a tenant mirrors its own event stream back into its warehouse.

## Acceptance criteria *(normative)*

1. `track` p95 \<100 ms under the k6 profile at the documented target throughput.
2. The same `idempotency_key` sent 100 times produces one row and one downstream effect.
3. Events arriving with `occurred_at` an hour in the past land in the correct partition and are
   ordered by `occurred_at`, not by arrival.
4. A batch of 500 is accepted; 501 is rejected with a typed error.
5. End-to-end effect visibility \<5 s p95: event in, points awarded, notification queued.
6. A month-boundary synthetic test leaves the DEFAULT partition empty.
7. Retention export produces a readable NDJSON file with a verified checksum before the partition
   is dropped.
8. **Negative:** a query against `tracked_events` without a time predicate fails the CI lint.

## Execution

Asynchronous event pipeline — the canonical archetype. Endpoint in `backend/api`, processor in
`backend/workers`, retention and partition jobs in `backend/scheduler`. This is TS-003, and
`trackEvent` (FS-LOY-0004 + TS-004) completes it.

## Open questions

| # | Question                                                                                                                                             | Decides | By              |
| - | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | --------------- |
| 1 | Does an anonymous contact's event history survive indefinitely if they never become known, or is it purged on a shorter clock? (PRD open question 2) | Daniel  | before approval |
| 2 | Do we accept events with `occurred_at` in the future, and by what tolerance?                                                                         | Daniel  | before approval |

## Changelog

| Version | Date       | Change        | Why | Author                 |
| ------- | ---------- | ------------- | --- | ---------------------- |
| 0.1.0   | 2026-08-17 | Initial draft | —   | daniel + claude-opus-5 |

## Delivery record

*Not implemented yet.*