Backfill and batch

Backfilling pulls historical data (installs, billing cycles, two years of order history) from a platform API into Cowliss. Two primitives make it safe: historical timestamps on any write, and a batch endpoint that takes up to 500 calls per request.

Historical timestamps

identify and track accept an optional timestamp. On a track, it becomes the event’s time in ClickHouse (receivedAt still records when Cowliss saw the call). On an identify, a first-write profile insert is backdated; on an existing profile the original createdAt is kept.

Validation rejects future timestamps (60 s of clock-skew tolerance) and timestamps older than the environment’s event retention window, both with 422.

POST /v1/batch

One request carries one source’s identify and track calls: { sourceId, items }, each item flattened and tagged by type. sourceId is the api source you send through, exactly as on identify and track; the server derives its app. A batch never spans sources, so it never spans apps or environments either:

{
  "data": {
    "sourceId": "src_01j2x8q7v9e3atn5m4kd7yz0bp",
    "items": [
      { "type": "identify", "identifiers": { "userId": "user_1" }, "traits": { "plan": "pro" } },
      {
        "type": "track",
        "identifiers": { "userId": "user_1" },
        "event": "subscription_started",
        "timestamp": "2026-03-01T00:00:00.000Z",
        "messageId": "stripe-evt_0001"
      },
      {
        "type": "identify",
        "identifiers": { "userId": "user_1", "anonymousId": "anon_9f3…" },
        "traits": {}
      }
    ]
  }
}

There is no alias item type: an identify carrying two identifiers links them, which is everything an alias ever did.

The response is positionally aligned with the request: one result per item, in order:

{
  "data": {
    "results": [
      { "ok": true, "type": "identify", "id": "usr_…" },
      { "ok": true, "type": "track", "id": "evt_…", "deduped": false },
      { "ok": false, "error": { "code": "trait_validation_failed", "message": "…" } }
    ]
  }
}

Item-level failures never fail the batch: one bad row reports the same error code the single-call endpoint would have returned, and the rest land. Request-level problems (malformed JSON, an array over 500 items, a bad envelope) are still request-level 400/422s. An item whose identifiers name two known profiles is not a failure: Cowliss merges the profiles, then applies the item, same as the single-call endpoints. See Identity for the merge rules.

Quiet writes

Batch writes land in storage but are quiet in exactly one sense: they skip raw-event journey triggers, so importing two years of history never fires automations at long-gone users. Segment membership does still recompute for imported data (segments tell the truth), so a segment-triggered journey can fire for an imported cohort, and a segment created after an import catches up over existing history.

Rule of thumb: raw-event triggers are the live funnel; segment triggers are cohorts, including imported ones.

Safe re-runs

Give every track item a stable messageId derived from the source system (stripe-evt_0001, not a random UUID). Then a crashed import is simply restarted: replays dedupe per item ("deduped": true) instead of double-writing. The request-level Idempotency-Key header does not apply to batch items. One shared key would collapse them onto the first.

The SDK exposes the same call as cow.batch(…); it requires explicit messageIds on track items precisely so re-runs are safe.

Cost

Backfilled events are metered like any other customer-written event, because they are rows you wrote. System events Cowliss writes itself are never metered. See Billing.