Dataset cloud sync

Recorded episodes sync automatically from the robot to cloud storage. You configure destinations (where), routing rules (which episodes go where), and retention (what happens to the local copy). The daemon handles transfers, retries, and resume.

Per episode and destination, sync progresses:

reported → granted → syncing → synced → verified

• reported — metadata indexed; the episode appears in GET /v1/episodes
• granted — upload URLs issued for a destination
• syncing / synced — transfer in progress / all files uploaded
• verified — uploaded checksums matched the episode's manifest (no read-back from the bucket required)

Transitions emit webhooks: episode.synced, episode.verified, and on transfer errors episode.failed.

Data custody

Sync is designed so Sentinel moves and indexes your data without access to it:

• The platform stores episode metadata — the index you query. The recorded bytes go only to your storage.
• Robots hold no storage credentials. Every transfer is a per-episode grant: presigned URLs issued by your grant provider, valid for about an hour, covering exactly the files in the episode's manifest.
• Verification compares upload checksums against the manifest — nothing is read back, so write-only storage works fully.

Configure destinations

GET    /v1/destinations
POST   /v1/destinations
DELETE /v1/destinations/{id}
POST   /v1/destinations/{id}/check

Requires the admin scope.

A destination is a type: "external" grant provider: an HTTPS endpoint you run that issues presigned upload URLs into your own storage. Avea relays the URLs to the robot and never has access to the data. Destination types backed directly by S3 (an IAM role you grant, or S3-compatible endpoints) are coming later.

curl -X POST https://api-prod.avearobotics.com/v1/destinations \
  -H "Authorization: Bearer ak_..." \
  -H "Content-Type: application/json" \
  -d '{
    "type": "external",
    "url": "https://grants.example.com/grants",
    "retention": "until_all_verified_plus_72h"
  }'

• The response carries a signing secret (dsec_...) once at creation — the key for verifying forwarded grant requests.
• DELETE revokes: no new grants are issued, synced data is untouched, routing rules pointing at it stop matching.
• POST /v1/destinations/{id}/check probes the endpoint for conformance — see grant provider.

Route episodes to destinations

GET    /v1/routing-rules
POST   /v1/routing-rules
DELETE /v1/routing-rules/{id}
PUT    /v1/routing-rules/order

Requires admin. First matching rule wins, in the order set by PUT /v1/routing-rules/order.

curl -X POST https://api-prod.avearobotics.com/v1/routing-rules \
  -H "Authorization: Bearer ak_..." \
  -H "Content-Type: application/json" \
  -d '{
    "destination_id": "dst_...",
    "match": { "task_id": "d81f42a7-...", "has_autonomous": "true" }
  }'

Match fields: robot_id, task_id, recording_session_id, episode_seq, episode_id, outcome, robot_platform, has_autonomous. Values are strings or arrays of strings; {} matches everything.

With no rules and exactly one destination, every episode routes there by default — registering a single destination is enough. Once any rule exists, rules alone decide: an episode that matches no rule stays local, and adding a rule later syncs the backlog. Multiple destinations with no rules route nothing — add rules to disambiguate.

Object keys are chosen by your grant provider and recorded per replica as object_key when the provider reports them — routing decides where an episode goes, your provider decides what it's called.

Retention and local pruning

Set per destination:

Pruning emits the episode.pruned webhook. Unverified episodes are never pruned, including under disk pressure — pressure shows up in sync health instead.

Track sync from the API

Episode rows include replicas, one entry per destination:

{
  "id": "3e1b...",
  "recorded_outcome": "success",
  "replicas": [
    {
      "destination_id": "dst_...",
      "state": "verified",
      "grant_id": "g_01H...",
      "error": null,
      "updated_at": "2026-08-10T21:04:11Z"
    }
  ]
}

GET /v1/episodes filters on it: ?sync_state=unverified|synced|verified, optionally &to={destination_id}. unverified = reported but lacking a verified replica.

Monitor sync health

GET /v1/sync-health/robots

Requires data:read. One row per robot: episodes_reported, unverified, unverified_over_72h, bytes_unverified, and local_missing (local files disappeared before a verified sync — usually hand-moved data). Alert on unverified_over_72h and local_missing.

On the robot, sentinel data status shows the daemon's view and sentinel data push triggers a sync pass immediately.