Artifacts

Artifacts are point-in-time, org-scoped exports of your First-Mile data. It generates asynchronously; once it reaches the completed state, mint download URLs for the file(s) — short-lived signed URLs, or a persistent revocable link when you need one that outlives the week. Today every artifact is a single GeoParquet bundle (type: relational_full, format: geoparquet); SQLite, XLSX, and non-geo Parquet are on the roadmap.

How an artifact is created

An artifact can be requested on demand at any time, or created automatically at ingestion as part of an upload. Both paths produce the same artifact resource and walk the same lifecycle — they differ only in what triggers generation and how the scope is set.

PathTriggerScopeBest for
On-demandPOST /artifacts with your own filtersThe single scope filter you supply (see Scope model)Periodic exports, regulatory packs, arbitrary slices
Ingestion fast-pathcreate_artifact: true on the upload requestExactly the locations from that upload (assets.task_id)"Give me a bundle of what I just uploaded"
  • On-demand — call POST /artifacts with exactly one scope filter and you get back a queued artifact (202). Poll it, or subscribe to a webhook, until it reaches a terminal state. This is the endpoint the rest of this page documents.
  • Ingestion fast-path — set create_artifact: true on POST /ingestions/assets and the ingestion worker auto-dispatches a generateArtifact task scoped to that upload (filter: assets.task_id == <task>) once it finishes — no second call, no filter to author. The upload's task carries the resulting artifact_id; you can also find it with GET /artifacts?filter[assets.task_id]=eq:<task>. See the ingestion fast-path docs for the metric requirement and the optional dashboard-link (signed_url_ttl_seconds) behaviour.

Either way, the fast path does not pre-mint download URLs — once the artifact is completed, mint signed URLs exactly as below.

Use cases

  • BI / data-warehouse handoff — pull a self-describing relational snapshot of your locations, assets, metrics, facilities, sheds, plots, and assessment slices into DuckDB, GeoPandas, BigQuery, or your warehouse of choice.
  • Regulatory submissions — produce a deterministic, hash-stamped bundle (sha256) tied to a specific filter window for due-diligence packs.
  • Dashboard handoff — every signed-URL response carries links.dashboard_url, a deep link that opens the artifact directly in Epoch's dashboard.
  • Event-driven pipelines — subscribe to artifact.completed / artifact.failed webhooks and trigger downstream processing the moment a bundle is ready.

Lifecycle

Artifacts are asynchronous. The status field on every artifact resource walks one of these paths:

queued → processing → completed → purged
                   → failed
                   → cancelled
  • queued — the artifact has been accepted and is waiting to start.
  • processing — the artifact is being generated.
  • completed — terminal success. storage_objects, size_bytes, sha256, and row_counts are populated; signed URLs can now be minted.
  • failed — terminal failure. error_message carries the reason. You can retry with the same idempotencyKey.
  • cancelled — terminal, caller-initiated via DELETE /artifacts/:id. Only valid from non-terminal states; cancelling an already-terminal artifact returns 409. You can reuse the same idempotencyKey afterwards.
  • purged — the artifact completed, and its files have since aged out of retention. The record survives as a tombstone carrying purged_at, action_required: "regenerate", and the original file metadata; downloads no longer resolve and signed URLs can no longer be minted (409 artifact_purged). Its idempotencyKey is freed, so you can regenerate under the same key. The record itself is removed after a further window, after which the id returns 404.

Retention never deletes the files behind a download link that is still in date and not revoked, so a link you handed someone works for its full lifetime. See the Retention section of the Artifacts guide for the windows.

API response field vs. prose name. In raw API responses (GET /artifacts/:id, GET /artifacts, list filters, etc.) the success state is literally "status": "succeeded". Webhook payloads (artifact.completed) and this documentation use "completed". They refer to the same terminal state. If you are filtering or matching on the response field, use succeeded. If you are subscribing to a webhook, use completed.

See task-and-entity-statuses.md for general lifecycle conventions.

Manifest

Every GeoParquet file carries a self-describing manifest embedded in the file's key_value_metadata under the manifest key. Open the parquet and you can recover, without further API calls:

BlockContents
name, description, versionHuman-readable identification of the bundle.
data_lineage.sourcesUpstream systems and what each contributed.
data_lineage.normalization_stepsOrdered list of transformations applied to produce the bundle.
data_lineage.model_versionsPinned model / pipeline versions.
data_lineage.generated_atISO-8601 generation timestamp.
data_lineage.artifact_idThe artifact's UUID.
usage_constraintsLicense, validity statement, redistribution restrictions, CRS notes.
tablesPer-table primary_key, foreign_keys, description, and column_descriptions.
linked_entitiesParent → child relationships with cardinality.

Rows from every logical table land in one GeoParquet file with a table_name discriminator column; split or filter on that column to recover individual tables. Geometry is OGC WKB in OGC:CRS84 (longitude, latitude) — check usage_constraints.restrictions before reprojecting.

Scope model (current limitations)

An artifact is scoped by exactly one filter — there is no whole-org export, and arbitrary or compound filters are not currently supported. The create request must carry a single filters entry with condition: "eq" and a single value, using one of:

AttributeValueScopes to
assets.task_id (alias asset_task_id)task UUIDOne ingestion/upload.
facility_idfacility id (text)One facility + everything beneath it.
location_idfacility / shed / plot public idOne location + everything beneath it.
supplier_idsupplier UUIDA supplier's locations + everything beneath them.
namelocation normalised nameLocations whose normalised name matches + everything beneath them.

How scoping resolves

  • The chosen filter resolves — org-scoped and index-backed — to a seed set of locations.
  • For every scope except task_id, the seed is expanded down the hierarchy (facility → shed → plot), so scoping by a facility / supplier / name pulls in that facility's sheds and their plots. A task_id scope is anchored on the upload itself and already enumerates its whole tree.
  • metrics are reduced to the latest validation per (location, assessment_type); the per-plot / per-shed assessment slices (plot_for, shed_for, …) are reduced to the latest assessment per plot/shed (so they are 1:1 with the plots/sheds in scope).
  • task vs entity scope: a task_id scope is upload-scoped (one asset row; a location shared with another upload does not drag in that upload's rows). The facility_id / location_id / supplier_id / name scopes are entity-scoped — a location uploaded several times contributes every distinct asset row across those uploads.

Not currently supported (returns 400): more than one filter, filterOverrides, conditions other than eq, list/array values (in), and any attribute outside the table above. The error codes are scope_filter_required, unsupported_scope_filter, and filter_overrides_unsupported. These were intentionally narrowed while the export pipeline was reworked to run indexed, scope-resolved queries (replacing a path that could full-scan the platform-wide plot table); broader filtering is a planned follow-up.

Partitioning

The partitioning field on the create request is null today — every artifact is a single bundle. The response shape is forward-compatible:

  • storage_objects[] already enumerates every file in the bundle (one entry today; many entries once hive partitioning ships).
  • signed-urls responses already return files[] — clients should iterate, not pluck a top-level URL.
  • When hive partitioning lands, partitioning will surface a { scheme: "hive", keys: [...] } block and storage_objects[] will expand accordingly.

Build clients against the array shapes today and partitioned outputs will be a no-op on the consumer side.

Consuming the parquet

Once a signed URL is minted (see Signed URLs → Mint Signed URLs), the file can be read directly by any GeoParquet-aware tool. A few common recipes:

DuckDB (httpfs)

INSTALL httpfs; LOAD httpfs;
INSTALL spatial; LOAD spatial;
SELECT table_name, count(*)
FROM read_parquet('https://...signed-url...')
GROUP BY 1;

DuckDB-Wasm — the same read_parquet call works in the browser; pass the signed URL straight in.

GeoPandas

import geopandas as gpd
gdf = gpd.read_parquet("https://...signed-url...")
plots = gdf[gdf["table_name"] == "plots"]

QGIS — open the signed URL directly as a vector layer (QGIS ≥ 3.28 supports remote GeoParquet via GDAL).

Dashboard — open links.dashboard_url from the mint response; it opens the artifact in Epoch's dashboard with no further setup. The link expires alongside the signed download URLs, and is returned only for short-lived (≤ 7 day) mints.

Sharing with a person — for a link that survives longer than a week, request a lifetime beyond 7 days and hand over the URL from files[]. It stays valid for up to 180 days, and you can retune or revoke it at any point. See Signed URLs.

Webhooks

Two events fire on terminal transitions:

EventWhenPayload data keys
artifact.completedArtifact reaches the completed state.artifactId, type, format, partitioning, fileCount, status: "completed", sizeBytes, sha256, rowCounts, completedAt
artifact.failedArtifact reaches the failed state.artifactId, type, format, status: "failed", errorMessage

The webhook payload does not contain download URLs. Receivers must call POST /artifacts/:id/signed-urls themselves to mint a URL. Webhook deliveries can be replayed, retried, or logged downstream, so embedding a signed URL would widen the surface area of any one of those copies. See webhooks.md for delivery semantics and HMAC verification.

Scopes

EndpointRequired scope
POST /artifactsartifacts:write
GET /artifactsartifacts:read
GET /artifacts/:idartifacts:read
DELETE /artifacts/:idartifacts:write
POST /artifacts/:id/signed-urlsartifacts:sign
GET /artifacts/:id/signed-urlsartifacts:read

artifacts:sign is intentionally separate from artifacts:write: a key that can read or generate artifacts does not implicitly receive the ability to mint download URLs. See scopes.md for general scope rules.

Org scoping

Artifacts are strictly scoped to your organization. A parent-org caller sees their own artifacts plus those of direct child orgs; the x-acting-as header narrows the request to a single child. Idempotency keys are scoped per-org and are never shared between a parent and a child — replays only resolve against the org that originally created the artifact.