Atlas API
A high-performance timeseries data platform for storing, querying, and streaming arbitrary timeseries data. Built with Go and TimescaleDB.
Quick Start
Reading Data
# First-time API users: acquire the "public" tag (Atlas UI does this automatically)
curl -X POST https://your-domain.com/api/tags/acquire/ \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"tag_name": "public"}'
# List available datasets
curl -H "X-API-Key: YOUR_API_KEY" https://your-domain.com/api/datasets/
# Search by name or description
curl -H "X-API-Key: YOUR_API_KEY" \
"https://your-domain.com/api/datasets/?search=bitcoin"
# Get data from a dataset
curl -H "X-API-Key: YOUR_API_KEY" \
"https://your-domain.com/api/rows/?dataset=1&limit=100"
Creating Data
# 1. Create a dataset
curl -X POST https://your-domain.com/api/datasets/ \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "my_btc_prices", "description": "Bitcoin hourly prices"}'
# 2. Insert data (use the returned dataset ID)
curl -X POST https://your-domain.com/api/rows/bulk_insert/ \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"dataset": 1, "rows": [
{"timestamp": "2025-12-16T10:00:00Z", "values": {"price": 104500}},
{"timestamp": "2025-12-16T11:00:00Z", "values": {"price": 104600}}
]}'
# 3. Share it by adding the "public" tag
curl -X POST https://your-domain.com/api/datasets/1/tags/ \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"tag_name": "public"}'
Authentication
All API requests require authentication using an API key passed in the X-API-Key header.
curl -H "X-API-Key: YOUR_API_KEY" https://your-domain.com/api/datasets/
Never expose your API key in client-side code or public repositories.
Health Check (No Auth Required)
Check if the service is running and connected to the database.
{
"status": "healthy",
"time": "2025-12-16T10:30:00Z",
"database": "connected"
}
Access Control & Roles
Atlas uses a tag-based authorization system combined with dataset ownership. Understanding this system is essential for sharing and accessing data.
Creators: You own what you create. Assign tags to share datasets.
Consumers: Acquire tags to access datasets. The public tag is free.
Rule: Matching tags = read access. Ownership = full control.
Ownership
When you create a dataset, you become its owner with full control: read, write, delete, and manage access tags. New datasets are private by default—only visible to the owner until tags are assigned.
Tag-Based Access
Users can read a dataset if they have at least one tag that matches a tag on the dataset. No matching tags means no access (unless you're the owner).
Tag Types
| Tag Type | How to Acquire | Use Case |
|---|---|---|
| Public Tags | Any user can acquire freely | public, test - Open datasets for everyone |
| Private Tags | Require a secret key to acquire | premium, tiingo - Restricted data access |
Permissions Matrix
| Action | Owner | Matching Tag | No Match |
|---|---|---|---|
| Read dataset / rows | ✅ | ✅ | ❌ |
| Write / Update / Delete | ✅ | ❌ | ❌ |
| Manage tags | ✅ | ❌ | ❌ |
List & Search Datasets
Text Search
List datasets with optional text search. The search parameter performs case-insensitive substring matching against the dataset name and description.
| Parameter | Type | Description |
|---|---|---|
search |
string | Substring to match against name or description (case-insensitive) |
limit |
integer | Max results to return (default: 250) |
offset |
integer | Number of results to skip (for pagination) |
Examples
# Search for datasets containing "bitcoin" in name or description
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/?search=bitcoin"
# Search for datasets with "price" anywhere in the text
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/?search=price"
# Get first 10 results
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/?search=market&limit=10"
The web interface search bar uses this endpoint—substring matching against name and description.
Metadata Search
Deprecated compatibility route; prefer unified discovery. Search datasets by metadata field values. Each query parameter (except limit and offset) is treated as a metadata filter with exact matching.
| Parameter | Type | Description |
|---|---|---|
{key} |
any | Metadata field name and value (exact match) |
limit |
integer | Max results to return (default: 250) |
offset |
integer | Number of results to skip |
Examples
# Find all datasets from Tiingo
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/search/?source=tiingo"
# Find BTC/USD data with 1-minute frequency
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/search/?ticker=btcusd&frequency=1min"
# Find premium-tier datasets
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/search/?tier=premium"
At least one metadata filter is required. Use /api/datasets/ to list all datasets.
Dataset Stats
Legacy compatibility route (use dataset search for new integrations). Batch row-count and time-range summary for up to 100 datasets in one call. Powers dashboards that would otherwise need a per-dataset freshness probe. Results come from the hourly dataset_stats continuous aggregate.
| Parameter | Type | Description |
|---|---|---|
ids |
integer list | Dataset ids, comma-separated (?ids=1,2,3), repeated (?ids=1&ids=2), or mixed. Required; maximum 100 per request. |
Response
{
"count": 2,
"next": null,
"previous": null,
"results": [
{
"dataset_id": 1,
"row_count": 145230,
"min_timestamp": "2026-01-01T00:00:00Z",
"max_timestamp": "2026-01-12T18:30:00Z"
}
]
}
Example
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/stats/?ids=1,2,3"
An id is omitted from results (never an error for the batch) when your key cannot read that dataset or the aggregate has no bucket for it yet — current-hour buckets, historical repair and pending invalidations can leave stats absent or stale. Treat absence as "stats unknown" and fall back to a per-dataset probe. next/previous are always null; count equals the number of returned results.
Dataset metadata and discovery
GET /api/datasets/ is the canonical paginated search. Without filters it returns the first page of datasets your key can read. GET /api/datasets/{id}/ returns one dataset without rows. Both return the flat fields below; no separate metadata GET is needed.
{
"dataset_id": 123,
"name": "tiingo_btcusd_1min",
"description": "One-minute BTC/USD candles",
"row_count": 120000,
"total_size_mb": 24.6,
"data_updated_at": "2026-10-19T16:00:12Z",
"earliest_timestamp": "2026-07-28T08:00:00Z",
"latest_timestamp": "2026-10-19T15:59:00Z",
"metadata_updated_at": "2026-10-19T16:05:00Z"
}
Illustrative values. Existing id (equal to dataset_id), metadata (owner JSON), owner_key_id, created_at and updated_at remain in actual responses. Old null descriptions read as ""; the database is not rewritten. Existing definition timestamp representations remain unchanged. New timestamp fields use UTC RFC3339.
| Field | Meaning |
|---|---|
| row_count | Stored records, not scalar values. Periodic aggregate snapshot; may lag writes. |
| total_size_mb | Logical uncompressed JSON size divided by 1,000,000. Not physical disk size or download transfer size. Each record is UTF-8 {"timestamp":"YYYY-MM-DDTHH:MM:SS.ffffffZ","values":JSONB_TEXT}, with PostgreSQL JSONB normalized text for values. No IDs, created_at, array separators or HTTP envelopes. The method is atlas_record_bytes v1. |
| data_updated_at | Latest observed successful content-changing statement (insert, correction or delete), independent of event time. Rolled-back changes and unchanged upserts do not advance it. Changes only to row created_at are bookkeeping. Not an exact commit timestamp. Tracking begins with migration 016; previously untracked history stays null until a change occurs. |
| earliest_timestamp / latest_timestamp | Minimum/maximum stored record timestamps in the snapshot. Synthetic keys retain their meaning; these bounds do not prove freshness or gap-free coverage. |
| metadata_updated_at | Time the shared metric snapshot was taken, not the dataset definition's edit time. Content-change time is read independently and can be newer. |
All measurement fields are present. In GET detail/search and global metadata responses, null counts/sizes mean that dataset or global summary has not yet been included in a published metadata snapshot. Creation/update responses use null metric placeholders; use GET to read snapshot values. After a snapshot, a dataset with no materialized hourly buckets has zero count/size and null date bounds, even when its raw data is waiting for aggregation. These convenience summaries may temporarily undercount new data or backfills. A failed refresh retains the last successful snapshot. Check metadata_updated_at for age; there is no guaranteed five-minute freshness SLA.
Filters, sorting and pagination
All criteria are ANDed with access control. IDs within ids are ORed, then intersected with every other filter. Unknown statistics do not remove datasets unless your filter excludes them.
| Parameter | Behavior |
|---|---|
| search | Case-insensitive substring across name OR description. Legacy SQL LIKE wildcards (%) and (_) remain supported; use __contains for literal matching. |
| ids | Comma-separated positive IDs, maximum 1,000 distinct IDs. Duplicates are collapsed. Empty or invalid lists return 400. |
| name / description | Exact equality by default; name__contains / description__contains are case-insensitive literal substrings. Use name=exact_name for exact lookup without rows. |
| Field filters | Every agreed field supports equality and __gt, __gte, __lt, __lte. Numeric values must be nonnegative and finite; dates must be RFC3339 (URL-encode timezone plus signs). Text comparisons other than contains are case-sensitive. |
| field__isnull | true or false, e.g. row_count__isnull=true. Other comparisons do not match nulls. |
| metadata.KEY | Exact match on a top-level owner-defined metadata value using its PostgreSQL text representation. Multiple keys are ANDed. No business meaning is inferred from names. |
| ordering | Any agreed field: dataset_id, name, description, row_count, total_size_mb, data_updated_at, earliest_timestamp, latest_timestamp, metadata_updated_at. Prefix - for descending. Nulls always last; dataset_id ascending breaks ties. Default remains created_at descending, with ID tie-break. |
| limit / offset | Default 250 / 0; maximum limit 10,000. Legacy normalization remains: nonpositive limit becomes 250, excessive limit is capped, negative offset becomes 0. Noninteger values return 400. |
Each parameter occurs once. Unknown fields/operators or invalid values return 400. The envelope remains {count,next,previous,results,data,total}; data/total mirror results/count. Count is matching readable datasets, not stored rows. Count and results share a snapshot within one request; separate pages are not a frozen snapshot as data changes.
curl -G -H "X-API-Key: YOUR_KEY" https://your-domain.com/api/datasets/ \
--data-urlencode 'ids=123,456' \
--data-urlencode 'search=btc' \
--data-urlencode 'metadata.source=tiingo' \
--data-urlencode 'row_count__gte=1000' \
--data-urlencode 'ordering=-data_updated_at' \
--data-urlencode 'limit=20'
Global Atlas metadata
GET /api/datasets/all/ returns one global summary, not a dataset list. It requires an authenticated API key with an unexpired public tag; other keys receive 403. The totals deliberately include private datasets, but expose no individual private dataset definitions. Dataset search/detail remain subject to their normal dataset permissions.
{
"dataset_count": 1067,
"row_count": 9251369,
"total_size_mb": 1894.2,
"data_updated_at": "2026-10-19T16:04:52Z",
"earliest_timestamp": "2018-01-01T00:00:00Z",
"latest_timestamp": "2026-10-19T16:04:00Z",
"metadata_updated_at": "2026-10-19T16:05:00Z"
}
Illustrative values. Counts/size are totals; dates are the outer bounds across all datasets. Global data_updated_at is the latest tracked content change, including removal of a populated dataset, as captured by the summary. Before the initial snapshot, measurements are null. No input/output traffic, user counts or usage history is included.
Compatibility and creation
Canonical GET /api/datasets/ now returns 400 for unknown query parameters; the previous list handler ignored them. Remove unused parameters and use metadata.KEY for owner-metadata filters. The legacy search adapter retains its separate syntax. Numeric equality is exact, not rounded: use range operators when searching an approximate size.
Creation now requires both name and description. POST /api/datasets/ rejects missing/blank names and missing/null/non-string descriptions with 400. An empty description is valid: {"name":"new_feed","description":""}. Update callers that previously omitted description. No owner backfill is required.
/api/datasets/search/ is a deprecated compatibility adapter to the same search implementation. It retains legacy syntax: every parameter except limit/offset is an exact owner-metadata filter, even keys named ids or ordering. Multiple filters now intersect. Migrate /datasets/search/?source=tiingo to /datasets/?metadata.source=tiingo; use the canonical route for IDs, text search and sorting. Responses carry a Deprecation header and a documentation link; no removal date is set.
/api/datasets/stats/?ids=... remains available with its original batch response and omission rules; new callers should use dataset search. Existing metadata PATCH (top-level merge) and PUT (replacement) still modify owner JSON only. Row queries, bulk downloads, live feeds and streams are unchanged and do not include dataset metadata. Creation/update responses always include the new metric fields as null placeholders, even for previously measured datasets; GET detail/search reads the current system measurements.
Refresh and operations
Migration 016 registers a separate hourly metadata aggregate, reusing Timescale invalidation and the existing resumable history-repair job. Recent buckets refresh every five minutes over three days; older changes use bounded 15-minute catch-up. The shared metadata snapshot runs every five minutes after a successful run. Migration 019 raises its runtime limit from two to four minutes without changing its cadence or reactivating a paused job, makes the metadata aggregate materialized-only and removes raw-row probes. Metadata computation and HTTP reads do not scan raw history or the current-hour tail; recent data appears after its hourly buckets materialize and the metadata snapshot runs.
Repairs/invalidations can lag; these are eventually consistent summaries, not transactionally current counts. Initial repair does not run synchronously at startup. Metadata snapshots do not wait for initial repair to finish. Global and per-dataset summaries reflect the materialized coverage available in one snapshot; missing buckets count as zero, including for new or backfilled datasets, and a dataset whose final bucket disappears has its prior metrics cleared. The current incomplete hour is excluded; data_updated_at can be newer than these approximate metrics. Snapshot refresh scans aggregate history and the catalog: monitor CPU, I/O and job duration before production promotion. A timeout keeps the last good snapshot. Migration 016 holds write-conflicting locks until its transaction commits; NOWAIT only bounds lock acquisition, not the hold. Drain ingestion and dataset writes for the migration window, allow migration completion, then resume writes. Concurrent replicas wait for the migration-state lock for up to five minutes (or the caller’s earlier deadline), then recheck applied versions. This coordination wait is separate from migration-body lock retries; expiry aborts startup without running the migration.
INSERT/UPDATE tracking groups changes per dataset; DELETE uses a row trigger because Timescale does not support DELETE transition tables. A statement-local cache (at most 2 KiB) avoids repeated dataset-state lookups and rolls back with the transaction. DELETE still invokes a trigger per row, including dataset cascades; large/compressed dataset deletions require an operational maintenance window and measurement on staging. No hard API delete-size cap is imposed. Direct SQL/COPY through the rows hypertable is covered. Do not bypass triggers or write internal chunk tables. TRUNCATE/drop_chunks are not supported tracked maintenance paths; retention is disabled by migration 006. Historical modification times cannot be reconstructed. No new physical-size accounting or per-row byte calculation runs in the ingestion path.
SELECT view_name, next_start, completed_at
FROM aggregate_history_repair WHERE view_name='dataset_metadata_buckets';
SELECT j.job_id, j.proc_name, s.last_successful_finish, s.last_run_status
FROM timescaledb_information.jobs j
LEFT JOIN timescaledb_information.job_stats s USING(job_id)
WHERE j.proc_name IN ('refresh_dataset_metadata','repair_aggregate_history');
SELECT * FROM atlas_metadata;
Create / Read / Update / Delete
Create a new dataset. You automatically become the owner.
{
"name": "btc_hourly_ohlcv",
"description": "Bitcoin hourly OHLCV candles from Tiingo",
"metadata": {
"source": "tiingo",
"ticker": "btcusd",
"frequency": "1h"
}
}
{
"id": 1,
"name": "btc_hourly_ohlcv",
"description": "Bitcoin hourly OHLCV candles from Tiingo",
"metadata": {"source": "tiingo", "ticker": "btcusd", "frequency": "1h"},
"columns": [],
"created_at": "2025-12-16T10:30:00Z",
"updated_at": "2025-12-16T10:30:00Z"
}
{
"error": "dataset already exists",
"details": {"name": "btc_hourly_ohlcv"}
}
Get a single dataset by ID. Requires read access (owner or matching tag).
Update dataset name, description, and metadata. Replaces all fields.
Permanently delete a dataset and all its data. This cannot be undone.
Metadata Operations
Update metadata without affecting other dataset fields.
Merge new fields into existing metadata (partial update).
curl -X PATCH https://your-domain.com/api/datasets/1/metadata/ \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"version": "2.0"}'
# Merges with existing: {"source": "tiingo"} → {"source": "tiingo", "version": "2.0"}
Replace all metadata (full overwrite).
Columns
Columns define the schema of your timeseries data. Each column has a name and data type.
You can insert data without pre-defining columns. The values field accepts any JSON object. Columns are primarily for schema documentation and Atlas UI display.
List columns for a specific dataset.
Create a new column definition. Use this to document your schema for the Atlas UI.
{
"dataset": 1,
"name": "close_price",
"dtype": "float"
}
| Field | Type | Description |
|---|---|---|
dataset |
integer | Dataset ID this column belongs to |
name |
string | Column name (should match keys in row values) |
dtype |
string | float, int, or string |
{
"error": "column already exists for this dataset",
"details": {"dataset_id": 1, "name": "close_price"}
}
Rows (Timeseries Data)
Rows contain the actual timeseries data. Each row has a timestamp and values for the dataset's columns.
Aggregated candles are eventually consistent: recent completed buckets refresh every five minutes, while historical changes at any age are processed in bounded batches every fifteen minutes. New datasets and historical backfills need no manual aggregation trigger. Large backfills can take multiple runs; these intervals are schedules, not freshness guarantees. Deployment starts a one-time, checkpointed historical rebuild in the background, so older aggregate results can remain incomplete until catch-up finishes. Raw rows are available independently of aggregate refresh.
Query one dataset with legacy pagination, or opt into grouped multi-dataset reads using datasets or dataset_names. All requests require read access.
| Parameter | Type | Description |
|---|---|---|
dataset |
integer | Single dataset ID. Use this or dataset_name for the existing paginated response. |
dataset_name |
string | Single dataset name, as an alternative to dataset. |
datasets |
integer list | Bulk mode: comma-separated IDs or repeated parameters, up to 1,000 datasets. |
dataset_names |
string list | Bulk mode: comma-separated names or repeated parameters. Do not mix with other selectors. |
start |
datetime | Inclusive start timestamp (RFC 3339, for example 2026-01-01T00:00:00Z). |
end |
datetime | Inclusive end timestamp (RFC 3339). Bulk requests reject a start later than the end. |
ordering |
string | timestamp (ascending, default) or -timestamp (newest first). Applies within each dataset. |
limit |
integer | Single dataset: default 250, maximum 10,000; 0 or values above 10,000 reset to 250. Bulk: positive per-dataset limit, default 10; dataset count × limit must not exceed 10,000. |
offset |
integer | Single dataset only: number of rows to skip (default 0, maximum 2,147,483,647). Rejected in bulk mode, even when 0. |
after |
datetime | Single raw dataset only: timestamp cursor, exclusive in the requested ordering direction, used instead of offset. Currently ignored for single-dataset aggregates. Rejected in bulk mode. |
count |
boolean | Single dataset only: true asks for a total on any page, false skips it even on the first page. When omitted, only the first page reports a total: offset 0, and for raw rows no after cursor (aggregates ignore after). Ignored in bulk mode. |
resolution |
string | Omit for original provider rows without a default time window. Explicit raw also returns original rows; 5m, 1h, and 1d return aggregated OHLCV candles. |
Example
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/rows/?dataset=1&start=2025-12-01T00:00:00Z&end=2025-12-16T00:00:00Z&ordering=-timestamp&limit=100"
Time windows and row formats
When resolution is explicitly supplied, maximum windows are 24 hours for raw, 7 days for 5m, 90 days for 1h, and unlimited for 1d. For bounded resolutions, omitting both timestamps selects the most recent maximum window; supplying only one timestamp derives the other using that window. A larger explicit window is rejected. Omitting resolution preserves the legacy unbounded time-window behavior.
Raw rows contain id, dataset_id, timestamp, values, and created_at. Aggregated rows contain dataset_id, timestamp (bucket time), open, high, low, close, volume, and count (source row count). Existing aggregation behavior is unchanged; this endpoint does not add a completed-candle-only guarantee.
Single-dataset responses retain {"count": ..., "next": ..., "previous": ..., "results": [...]}. count is -1 when the total is unknown: the page did not ask for one (see count above), counting took longer than two seconds, a ranged count passed 100,000, the dataset has not been measured yet, or its metadata snapshot could not be read. Raw rows without start, end or resolution report the dataset's row count from the metadata snapshot (never less than the rows up to the end of a non-empty page). The snapshot lags new rows by minutes, and longer while historical backfills are being picked up, so this total is approximate. With a time range (including the default window of an explicit resolution=raw), raw rows count the range up to 100,000 rows; aggregates count their buckets up to 100,000. Only an exact total (a ranged or aggregate count within the cap) can end next: with one, next is omitted once the page reaches the total. With an approximate or unknown total, a full page always links to the next one and a short page is the last. To page through a large range, prefer the after cursor (raw rows) over a growing offset; cursor pages have no previous link.
Multiple Datasets — Recent Raw Candles
In bulk aggregate responses, unavailable OHLCV values are returned as JSON null, not zero; such values do not fail the request. Raw rows and single-dataset responses are unchanged.
Fetch the newest ten original provider rows per asset in one HTTP call. For one-minute provider candles, omit resolution; resolution=5m would aggregate into five-minute candles instead. Ten returned rows may span more than ten minutes when data has gaps.
# By ID
curl -G -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/rows/" \
--data-urlencode "datasets=123,456" \
--data-urlencode "limit=10" \
--data-urlencode "ordering=-timestamp"
# By name (same response shape, with resolved dataset IDs)
curl -G -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/rows/" \
--data-urlencode "dataset_names=btc_usd_1m,eth_usd_1m" \
--data-urlencode "limit=10" \
--data-urlencode "ordering=-timestamp"
Bulk requests must use exactly one plural selector, without dataset or dataset_name. Duplicate datasets, empty selectors, invalid IDs, and offset/after are rejected with HTTP 400. Shared time filters, ordering, and resolution apply independently to each dataset.
A single plural parameter is parsed as a CSV list. When the parameter appears more than once, each value is one literal ID or name: dataset_names=btc_usd_1m&dataset_names=eth_usd_1m. Do not combine repeated parameters with comma-separated lists. For a name containing a comma, use repeated literal name parameters, or CSV-quote the name in a single parameter (for example, --data-urlencode 'dataset_names="btc,usd"'). URL-encode names and other query values.
Maximum 1,000 datasets and 10,000 requested rows, calculated as dataset count × per-dataset limit, regardless of how many rows actually exist. Thus 1,000 datasets × 10 rows is valid; 1,000 × 11 is HTTP 400. For 200 assets, requesting ten rows each returns at most 2,000 rows. An omitted bulk limit defaults to 10.
{
"dataset_count": 2,
"row_count": 1,
"datasets": [
{
"dataset_id": 123,
"rows": [
{
"id": 1001,
"dataset_id": 123,
"timestamp": "2026-01-01T12:00:00Z",
"values": {"open": 100, "high": 105, "low": 99, "close": 103, "volume": 1200},
"created_at": "2026-01-01T12:01:00Z"
}
]
},
{"dataset_id": 456, "rows": []}
]
}
Groups follow request order and include empty datasets. row_count is the actual total returned, not a historical count. Bulk responses have no pagination links or historical totals. Names resolve to IDs; all datasets must exist and be readable before any rows are returned. A missing dataset returns HTTP 404, denied access HTTP 403, and a query failure fails the entire response rather than returning partial results. Single-dataset interfaces remain unchanged.
Insert a single row of data.
{
"dataset": 1,
"timestamp": "2025-12-16T10:00:00Z",
"values": {
"open": 104500.00,
"high": 104800.00,
"low": 104200.00,
"close": 104600.00,
"volume": 1234567
}
}
Insert or update rows using the PostgreSQL COPY fast path. Existing rows with the same dataset and timestamp are updated. The endpoint accepts either the original single-dataset body or a multi-dataset body.
Single Dataset
{
"dataset": 1,
"rows": [
{
"timestamp": "2025-12-16T10:00:00Z",
"values": {"open": 104500, "close": 104600}
},
{
"timestamp": "2025-12-16T11:00:00Z",
"values": {"open": 104600, "close": 104700}
}
]
}
{
"inserted": 2,
"total": 2
}
Multiple Datasets
Use datasets to update hundreds of datasets in one HTTP request. Each item must specify exactly one of dataset or dataset_name. The complete request is validated and written atomically: if any dataset is missing, unauthorized, or invalid, no rows are changed.
curl -X POST https://your-domain.com/api/rows/bulk_insert/ \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"datasets": [
{
"dataset": 1,
"rows": [
{"timestamp": "2025-12-16T10:00:00Z", "values": {"price": 104500}}
]
},
{
"dataset_name": "eth_usd_1m",
"rows": [
{"timestamp": "2025-12-16T10:00:00Z", "values": {"price": 3890}}
]
}
]
}'
{
"inserted": 2,
"total": 2,
"datasets": [
{"dataset": 1, "inserted": 1, "total": 1},
{"dataset": 2, "inserted": 1, "total": 1}
]
}
A request may contain up to 1,000 datasets, 100,000 total rows, and a 64 MiB encoded body. Dataset entries may not be repeated. Replaying a request reapplies its values and can overwrite newer writes to the same dataset/timestamp pair.
Validation and authorization errors identify the failing entry with a zero-based path such as datasets[3] or datasets[3].rows[7].
Go client callers can use apitypes.MaxBulkInsertDatasets and apitypes.MaxBulkInsertRows when chunking work. BulkInsertDatasets relies on the supplied context deadline instead of the client's normal 30-second request timeout; allow at least three minutes for large requests.
Raw row reads reflect successful upserts immediately. Bulk upserts skip rewrites when the stored and incoming JSONB values are equal; the returned count still includes every submitted row, including unchanged rows and duplicates. Continuous aggregates refresh asynchronously: recent buckets are checked every five minutes and historical changes every fifteen minutes, with bounded work per run. Historical corrections and new-dataset backfills are picked up automatically; large backlogs can take multiple runs. SSE streams deliver newly appended timestamps and do not re-emit an update to a timestamp the subscriber has already seen.
Streaming & Downloads
Efficient endpoints for bulk data access and real-time streaming.
Download all data from a dataset in CSV or JSON format.
| Parameter | Type | Description |
|---|---|---|
dataset |
integer | Dataset ID (required) |
output |
string | json (default) or csv |
start |
datetime | Optional start timestamp |
end |
datetime | Optional end timestamp |
# Download as CSV
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/rows/bulk_download/?dataset=1&output=csv" \
> data.csv
# Download as JSON
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/rows/bulk_download/?dataset=1&output=json" \
> data.json
Bulk download recent data as NDJSON (newline-delimited JSON). Memory-efficient for large datasets.
# Download most recent 1000 rows
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/rows/live/?dataset=1&limit=1000"
# Output: one JSON object per line
# {"timestamp":"2025-12-16T10:00:00Z","close":104600}
# {"timestamp":"2025-12-16T11:00:00Z","close":104700}
Real-time Server-Sent Events (SSE) stream. Connection stays open and pushes new data as it arrives.
| Parameter | Type | Description |
|---|---|---|
dataset |
integer | Dataset ID (required) |
poll_interval |
integer | Seconds between checks for new data (default: 2) |
Event Types
connected- Initial connection establisheddata- New data point receivedheartbeat- Keepalive (no new data)error- Query error occurred
# Connect to real-time stream
curl -H "X-API-Key: YOUR_KEY" \
-H "Accept: text/event-stream" \
"https://your-domain.com/api/rows/stream/?dataset=1&poll_interval=2"
# Output (connection stays open):
# event: connected
# data: {"dataset_id":1,"message":"Connected to real-time stream"}
#
# event: data
# data: {"timestamp":"2025-12-16T10:00:00Z","close":104600}
#
# event: heartbeat
# data: {"timestamp":"2025-12-16T10:00:02Z"}
Error Responses
All errors return a JSON object with an error field and optional details.
{
"error": "Dataset not found",
"details": "No dataset with ID 999"
}
| Status Code | Meaning |
|---|---|
400 |
Bad Request - Invalid parameters or request body |
401 |
Unauthorized - Missing or invalid API key |
403 |
Forbidden - You don't have permission for this action |
404 |
Not Found - Resource doesn't exist |
409 |
Conflict - A record with the same unique identity already exists (dataset name; column name within a dataset) |
413 |
Content Too Large - Request body exceeds the endpoint limit |
429 |
Too Many Requests - Rate limit exceeded |
500 |
Internal Server Error - Something went wrong |
Common 403 Forbidden Scenarios
- "No matching tags" - You need to acquire a tag that's assigned to the dataset
- "Owner access required" - Only the dataset creator can perform write operations
- "Tag secret required" - The tag is private and requires a secret key
Pagination
List endpoints return paginated results with metadata for navigation.
{
"count": 150,
"next": "https://your-domain.com/api/datasets/?limit=50&offset=50",
"previous": null,
"results": [...]
}
| Field | Description |
|---|---|
count |
Total number of results matching the query |
next |
URL for the next page (null if no more results) |
previous |
URL for the previous page (null if on first page) |
results |
Array of items for the current page |
Pagination Parameters
limit- Number of results per page (default: 250)offset- Number of results to skip
# Get page 2 with 50 items per page
curl -H "X-API-Key: YOUR_KEY" \
"https://your-domain.com/api/datasets/?limit=50&offset=50"
Troubleshooting
Empty dataset list
You need the public tag to see public datasets. The Atlas UI acquires this automatically on login. API-only users should run:
curl -X POST https://your-domain.com/api/tags/acquire/ \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"tag_name": "public"}'
Others can't see my dataset
New datasets are private. Assign a tag to share:
curl -X POST https://your-domain.com/api/datasets/YOUR_DATASET_ID/tags/ \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"tag_name": "public"}'
403 Forbidden on read
You lack matching tags. Check what tags the dataset requires (ask the owner), list available tags with GET /api/tags/, then acquire the matching tag.
403 Forbidden on write
Only the owner can write. Tags grant read access only. Create your own dataset if you need write access.
Check your access
curl -X POST https://your-domain.com/api/access/check/ \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"dataset_id": 1}'
# Returns: {"has_access": true, "is_owner": false, "matching_tags": ["public"]}
Allora Atlas Team