Telemetry query & retention

This page is the reference for reading telemetry back out of CORE-M and for the data-governance controls that surround it: how long data is kept, how it is rolled up, how it is exported, and how it is deleted without falsifying blockchain proofs. For how data is written in the first place, see Sending telemetry.

Querying

CORE-M supports querying multiple devices and multiple metrics over a time range, with aggregation, downsampling, timezone-aware bucketing, and pagination. A query returns one series per device/metric pair.

Query parameters

Parameter	Description
devices	One or more device IDs.
metrics	One or more metric names.
start / end	Time range bounds (ISO 8601).
aggregation	Reduction per bucket, e.g. avg, min, max, sum, count.
interval	Bucket width, e.g. 5m, 1h, 1d.
timezone	Bucket-alignment timezone (defaults to the tenant timezone).
fill	How missing buckets are represented (fill policy).

POST /api/v1/telemetry  (query via QueryTelemetrySeries)
{
  "devices": ["D1", "D2"],
  "metrics": ["temperature", "humidity"],
  "start": "2026-05-01T00:00:00Z",
  "end": "2026-05-08T00:00:00Z",
  "aggregation": "avg",
  "interval": "5m"
}

Timezone bucket alignment

When an aggregation interval is a calendar unit (a day, for example), bucket boundaries align to the tenant’s local days, not to UTC. A daily query that spans a daylight-saving transition still produces correctly aligned local-day buckets, and the response carries timezone metadata so the client can interpret them.

Query-size limits

A query that would return more raw points than the tenant’s max_query_points is rejected rather than served partially.

Condition	Result
Estimated result within max_query_points	Query runs
Estimated result exceeds max_query_points	RESOURCE_EXHAUSTED; response suggests a coarser interval that would satisfy the limit

Tip

If a query is rejected for size, take the suggested aggregation interval from the error and re-issue the query downsampled. For long dashboard ranges, query a rollup directly (next section) — it is both cheaper and within limits.

Rollups and continuous aggregates

CORE-M maintains downsampled rollups at fixed resolutions so dashboards and long-range queries never scan raw telemetry.

Resolution	Use
1m	Short-range, high-detail views
5m	Medium-range views
1h	Multi-day to multi-week views
1d	Multi-month views

• Rollups are computed as TimescaleDB continuous aggregates and indexed in Aerospike (telemetry namespace, rollups set).
• A scheduler refreshes the aggregates as new telemetry arrives and updates the corem_telemetry_rollup_lag_seconds metric.
• The query planner selects the appropriate rollup automatically. For example, a 90-day hourly request resolves against the 1h rollup, and the response metadata reports source="rollup_1h".

Retention policies

Retention is policy-driven and applied per selector. A policy never deletes blockchain proof records needed to verify already-anchored data.

Policy fields

Field	Meaning
policy_id	Policy identifier
tenant_id	Owning tenant
selector	What the policy targets — device, group/profile, metric, or tag
raw_retention_days	How long raw telemetry is kept
rollup_retention_days	How long rollups are kept
proof_retention	Retention for proof records (preserved independently of raw data)
delete_behavior	What cleanup does on expiry: redact, delete, or archive
legal_hold	If set, suspends all deletion for the selector
created_at / updated_at	Audit timestamps

Behaviour

Scenario	Outcome
Raw telemetry past raw_retention_days, point is final	Raw TimescaleDB rows may be deleted; the proof row and portable proof metadata remain; the proof lifecycle record remains or is compacted to a final summary
Selector under legal hold	No raw, rollup, proof, alarm, or audit data is deleted; cleanup reports a skipped_legal_hold count
Policy updated (e.g. 90 → 365 days)	Future cleanup uses the new value; audit event retention.policy.updated records old and new values

Caution

Retention is independent of anchoring. Deleting raw telemetry after its retention window does not weaken existing proofs — the hash, Merkle path, and on-chain record are kept. Verification simply discloses that CORE-M no longer retains the raw payload. See Verification.

Legal hold

legal_hold on a selector freezes deletion entirely. While a device, group, or tenant is under hold, retention cleanup skips all of its raw telemetry, rollups, proofs, alarms, and audit data and reports the skipped count in cleanup metrics. Lifting the hold resumes normal policy-driven cleanup.

Export jobs

Exports are asynchronous jobs covering telemetry, rollups, alarms, audit logs, and proof bundles. Job records live in the Aerospike telemetry namespace, export_jobs set, keyed {tenant_id}:{job_id}.

Step	Behaviour
Create	Requires the exports permission; job is created with status="queued" and event export.requested.{tenant}.{job} is published
Build	A worker writes a manifest of telemetry files plus proof-bundle references (when include_proofs=true)
Size guard	If estimated size exceeds max_export_size_bytes, the job is rejected with RESOURCE_EXHAUSTED, returning estimated_size_bytes and max_export_size_bytes
Download	The completed job exposes a signed, expiring download URL (e.g. 24 h); after expiry, an authorized user can regenerate one

Compliance deletion with proof preservation

Operational deletion and immutable proof preservation are deliberately separated. Deleting telemetry for compliance must not falsify or remove blockchain proof facts.

Scenario	Outcome
Redact a personal field (e.g. operator_name) for a device	Query APIs stop returning the field; proof records keep the hashes and metadata needed to prove historical existence; verification discloses the raw payload is no longer retained
Request deletion of a final proof row	Rejected. The system refuses direct proof deletion, suggests redaction/export workflows, and writes audit event proof.delete.denied

See Security & compliance for the data-subject workflows and audit trail around deletion.

Tenant quotas

Quotas are enforced per resource and reject with RESOURCE_EXHAUSTED when exceeded. They are distinct from the gateway’s per-tenant request rate limit (see API overview).

Quota	Enforced on
Ingest rate	Points per minute accepted at ingest
Storage	Total stored bytes per tenant
Query range	max_query_points per query
Export size	max_export_size_bytes per export job
Dashboard connections	Concurrent WebSocket connections
API requests	Per-tenant request rate
Anchoring backlog	Pending batches per tenant

Scenario	Outcome
Ingest quota exceeded	Telemetry request rejected with RESOURCE_EXHAUSTED; no proof lifecycle record is created; corem_quota_rejections_total{quota="ingest_points"} increments
Storage at 85% of quota	Event quota.warning.{tenant}.storage published; tenant admins see a warning in settings

Usage counters

CORE-M tracks per-tenant usage so quotas and billing can be reasoned about.

Counter	Granularity	Description
Points ingested	per day	Accepted data points per tenant per day
Storage	per day	Stored bytes per tenant per day