High-throughput ingestion

A common job: push a steady, high-volume stream of sensor readings in so a dashboard or alerting rule can act on them. The SDK does the heavy lifting — it chunks the data into batches and sends them concurrently, retrying transient failures.

How it works

Datapoints are split into batches (default 10,000 per request — the store is optimised for large batches), sent concurrently up to a bounded in-flight limit, and transient failures (HTTP 429/5xx, network) are retried. The Java client returns an IngestResult summarising what landed and what didn't.

Stay inside the ingest caps

The defaults already fit, so this only matters if you have tuned something:

Cap	Value
items in one request	10,000, exactly the default batchSize
Datapoints in one collection, numeric	100,000
Datapoints in one collection, TEXT / MIXED	10,000
Body of POST /timeseries/data	16 MiB (10,000 numeric points is roughly 500 KB)

Over a cap is a 400; an oversized body is a 413, which no retry will fix. Sustained volume is bounded separately by a per-minute rate limit and a daily ingest quota, both 429 and both retried for you. See Limits & quotas.

Ingest a million readings

Java

ingest groups datapoints by series external id and returns an IngestResult.

var client = DatahubClient.fromEnv();

// readings: Map<String, List<Datapoint>> grouped by time-series external id
IngestResult result = client.timeseries().ingest(readings,
    IngestOptions.builder()
        .batchSize(10_000)     // datapoints per request
        .parallelism(16)       // concurrent in-flight batches
        .maxRetries(3)
        .build());

System.out.printf("ingested %,d, failed %,d%n", result.succeeded(), result.failed());

Python

insert_from_lists takes whole arrays (NumPy / pandas) and batches them for you.

import numpy as np, pandas as pd

client = DataHubClient.from_env()

client.timeseries.insert_from_lists(
    timestamps=pd.date_range("2026-01-01", periods=1_000_000, freq="s", tz="UTC"),
    values=np.random.rand(1_000_000),
    ts="engine_temperature")

Rust

insert_datapoints auto-batches large inputs (chunks above ~100k points).

use intellistream_datahub_sdk::create_api_service;
use intellistream_datahub_sdk::generic::{DataWrapper, DatapointsCollection};

let api = create_api_service();

let mut dw = DataWrapper::new();
dw.add_item(DatapointsCollection {
    external_id: Some("engine_temperature".into()),
    datapoints: readings,        // Vec<DatapointString { timestamp, value }>
    ..Default::default()
});
api.time_series.insert_datapoints(&mut dw).await?;

Tuning (Java)

The ingest knobs let you trade throughput against load on the server:

Option	Default	Meaning
batchSize	10_000	Datapoints per request. Do not raise it above 10_000, the server's items cap.
parallelism	8	Concurrent in-flight requests.
maxRetries	3	Retries for transient failures.
failFast	false	Abort on the first failed batch instead of collecting errors.

When failFast is off, inspect result.errors() for the per-batch failures. See the Time-series reference for the full result shape.

Then chart the trend

Once the data is in, roll it up to hourly or daily buckets for a dashboard — see Query & aggregate time-series.

Ordering

Batches are sent in parallel, so there is no cross-batch ordering guarantee — which is fine for time-stamped data, since each datapoint carries its own timestamp.