Installing
An evaluation stack is one command from a source checkout. The services are compiled inside the containers, so a container runtime is the only thing you install, no Java and no build tools. It brings up the three DataHub services plus every backing store, seeded with demo tenants.
That stack is not production. It runs without TLS, with development credentials and permissive database authentication. The changes needed for production are listed at the bottom of this page.
What you need
- A container runtime, Docker with the
docker composeplugin, or Podman. - Git, to get the source. DataHub is built from source: there are no published application images to pull, so there is nothing to authenticate to and nothing to obtain in advance.
- At least 16 GB of RAM. The stack starts on less, but the backing stores compete for memory and evaluation becomes misleadingly slow. Plus a few GB of disk for images and volumes.
Installing the evaluation stack
git clone https://github.com/IntelliStream-DataHub/datahub-platform.git
cd datahub-platform
./scripts/up.sh --build # builds the images, then starts everything
podman compose logs -f # watch it come up
The first run compiles the platform and pulls the backing-store images, so expect it to take
several minutes. Later runs reuse both, and plain ./scripts/up.sh starts the stack without
rebuilding.
Two more commands worth knowing:
podman compose down # stop, keeping all data
podman compose down -v # stop and wipe all data
Substitute docker compose for podman compose if that is your runtime; the launcher
detects which one you have and uses it either way.
By default the stack comes up with a small demo data set already loaded, so the console shows
something real on the first visit. Use ./scripts/up.sh --no-demo for an empty stack
instead.
Reaching it
Open it in a browser on the same host:
| Service | URL | Login |
|---|---|---|
| Console | http://localhost:8080 | foo / foo |
| API | http://localhost:8081 | Bearer token |
| Identity provider | http://localhost:8090 | admin / admin |
The stack assumes the browser runs on the host, login redirects and the dashboard's direct
API calls are locked to localhost origins by design.
This is fiddlier than it looks, and a plain SSH tunnel does not work: sign-in redirects your browser to the identity provider at the server's routable IP on port 8090, which a localhost-only tunnel cannot satisfy, so login fails after the redirect.
For an evaluation, the reliable options are running the browser on the host itself, or making ports 8080 and 8090 on the server's IP directly reachable from your machine and accepting that this is an unhardened evaluation stack you have just exposed. Treat remote access as part of production hardening rather than a quick tweak.
If login redirects fail
OAuth login sends the browser to the identity provider, and the browser and the
in-network containers must agree on a single issuer URL. The launcher sets that to the
machine's routable IP automatically. When auto-detection gets it wrong, or when you use
plain docker compose, set it explicitly:
KC_ADDR=192.168.1.50 ./scripts/up.sh
This is by far the most common installation problem. If the console loads but signing in loops or errors, the issuer address is the first thing to check.
What the stack contains
Three DataHub services:
| Service | Role |
|---|---|
| API | The REST interface and the platform's core. Owns the model and the ingestion endpoints |
| Console | The web interface people use |
| Datapoint and event consumer | Lands measurements and events, and fans live data out to subscribers |
Plus the backing stores, a relational database, a streaming platform, a columnar store for measurements, a graph database, two key-value stores, a secret store and an identity provider. What each one does →
About the build
./scripts/up.sh --build compiles the services inside a build container, so nothing beyond
the container runtime has to be installed on the host.
Before production
The evaluation stack is deliberately convenient and deliberately insecure. Everything below must change.
The evaluation stack runs plaintext. Production needs TLS on the console, the API, the streaming platform and every database connection.
Demo tenants, demo users and default administrator passwords all exist for convenience. None should survive into production.
Users belong in your organisation's OAuth2/OIDC provider, with your existing groups and joiner-mover-leaver process. The realm, the clients, the organisation claim and any Entra ID brokering are covered on their own page. Identity providers, including Entra ID →
The evaluation stack uses permissive local authentication. Production wants proper credentials, per-tenant database roles, and network restrictions.
For anything beyond a small single-tenant install. Without one, every request opens its own database connection, which does not scale. Why →
All services run as multiple instances. The load balancer needs WebSocket-upgrade support and long connection timeouts, but no session affinity, because nothing holds unrecoverable per-instance state, a subscription's position lives on the platform, so a reconnect resumes on any instance. Give it a per-IP request limit as well, and keep its body size ceiling above the API's own. Why → · The proxy settings → · A worked example →
Defaults are for evaluation. Storage grows with ingestion whether or not anyone reads it. Data lifecycle →
Every tenant arrives with a request rate, a daily ingest allowance and a cap on live connections. The defaults are sized for an evaluation tenant, so a production one usually needs them raised, which is one database update per tenant and no restart. The ceilings on a tenant's total size are a separate switch and ship turned off; read the order of operations before turning them on. Limits and quotas →
Every store is database-per-tenant, so backup is per-database and restore can be scoped to one tenant. How →
Running on dedicated servers
A production installation runs the services as ordinary system services on machines of their
own rather than in the evaluation stack. The repository ships two sets of worked examples for
that, in its nginx and systemd folders, each with a README that walks through the
installation. They are starting points to copy and edit, not something the platform installs
for you.
The edge. The nginx examples put a load balancer in front of two API and two console instances. TLS ends at the edge with a modern cipher set and a certificate covering both hostnames; the live data connections are upgraded to WebSocket with long timeouts; file uploads and downloads stream through unbuffered; the per-IP request limit and the body ceiling from limits and quotas are in place; and the kernel settings are sized to the uplink. There is a variant for Debian and Ubuntu and one for AlmaLinux and RHEL, and the README covers both, including the SELinux setting without which every proxied request fails.
The services. The systemd examples run every DataHub service from one unit template, with a short drop-in per instance saying which part of the machine it may use, an environment file per service holding the Java settings, and a configuration file per service tuning the embedded web server for sitting behind the proxy. The units are hardened, with a read-only system, a private temporary directory and no new privileges, and they write no memory dumps of any kind, because a dump is a copy of process memory with credentials and tenant data in it. The README says how to take one deliberately when a fault needs it, how to turn on the metrics endpoint, and what to check on a running service.
What you supply: the two hostnames and their certificate, the addresses of the load balancers (the API trusts forwarded headers from those and nothing else), the secret store's credentials, a PostgreSQL connection pooler on each application host, and the file-storage mount. Sizing is yours to decide: the examples are written for large servers and say so, and every figure in them is meant to be read and changed rather than copied.
Two placement rules carry over from the architecture page: the API and console scale by adding instances behind the balancer with no session affinity, and the housekeeping service runs as exactly one instance.
Air-gapped installation
DataHub runs entirely disconnected from the internet. Nothing in the platform requires an outbound connection at runtime, no licence check, no telemetry callback, no hosted dependency.
The build is the part that needs network access, because it resolves its dependencies. The usual approach is to build on a connected machine and carry the resulting container images across the boundary, together with the backing-store images. That is a normal artefact-transfer problem rather than a platform one.
This is a common requirement in regulated and critical-infrastructure environments, and it is one of the practical consequences of the platform being open source. Security →
- Architecture, without the jargon: what each component does
- Limits and quotas: the per-tenant budgets, and raising one
- Organisations and tenants: planning tenant boundaries before onboarding
- Data lifecycle: retention, backups and cost
- Security and compliance: the production posture