The technical answer to what this is underneath
Architecture
The product pages are written for the people who sign. This one is written for whoever has to run it, and says what the platform is made of, where the data sits, and what you take on if you host it yourself.
The write path
Four ways in, one place to land
However data arrives, it lands under the same model with the same rules. Edge collectors sit next to the control systems and speak OPC-UA and Modbus. The built-in flow engine, coming soon, runs small processor functions that clean, convert and derive as data streams through. A plain REST API accepts pushes from anything that can speak HTTP, and the SDKs make ingestion ordinary, testable code in Java, Python or Rust.
However it arrives, it arrives once: same model, same permissions, same lineage.
The flow engine
One engine, every pipeline
First, the honest caveat: the flow engine is being built right now, in the open, and it is coming soon. A flow is a small pipeline: read data from somewhere, transform or check it, write it somewhere, on a schedule, when an event arrives, or when someone asks. Each step is a function that declares its outcomes, and connections route on them, so "what happens on failure" is drawn in the definition instead of buried in a script. A flow is a versioned definition, not a cron job on a server: validate it, preview it against the last hour without writing anything, deploy it, and watch its runs.
The engine is why the platform's promises hold everywhere. Every step records provenance, every checkpoint records quality, and the same machinery is designed to carry what comes next: agent orchestration, policies, change-data-capture, pulls from external systems. One engine underneath them all, instead of one mechanism each, and everything you build on it composes with everything else.
A pipeline is a definition
Editing a flow creates a new version, old versions are kept, and every run records the version it used. What ran in March is answerable in November, which is what makes the audit trail real.
Your code, inside the fence
Most steps are configured built-ins. When they are not enough, upload your own function in Python or WebAssembly: it is scanned, reviewed and approved by a person before it can be deployed, and it always runs in a sandbox, whatever the review said.
Dies somewhere, finishes anywhere
Instances are identical and share their work through leases: a run on an instance that dies is picked up by another within a minute, and sources are only acknowledged once a run completes, so capacity and resilience are a matter of instance count.
New capabilities are new flows, and every flow inherits provenance, quality and the fence.
Where the data sits
Every store does the one thing it is best at
General-purpose storage is how platforms end up slow and expensive at the same time. Measurements and events go to ClickHouse, columnar and built for scanning billions of rows. The knowledge graph lives in Neo4j, where following a relationship is a step, not a join. PostgreSQL holds the ontology's definitions and the platform's own records. In between, Apache Pulsar carries every datapoint, resource change and event as a stream, with Valkey and Kvrocks covering hot cache and durable key lookups. You never address any of them directly: everything goes through the model.
You query the model. Where the bytes sit is the platform's problem.
The read path
One API, and everything is a client of it
The console is not special, the SDKs are not special and the agents are not special: all of them go through the same stateless REST API, with live values streaming over WebSocket subscriptions. Identity comes from Keycloak, every call carries a token that says who is asking, and per-dataset access is decided by the same rules whoever the caller is.
One surface to secure
A single API means one place where permissions are enforced and logged. There is no side door for internal tools, because the internal tools use the same door.
Three SDKs over it
Java, Python and Rust, Apache-2.0 licensed, so your own code is never touched by copyleft. If the console can do it, your code can do it.
Agents through MCP
An agent reaches the model through an MCP server rather than a bespoke integration. What it may read and change is the same policy question as for a person, answered by the same rules.
Storage economics
Full-resolution history, kept because keeping it is cheap
Columnar storage changes the two numbers that used to force downsampling. Industrial time-series compress around 100 times, because neighbouring values resemble each other and a column of them squeezes well. And a scan touches only the columns a query asks for, which is how ClickHouse reads over 10 billion rows per second in our setup. Ten years of second-level history stops being a retention argument and becomes a line item you can afford.
The squeezing is not magic: timestamps and slowly moving values are stored as deltas and compressed with ZSTD, and floating-point signals use Gorilla encoding, invented for exactly this shape of data. The ceiling is still rising, too: ALP, adaptive lossless floating-point compression, beats the current codecs on ratio and speed alike, and it is where we are headed next.
Running it
Self-host it, or run on hardware we own
Self-hosting is Docker Compose for a small footprint, or bare metal when the machines should be yours, and it is real self-hosting: PostgreSQL, ClickHouse, Neo4j, Pulsar and the platform services are yours to run, upgrade and back up. The hosted alternative runs on machines we own in Stavanger: AlmaLinux on more than 50 servers with 30 TB of RAM, ZFS on 2 PB of disk and 400 TB of SSD, with InfiniBand and NVMe-oF over RDMA between them. The orchestration software that runs it we wrote ourselves, small enough to debug, and we watch it in Grafana like everyone else.
What you are agreeing to operate
Self-hosting is real self-hosting. You run PostgreSQL, ClickHouse, Neo4j, Pulsar and the platform services, and upgrades and backups are yours. That is a genuine operational load, and it is the price of the code being yours under AGPL-3.0. If you would rather not carry it, we host it on hardware we own and operate.
Common questions
Asked and answered
What does IntelliStream run on underneath?
PostgreSQL for the ontology and the platform's records, ClickHouse for time-series and events, Neo4j for the knowledge graph, Apache Pulsar for streaming, with Valkey and Kvrocks for caching and key lookups, and Keycloak for identity. All of it is open source. The platform itself is AGPL-3.0, while the client SDKs are Apache-2.0, so the code you write against them is never touched by copyleft.
Can we self-host it, and what does that take?
Yes, with Docker Compose for a small footprint or on bare metal. It is real self-hosting: you run the databases, the message broker and the platform services, and upgrades and backups are yours. If you would rather not carry that load, we host the same platform on hardware we own in Stavanger.
How do AI agents connect to the platform?
Through an MCP server that fronts the same API everything else uses. The agent authenticates like a person, via Keycloak, and per-dataset permissions are evaluated identically. There is no separate agent pathway to secure or to audit.
Why ClickHouse for time-series?
Because industrial time-series are exactly what columnar storage is good at. Neighbouring values resemble each other, so they compress around 100 times, and scans read only the columns a query touches, which is how it reads over 10 billion rows per second on our setup. Full-resolution history becomes affordable instead of aspirational.
Why Apache Pulsar for streaming?
Because the stream is where everything meets: every datapoint, resource change and event passes through Pulsar topics on its way to the stores. Pulsar treats tenants as a first-class concept, so isolation does not have to be bolted on, and subscription cursors live on the broker, so a consumer, or a live WebSocket subscription, can drop, reconnect on another instance and resume where it left off. It also lets one stream feed two very different consumers at their own pace: batched inserts into ClickHouse, and order-sensitive updates into the graph.
Why Valkey?
For the data that should live in memory because losing it costs nothing: sessions and hot cache. Externalized sessions are what let any instance serve any request. Valkey is the Redis-compatible store governed by the Linux Foundation, which matters to us because the whole stack is open source and we intend to keep it that way. And on our hosted setup it runs on 2 TB of Intel Optane persistent memory per cache node, so the hot cache is measured in terabytes, not gigabytes.
Why Kvrocks?
Kvrocks speaks the same protocol as Valkey but keeps the data on disk, so a restart loses nothing and memory stays reserved for the work that actually needs it. Underneath it is RocksDB, and it supports transactions. Same interface, different promise.
Why Neo4j for the knowledge graph?
Because the model is mostly relationships, and Neo4j stores relationships as first-class records rather than reconstructing them through joins, so following one is a constant-time step no matter how large the graph grows. Cypher keeps the queries readable, for engineers and for agents, and new kinds of relationships are added without a schema migration, which is exactly how an operational model actually evolves.
Why Keycloak for identity?
Because identity is the last thing a platform should invent for itself. Keycloak is the mature open-source identity provider: it issues the tokens every call carries, and per-dataset access follows its organization groups. Just as important, it integrates excellently with Microsoft Entra ID, where most industrial organisations already keep their people, so your engineers sign in with the account they already have, and offboarding still happens in one place, your own directory.
Read it before you talk to anyone
The platform documentation covers deployment and operations. The SDK documentation covers writing code against it. Both are open, and neither asks for an email address first.