DataHub is open source. You can run the whole platform on your own machine — the API, the console, and every service they need — with one clone and one command.
This guide uses Podman. Docker works the same way; swap podman for docker in
any command below.
This setup is for trying DataHub locally. It runs without TLS and with weak default passwords, so do not expose it to a network or reuse any of it in production.
What you need
- Podman, with
podman composeavailable. - About 10 GB of free disk for the container images.
- Nothing else. The build happens inside the containers, so you do not need Java installed.
Check Podman is ready:
podman compose version
Step 1 — Get the code
git clone https://github.com/intellistream/datahub.git
cd datahub
Step 2 — Start it
./scripts/up.sh --build
That is the whole install. The first run takes a while — it builds the application images and pulls Postgres, ClickHouse, Neo4j, Valkey, Kvrocks, Pulsar, Vault and Keycloak. Later runs are much faster.
--build builds the applications. You only need it the first time, or after you pull
new code.
The script does two useful things for you:
- It picks
docker composeorpodman composeautomatically, preferring Docker if both are installed. - It works out your machine's IP address and points Keycloak at it, so logging in
through the browser works without editing
/etc/hosts.
Everything else is automatic: Vault initialises and unseals itself, Pulsar gets its tenants, and each application waits for what it depends on before starting.
If you want to run the commands yourself
To use Podman explicitly, or to see what the script does:
podman compose -f docker-compose.yml -f docker-compose.apps.yml build
podman compose -f docker-compose.yml -f docker-compose.apps.yml up -d
Both compose files are needed. The first brings up the backing services, the second adds the four applications.
Taking this route, browser login needs keycloak to resolve on your machine. Add
127.0.0.1 keycloak to /etc/hosts, or use ./scripts/up.sh, which avoids the
problem.
Step 3 — Check it started
podman compose -f docker-compose.yml -f docker-compose.apps.yml ps
Pass both files to every compose command, otherwise you only see the backing services and not the applications.
Pulsar and Keycloak take around 30 seconds to finish booting. If something looks stuck, read its log:
podman compose -f docker-compose.yml -f docker-compose.apps.yml logs -f keycloak
Step 4 — Open it
- Console — http://localhost:8080, log in as
foo/foo - API — http://localhost:8081, with Swagger UI at http://localhost:8081/swagger-ui.html
- Keycloak — http://localhost:8090, log in as
admin/admin
The console starts with two demo tenants, foo and bar, so there is something to
click through immediately.
Step 5 — Stop it
podman compose -f docker-compose.yml -f docker-compose.apps.yml down
This stops and removes the containers but keeps your data. Start it again with
./scripts/up.sh and everything is where you left it.
To wipe the data as well:
podman compose -f docker-compose.yml -f docker-compose.apps.yml down -v
The -v removes the volumes. The next start creates the demo databases from scratch.
Use it when you want a clean slate, or when a database will not come up and you would
rather start over than debug it.
If something goes wrong
A port is already in use. The containers need 8080 and 8081. If you are running the console or API from your IDE or from Gradle, stop those first — two things cannot listen on the same port.
Login redirects somewhere unreachable. The browser and the containers have to agree
on one Keycloak address. Use ./scripts/up.sh, which sets this for you. If it picks the
wrong address, set it yourself:
KC_ADDR=192.168.1.50 ./scripts/up.sh
A database looks wrong after changing config. The demo databases are only created
when the volume is empty. Run down -v and start again.
What to do next
- Build your first network of assets — see the guide on building a resource network.
- Set up a dataset and a policy — see the guide on data governance and policies.