Skip to main content

Oil & gas — upstream production monitoring

The problem. An offshore field has dozens of wells, each with an electric submersible pump (ESP) and a cluster of sensors — wellhead pressure, casing temperature, flow rate. Engineers in an onshore control room need a live picture of every well, fast roll-ups for daily production reports, and an alarm the moment a pump trends toward failure.

This scenario wires that together: model the field as a graph, stream sensor data in at volume, and raise an event when a reading goes out of bounds.

Set up demo data​

New workspace? Run this once (Python) to create the well's sensor series and feed in a pump-intake pressure that sags toward the anomaly threshold — so the alarm in step 3 actually fires. The field/well/pump graph is created by step 1 below. Safe to re-run.

import datahub_sdk, numpy as np, pandas as pd

client = datahub_sdk.DataHubClient.from_env()

sensors = {"wellhead_pressure_bar": "bar", "casing_temperature_c": "deg_c",
           "flow_rate_bpd": "bpd", "pump_intake_pressure_bar": "bar"}
for s, u in sensors.items():
    client.timeseries.create([datahub_sdk.TimeSeries(external_id=s, name=s, unit=u, value_type="float")])

# intake pressure: healthy ~95 bar, then sagging below the 80-bar ESP-anomaly line
idx = pd.date_range(end=pd.Timestamp.now(tz="UTC"), periods=120, freq="1min")
intake = np.full(120, 95.0); intake[-30:] -= np.linspace(0, 20, 30)
client.timeseries.insert_from_lists(timestamps=idx, values=intake, ts="pump_intake_pressure_bar")

1. Model the field​

A field contains wells; a well contains a pump and produces sensor series. Modeling it as a graph lets you walk from a field down to any individual sensor.

ResourceForm field = new ResourceForm();
field.setExternalId("field_north_sea");
field.setName("North Sea field");
field.setLabels(List.of("Field"));

ResourceForm well = new ResourceForm();
well.setExternalId("well_a12");
well.setName("Well A-12");
well.setLabels(List.of("Well"));

ResourceForm pump = new ResourceForm();
pump.setExternalId("pump_esp_a12");
pump.setName("ESP — Well A-12");
pump.setLabels(List.of("Pump"));

RelForm fieldWell = new RelForm();
fieldWell.setName("contains");
fieldWell.setFromExternalId("field_north_sea");
fieldWell.setToExternalId("well_a12");

RelForm wellPump = new RelForm();
wellPump.setName("contains");
wellPump.setFromExternalId("well_a12");
wellPump.setToExternalId("pump_esp_a12");

client.resources().create(List.of(field, well, pump), List.of(fieldWell, wellPump));

Create a series per sensor the same way — wellhead_pressure_bar, casing_temperature_c, flow_rate_bpd — and relate each to well_a12.

2. Stream sensor data at volume​

Each well emits readings every second across many sensors — easily millions of points an hour at field scale. Hand them to the ingester grouped by series; it chunks and parallelises the writes. See high-throughput ingestion for the full mechanics.

client.timeseries().ingest(Map.of(
    "wellhead_pressure_bar", pressureReadings,   // List<Datapoint>
    "casing_temperature_c", temperatureReadings,
    "flow_rate_bpd", flowReadings));

3. Alarm on an out-of-bounds reading​

A creeping intake pressure is an early sign of an ESP problem. Read the recent window and raise an esp_anomaly event when it crosses the line — the control room queries open events for the field, and the event references the well it concerns.

var series = client.timeseries().retrieve(lastHourOf("pump_intake_pressure_bar"))
    .getItems().get(0);

if (series.getDatapoints().stream().anyMatch(p -> Double.parseDouble(p.getValue()) < 80.0)) {
    EventModel event = new EventModel();
    event.setExternalId("esp_anomaly_a12_" + System.currentTimeMillis());
    event.setType("esp_anomaly");
    event.setStatus("open");
    event.setMetadata(Map.of("well", "well_a12", "pump", "pump_esp_a12"));
    event.setEventTime(ZonedDateTime.now());
    client.events().create(List.of(event));
}

4. Are two alarms one fault? Ask the graph​

On a busy platform, alarms rarely arrive alone. When a wellhead temperature alarm and a separate pump alarm fire together, the control room needs to know fast: two independent problems, or one upstream cause? The graph answers it. Walk out from each alarmed sensor and look for a subsystem they share — if both are PART_OF the platform cooling_system, a single cooling failure is driving both.

ResourceNetwork na = client.resources().fetchRelated("wellhead_temp_a12", 5);
ResourceNetwork nb = client.resources().fetchRelated("pump_temp_a12", 5);

Set<String> aNodes = na.nodes().stream()
    .map(Resource::getExternalId).collect(Collectors.toSet());
Set<String> shared = nb.nodes().stream()
    .map(Resource::getExternalId).filter(aNodes::contains).collect(Collectors.toSet());

// shared contains "cooling_system" → one root cause, not two

That single query turns two alarms into one incident — see Correlate alarms with the graph for the general pattern.

See the result​

Chart the intake pressure the alarm watches — it sags below the 80-bar line in the last half hour, which is exactly what trips the esp_anomaly event:

import matplotlib.pyplot as plt

rf = datahub_sdk.RetrieveFilter(ts="pump_intake_pressure_bar",
    start=pd.Timestamp.now(tz="UTC") - pd.Timedelta(hours=2), end=pd.Timestamp.now(tz="UTC"))
s = pd.Series({pd.to_datetime(p.timestamp): float(p.value)
               for p in client.timeseries.retrieve_datapoints(rf)[0].get_datapoints()}).sort_index()
s.plot(title="Pump intake pressure (bar)"); plt.axhline(80, color="r", ls="--"); plt.show()
# the line drops from ~95 to ~75 bar — below the threshold, so the alarm fires

See also​