Synthetic data
Synthetic data is data nobody measured. A computer that knows how a piece of equipment behaves works out what the readings would have been in a situation you have never actually had, and those numbers are written down in the same form as real ones.
It exists because the failure you most want to predict is almost always the one you have barely seen, and nothing learns a pattern from three examples. It is also the fastest way to ruin a data set permanently, so the separation rules on this page matter as much as the technique.
What it actually is
Ordinary data is a record of something that happened. Synthetic data is a record of the same kind, for something that did not.
Two words first, because everything here is built from them. A datapoint is one reading with a time attached: 71.2 °C at 09:14 on the pump's bearing. The datapoints from one sensor, in order, are a time series. Time series →
Synthetic data means datapoints that no sensor produced. Something else worked out what the reading would have been, and it was stored in exactly the same shape: same equipment, same timestamps, same units, same charts.
The closest everyday comparison is a flight simulator. A pilot trains for an engine failure by watching a full set of instruments behave exactly as they would in a real one. Nothing is burning, no sensor measured anything, and the training still works, because the readings are realistic enough to learn from. Synthetic data is that, written to a data set instead of a cockpit.
Scale is the thing people underestimate. Three real bearing failures might amount to a few thousand datapoints across the days that led up to each one. Generate three hundred variants, each at a slightly different load, temperature and speed, and you have hundreds of thousands of datapoints describing the same failure arriving under conditions you have not personally lived through. That is the difference between having a story and having a pattern.
Why you would want that
The trap is that this looks like a modelling failure and gets treated as one. Teams try another algorithm, then another, then conclude that machine learning does not work for their plant. The actual finding was available at the start: there were three examples. No method fixes that, and the honest options are to gather more, which means waiting for more failures, to borrow from similar equipment, or to generate.
Where generated data comes from
| Source | How it works | Best for |
|---|---|---|
| Physics or a simulation | A model of how the equipment behaves is run under conditions you choose, including the failure conditions you have never survived | Rare failures, and anything where the mechanism is understood |
| A digital twin | The same idea, driven by your actual configuration rather than a generic model | Plant-specific behaviour, commissioning, operator training |
| A statistical model fitted to real data | Learns the distribution of what you have seen, then draws new samples from it | Filling out a class that is thin rather than empty |
| Deliberate variation of real examples | Take the three real failures and vary load, ambient, speed within plausible ranges | The quickest useful option, and the easiest to explain |
The last row is worth noticing because it needs no new technology at all. In machine learning this is usually called data augmentation, and for industrial signals it is often enough.
Where it earns its place
| Situation | Why generating helps |
|---|---|
| Rare failures | Turns three examples into a training set, with conditions varied the way they vary in life |
| Equipment with no history | A new plant, or a machine installed last month, can borrow from a model of how its type behaves |
| Sensitive data | Patient, personnel or commercially confidential records that cannot leave a boundary can be represented by generated data carrying the pattern but not the person |
| Testing before commissioning | Alerts, dashboards, integrations and agents can be exercised against generated conditions long before the real ones occur |
That last one is quietly the most useful and the least discussed. It lets you find out that your alarm logic is wrong in a workshop rather than during the event it was built for.
The field that runs entirely on this: robotics
If you want to see how far this technique goes, look at what is happening in robotics right now, because the whole field is bottlenecked on training data and has organised itself around generating it.
The reason is worth understanding, because it is the opposite of the situation with text and images. Everything a language model learned from already existed and could be downloaded. The data a robot needs, what it feels like to grip a fitting that is slightly the wrong size, does not exist anywhere. Somebody has to produce it, one attempt at a time.
Two techniques do the work, and they are used together:
- Imitation learning. A person performs the task, usually by operating the robot remotely, and the robot learns to copy. A few hundred good demonstrations get it to competent, which is remarkable, and a few hundred is about all a person can produce in a day, which is the problem.
- Reinforcement learning. The robot tries, scores the result against a reward, adjusts, and tries again. It works, and it needs millions of attempts. So the attempts happen in a simulator, where a fleet of virtual robots practises for a thousand hours in an afternoon, with the physics deliberately varied so the robot learns the task rather than one exact world. Only the final validation happens on real hardware.
That last sentence is the rule from the section below, arrived at independently by a different field: train on generated data, test on the real thing.
And the volumes are extraordinary. A rig used to teach a robot manipulation carries several cameras, typically one or two mounted at the wrist so they move with the gripper and two or three watching the workspace from fixed angles, alongside joint encoders, motor currents, force and torque sensing at the wrist, and tactile fingertips. Those fingertips are worth a footnote of their own: the most capable designs are literally small cameras inside the finger. A GelSight-style sensor, first built at MIT, puts a camera behind an opaque gel pad lit from several angles, so when the pad moulds itself around whatever it touches the camera reads the contact geometry as an image.
Add it up and half a dozen video streams at 1080p and 30 frames a second is on the order of a gigabyte every second before compression, so one robot practising for an hour produces terabytes. Robot learning is among the most data-intensive activities anyone currently attempts, and the overwhelming majority of that data never came from the world.
The robots arriving on industrial sites are trained exactly this way, and the simulator they practise in is a digital twin of somewhere real. An operation described well enough for a machine to work in it is the same asset whether the worker is software, a person or steel. The workforce will not stay virtual →
The discipline that keeps it honest
This section matters more than the technique. Every serious failure involving synthetic data is a plumbing accident rather than an act of dishonesty.
A model scored against data from its own generator has proved one thing: that it learned the generator. Validation belongs on real observations, always, however few there are.
Synthetic data cannot contain a failure mode nobody modelled. It multiplies your understanding; it does not extend it. A model trained mostly on generated data will be confident precisely where your simulation was confident, including where it was wrong.
Not a regulator, not a board, not an investor deck, not a KPI. This is the failure that ends programmes, and it is why the separation has to be structural rather than a matter of everyone remembering.
With lineage not yet shipped, nothing in the platform will tell you automatically that a figure has generated data in its ancestry. Until it does, the separation lives in your data sets, labels and naming, and in the discipline of the people building models. Decide the convention before you generate the first datapoint.
Where agents help
An agent can run the generation loop: vary the conditions, produce the cases, retrain, measure whether the model actually improved on real held-out examples, and record what it tried. That last part is what makes it safe to do at volume, because the alternative is a folder of generated files nobody can account for.
It also suits the sensitive-data case, where the agent that generates never needs to see, or be allowed to see, more than the boundary permits. How agents do this work →
- Synthetic data
- Data augmentation
- Training, validation and test data sets
- Simulation
- Class imbalance
- Robot learning
- Reinforcement learning
- Apprenticeship learning, the formal name for learning from demonstrations
- Teleoperation
- Tactile sensor, including the camera-behind-gel designs
- GelSlim: a high-resolution, compact tactile-sensing finger
- GelSight360: an omnidirectional camera-based tactile sensor
- What is machine learning?: what the generated data is for
- Feature extraction: the other half of making data learnable
- Data cleaning: generated data cannot rescue measurements nobody trusts
- Digital twin: the model that can produce conditions you have never survived, and the simulator a robot practises in
- Data sets: the boundary that keeps generated data separable
- Building AI agents: running generation and validation as standing work