FoundryNet / Answers / How do you normalize industrial data?
Answers

How do you normalize industrial data?


Normalizing industrial data means mapping every vendor's tag onto one shared field, and then guaranteeing something about the value behind it. Most projects do the first half and skip the second, which is why the resulting dataset still cannot be queried across machines.

The naming layer is what people expect: SPINDEL_AUSLASTUNG, SP LOAD, SLOAD and spindle_load all become spindle_load_pct. It is tedious but conceptually simple, and every plant has a half-finished spreadsheet attempting it.

The layer underneath is where the value is, and where the failures are. A canonical field has to carry a unit (feed rate arrives as mm/min, in/min, mm/s, m/s and ft/min — a 25.4x error looks entirely plausible on a chart), a sign convention (half of all power meters report export as positive and half as negative, so a mixed site totals to something quietly wrong), a measurement type (a cumulative counter resets, and a naive difference produces a large negative spike several times a shift), and physics bounds so a sentinel value like 65535 is rejected rather than stored.

A practical approach: pick canonical names from an existing schema rather than inventing one; make the unit part of the field definition, not a separate column somebody forgets to join; validate at write time and again at read time, because a sensor that fails stuck-at-value passes every bounds check ever written; and keep commanded and actual values in separate fields, because the difference between them is usually the thing you are measuring for.


Worked example

Real tags from the corpus

# four vendors, one field, with the unit contract attached
{"SPINDEL_AUSLASTUNG (%)": 78}
{"SP LOAD (percent)": 78}
{"SLOAD": 78}
{"spindle_load_pct": 78} # unit %, bounds 0-100, instantaneous

Try it

No account, no key, no data leaving your machine

docker run -p 8000:8000 ghcr.io/foundrynet/forge-sandbox
Get an API key Canonical field reference

Related questions

What is a canonical field?

A single agreed name for a measurement, plus a contract about the value: its SI unit, which input units are accepted and converted, the sign convention, whether it is instantaneous or cumulative, and the physical range outside which a reading is rejected.

Should I build my own canonical schema?

You can, and the naming part is straightforward. The expensive part is the long tail of vendor spellings and the unit metadata behind each field — Forge's corpus holds 16,908 mappings, and the schema itself is published under MIT at github.com/FoundryNet/canonical-schema if you would rather start from it than from a blank sheet.

Where does normalization usually go wrong?

Units, and specifically units encoded in the tag name rather than in a unit field. A tag called ambient_temp_f holds Fahrenheit; a converter that assumes the field name is authoritative and converts anyway produces a value that has been converted twice.


Keep reading

All canonical fields spindle_load_pct Modbus registers to JSON Mixed-fleet normalization
More answers like this one, and the thesis behind them — The Machine Agent, weekly. Read it →
Get the thesis behind this infrastructure — The Machine Agent, weekly.