Not industry custom, a paragraph of law
ValidationThe first step in VEE - automated checks of metering data against technical and logical limits to detect obviously erroneous values., estimationThe second step in VEE - calculation of missing or rejected metering values based on historical patterns, neighbouring meters or statistical models. and editingThe third step in VEE - manual correction of metering values that cannot be resolved automatically, with full audit trail for all changes. of metering values is often described as good practice. That undersells it. The requirement is written into the Norwegian metering regulation § 3-6, "Standard for validationThe first step in VEE - automated checks of metering data against technical and logical limits to detect obviously erroneous values., estimationThe second step in VEE - calculation of missing or rejected metering values based on historical patterns, neighbouring meters or statistical models. and editingThe third step in VEE - manual correction of metering values that cannot be resolved automatically, with full audit trail for all changes. of metering values for interval-settled metering points".
It is worth noting who the paragraph addresses. Responsibility rests with whoever establishes the metering values, the metering data responsible party. That is the grid company, the power producer or the district heating producer who collects, validates and passes on. Whoever merely receives approved values does not have that job, and cannot do it either.
Which means the quality of a metering valueOne measurement from one metering point for one period of time. The building block all settlement rests on. is decided early in the chain, by someone who often never meets the person who ends up paying for it.
What a validation rule actually looks for
The rules are not one filter, they are several different looks at the same value. Roughly three categories.
The physically impossible. Consumption exceeding what the installation can draw. A negative value where only positive ones can exist. Values too large to belong in the unit they are stated in.
The temporally wrong. Clock drift in the meter, meaning timestamps gradually sliding away from reality. Values missing from a series that is otherwise dense. Values arriving out of order, or twice.
The suspiciously quiet. A meter that is stuck does not deliver nothing. It delivers the same number, hour after hour, and a total that looks perfectly normal. That is the hardest fault to see with the eye and the easiest to see with a rule.
None of these asks whether the value is correct. They ask whether it could be.
Why the thresholds must be adjustable
A validationThe first step in VEE - automated checks of metering data against technical and logical limits to detect obviously erroneous values. rule always has a threshold, and the threshold is a choice with two failure modes.
Set it too tight and real values are rejected. A cabin visited for the first time in six months looks like a fault. An industrial plant starting up a line looks like a fault. Every false rejection becomes an exception a human has to look at, and a human who looks at a hundred false exceptions stops looking properly at the hundred and first.
Set it too loose and the faults pass through, to be discovered later, in the settlementTurning metering data into money. Consumption and production per metering point become what each party owes or is owed., at the customer, or in a complaint.
That is why thresholds and business rules are configurable, and why that is no excuse to leave them at their defaults. A grid with many holiday cabins has a different normal distribution than a grid of flats. The rules should recognise the grid they are actually looking at.
Passing does not mean true
This is the sentence worth carrying onwards.
A value that passes all eight rules has not been proven correct. It has survived eight attempts to prove it wrong. That is something else entirely, and the difference is the whole discipline.
A meter can deliver a wrong value for months without a single rule firing, as long as the error is small enough, steady enough and physically possible. Faults like that are not found through validationThe first step in VEE - automated checks of metering data against technical and logical limits to detect obviously erroneous values.. They are found through deviation against something else: against the neighbouring installation, against the same period last year, against the total in the substation above.
ValidationThe first step in VEE - automated checks of metering data against technical and logical limits to detect obviously erroneous values. is therefore the floor, not the ceiling. It catches the obvious automatically, so that people can spend their time on what is not obvious.
Same engine, different values
One thing that surprises many: different measurement objects do not require different systems.
Consumption from an end-user meter, production from hydro, wind or solar, and heat from a meter in a district heating network all go through the same workflow. What separates them is the values and the rules, not the process. A wind turbine and a flat have different physical limits, but both must be validated, both can have missing values, and both must be documentable afterwards.
That is a useful test of a system: can it handle production as well as consumption, without being two systems in one?
Where we have built it
The eight rules, the configurable thresholds and the shared workflow for consumption, production and district heating are how we have built SEFTATvimenning’s VEE engine: validates, estimates and edits metering series with versioned time series and a full audit trail, before data goes to Elhub., our engine for validationThe first step in VEE - automated checks of metering data against technical and logical limits to detect obviously erroneous values., estimationThe second step in VEE - calculation of missing or rejected metering values based on historical patterns, neighbouring meters or statistical models. and editingThe third step in VEE - manual correction of metering values that cannot be resolved automatically, with full audit trail for all changes.. It is built against metering regulation § 3-6, not against an interpretation of what counts as good practice.
Where it stands today: verified against simulated market interfaces, ready to move into the platform, and not in production at a customer yet. We will say so when it is.