Video AI technology

Two kinds of models

Large models for training, small models at the site.

The most capable video models are too large to run on every camera, all day, at every site. Small models are fast enough to run there, and their accuracy depends on what they learn from, so VideoBI uses the large models to teach the small ones.

Teachers · in training

Frontier video models

The largest, most capable video AI models watch footage and describe it: every car in the lane, every pan in the cabinet, every trailer at the door. They label at a scale and consistency no team of people could, and they find the rare cases a small model needs to see.

Label and describe footage at scale
Surface rare and hard cases
People review what the models are unsure of

Students · on site

Edge models, fine-tuned per task

Compact detection and tracking models are fine-tuned on those labels, one task at a time, then compressed for the edge device's accelerator. They read every stream in real time, inside the building, so the video never has to leave it.

One model per task, many camera angles
Real time on every stream at the site
Updated remotely when a better model passes its benchmark

Frontier-class AI on a small device

A large model's accuracy in a small model.

The main engineering challenge is running video AI in real time on every camera at a site, on a device the size of a book. VideoBI's edge technology compresses what the frontier models know into models small and fast enough to do it, and keeps nearly all of the accuracy.

Distillation. The compact model learns to match the frontier model's answers on the task it was built for.
Quantization. Weights and activations in low precision, so the model runs on the device's AI accelerator.
Compilation. Each model compiled for the device it runs on, with every stream at the site scheduled together.

50×placeholdersmaller than the frontier model that taught it

96%placeholderof the teacher's accuracy kept on the benchmark

18 msplaceholderper frame on the edge device

< 1 splaceholderfrom camera to reading in the cloud

How a model is made

From frontier model to edge device, and back.

Every model follows the same path, and passes the same gate. What the models get wrong in the field becomes the next round's training.

Training01LabelFrontier video models label and describe footage at scale.
Training02ReviewPeople review the cases the models are unsure of.
Training03Fine-tuneCompact models learn one task each, across sites and angles.
Training04CompressDistilled, quantized and compiled for the device's accelerator.
Benchmark gate05BenchmarkScored against hand-counted video. Ships only if it beats the model in the field.
Edge fleet06DeployRolled out remotely to edge devices, site by site.

07 · Learn from the field. Readings the models were unsure of, and alerts people dismissed with a reason, become the next round's hard cases.

What the models see

Trained for the work of physical operations.

Each task has its own model, trained on the scenes where it matters. Here is what they read in a single frame.

A drive-thru lane with each car boxed, numbered and scored

edge · 10 cars tracked · lane wait 3:42

Tags and scores are illustrative.

Task	What the model sees	What it writes
Lines and lanes	People in a line, cars in a drive-thru lane, each tracked through the stages	count, wait, time at each stage, walk-outs
Counters and registers	A register staffed or not, a guest reaching the counter	staffed minutes, capture rate
Holding pans	How full each pan in a holding cabinet is	fill level, hold time, batches cooked and thrown away
Tables and seating	Tables occupied, left, and waiting to be cleared	occupancy, time to clear
Dock doors	A trailer at the door, a crew working it	door occupancy, time to crew, turn time
Lines and stations	Units crossing a line on a conveyor, people at each station	throughput, staffed stations, work waiting
Yards and gates	Trucks and trailers arriving, parking, leaving	arrivals, yard fill, gate-to-door time

Benchmarks

Every model is benchmarked against hand counts.

Every model is scored against video counted by hand, frame by frame, before it ships. A new model replaces the one in the field only when it scores better on the same benchmark.

Task	Measured as	Against	Result
Counting in a zone	Count accuracy per interval	People and vehicles counted by hand, frame by frame	97.8%placeholder
Waits and stage times	Error in seconds against the true time	Arrival and departure marked by hand for each visit	±4.2 splaceholder
Fill levels	Error in the level of each pan	Levels read by hand from the same frames	±4.6 ptsplaceholder
Events	Precision and recall	Every event marked by hand: a walk-out, a crossing, a crew start	96.1% · 94.7%placeholder

Scored again at every site

At go-live, counts at the new site are checked against hand counts, so each camera angle is measured, not assumed.

Confidence on every reading

Each reading carries the model that made it and its confidence. Intervals the model was unsure of are left out of the metrics.

Coverage on every figure

When a camera was down or blocked, the figures it feeds say so, instead of reading low.

Edge performance

Real time on every stream.

Each edge device runs its models on every camera at the site, continuously, with room for more than one model per stream.

	Edge S	Edge M	Edge L
Video streams	4placeholder	16placeholder	64placeholder
Analysis rate per stream	15 fpsplaceholder	15 fpsplaceholder	15 fpsplaceholder
Inference time per frame	22 msplaceholder	18 msplaceholder	12 msplaceholder
Models per stream	up to 3placeholder	up to 4placeholder	up to 6placeholder
AI compute	20 TOPSplaceholder	100 TOPSplaceholder	400 TOPSplaceholder
Power	15 Wplaceholder	40 Wplaceholder	150 Wplaceholder

The edge devices →

The team

Built in Santa Clara.

VideoBI's engineering team works from Santa Clara, California, on one problem: making the most capable video AI run on small devices, reliably, at every site.

130+engineers across LatticeWork

10+people on LatticeWork's AI team

Fine-tuningAdapting the most capable video models to the tasks of physical operations.
Edge inferenceRunning those models in real time on small devices, on every stream at a site.
Video understandingDetection, tracking and scene reading across lanes, kitchens, docks and yards.
BenchmarksHand-counted ground truth, and the gate every model must pass before it ships.

Language models

The most capable language models, for the reasoning.

Ask uses large language models to investigate a question the way an analyst would: which sites, which hours, which cause. The models plan and explain; the numbers come from the datasets, and every figure in an answer is checked against the data.

How Ask works →

Privacy in the model

Built to count and time.

The models detect and track within one camera view, with short-lived track numbers. They don't recognize faces or follow a person from camera to camera, and people can be blurred on the device before any still leaves it.

Security and privacy →

See the models at work on a live chain.

Talk to sales Explore the live demo

State-of-the-art video AI, at the edge.