SDK Overview

Perception SDK

Cameras in, detections, tracks, counts and events out. An SDK for the whole life of a vision system: govern the data, train and approve the model, and run it on your own GPUs, with every stage tied to the one before it.

Why Perception SDK

Vision projects rarely fail at the model. They fail at the joins. Data sits in folders on a shared drive, training happens on someone's laptop, the model arrives as a file, and the pipeline running it was hand-written on a box at the site. Each part works. Nothing connects them - so when accuracy slips three months in, nobody can say what changed.

Perception SDK makes the joins the product. Data is frozen into numbered versions. Models train against a version and ship only if they clear a bar you set. An approved model drops straight into the runtime, and what the runtime sees in the field comes back as the next version of the data. Every step knows what produced it - and all of it runs on your footage, your GPUs, your account.

What makes it different

One lifecycle, not three disconnected stages

A data version makes a model version makes a running pipeline, each pinned to the last. Ask what is serving on camera 12 and you can walk back to the model, the data it learned from and the files inside it - months later, from the record rather than from memory.

Shipping is a rule, not a judgement call

A new model replaces the live one only if it clears rules you declared in advance. The gate separates a real improvement from a lucky test set, checks the model holds up for every group in the data rather than only on average, and throws out any result where the test data overlapped the training data. It also enforces the speed, size and memory budget of the device the model is headed for.

The runtime comes with it

Most platforms hand you a trained model and stop. Here the Perception Engine is part of the SDK: it turns one configuration file into a whole running pipeline, and an approved model loads into it with no conversion step - and a check that proves the exported model still agrees with the one you trained.

It tells you when it is going wrong

Point the monitor at a batch of live data - labelled or not - and it reports how far that data has moved from what the model learned, and what that has likely done to accuracy. The answer comes back as one word anyone can act on: fresh, aging or stale.

How it works

Four stages. The first two produce a candidate, the third decides whether it ships, and the fourth runs in production and feeds the first one back.

1. Bring the data in

Point the SDK at a folder of images, video frames or tables. It imports them into a governed store, drops duplicates, recognises the annotation layout already there, and records where every file came from. Before any GPU time is spent you can see what is actually in the set, what changed since the last version and which files are broken - and repair them in one recorded step. Commit, and the set freezes into a numbered version.

2. Train and score

Train from the built-in catalogue, or from your own code, in a single call - the settings that normally need a specialist are chosen for you. Scoring runs through one engine covering every kind of model the platform supports, so numbers stay comparable between models and over time. It ranks what data to fix first, and picking any pair of classes the model confused gives you back the exact examples.

3. Gate and ship

The gate decides. Past it, the model exports as a bundle the runtime runs as it stands, alongside smaller variants for camera-class devices. From there a deployment record tracks what is live: point a target at a version, roll it back, retire it - every move written to a log nothing can be edited out of.

4. Run and close the loop

The Perception Engine takes one file: your video sources, a detector, a tracker to hold identities across frames, and whatever else the job needs - body pose, exact outlines, a second model classifying what the first one found, text recognition for numbers and plates. Rules you define count what crosses a line or sits inside a marked area. Output goes to a screen, a file, a stream or your event system, several at once, from one pass over the footage.

What it sees in production is written down and sampled for review. Approve a batch and it becomes the next data version, which goes back through stage two and the same gate. The model improves from the field, but only ever through the bar a human release would have to clear.

Three packages, one lifecycle

Perception SDK ships as three installable packages, designed to work as one lifecycle and usable on their own.

Built for production

A record that holds up

A version cannot be altered once written, so any result can be reproduced from the exact data behind it. An activity log turns "who broke this dataset" into a query rather than an investigation. Holds stop anything under audit being cleaned up, and an erasure request deletes the bytes for real and leaves proof it happened.

Fixing the data, not just the model

A handful of bad labels usually caps accuracy more than the choice of model does. The platform finds the examples that look mislabelled and the ones that are genuinely hard, and hands you a folder to review; corrections come back as a new version and a gated retrain. It also picks which unlabelled samples are worth labelling next, so you pay only for the ones that move the model.

Retraining with a handbrake

Retraining fires on a schedule, when the incoming data has visibly moved, or when accuracy drops. Two safeguards keep it honest: a cap on how often it may run, and a check that holds back any new model which got worse on data the old one handled fine.

Room to grow, and room to shrink

Training spreads across several GPUs and hosts, so a big job finishes in a day rather than a week, and it can run on a managed fleet without your data leaving your infrastructure. The same model also exports small enough for camera-class hardware - the Perception Engine is deployed on NVIDIA, Qualcomm and Huawei - and the result is verified on the target before a pipeline relies on it.

Enterprise from the start

Single sign-on and role-based access plug into the identity system you already have, and residency rules pin a workspace's data to a region. Storage and GPU time are allocated per tenant, so one heavy user cannot crowd out the rest. Each version is signed and checked on the way out, so a shipped model can be proven to be the one that was approved.

Drive it the way you work

Everything is available three ways: the Python SDK, a command-line tool, and versioned HTTP APIs. A whole data or model job can also be written as a single file and run unattended in your build system, so other teams and non-Python services drive the platform without embedding anything.

Where it fits

• Sports and broadcast - hold identity on every player across a multi-camera feed, read numbers off the video, and turn positions into the counts an analyst reports on.
• Retail, venues and public space - occupancy, queues, dwell and line crossings, counted at the edge and published as events other systems can use.
• Industrial safety and quality - posture and gesture, exact outlines where contact and area matter, and a second model reading grade or defect off each object found.
• Transport and traffic - plates and labels read off the stream, with a site's whole camera set running as one job on one GPU.
• Regulated environments - where a model has to arrive with a paper trail, an erasure request has to be provable, and licences have to be cleared before anything ships.

What's next

Perception SDK ships today with everything above. Next on the roadmap:

• Marketplace distribution and usage metering - the hooks for licence checking and usage reporting are in place and switched off by default; the connector follows.
• Automated data policies - datasets that version themselves when their contents actually change, and per-workspace limits.
• Broader runtime coverage - more model types validated end to end, from export through to a running pipeline.

Get started

• User Guide - full documentation on the Perception SDK docs site