Annotation Maker — Local CV Workbench
Takes a folder of video to a trained detection model on your own GPU, without a single byte leaving the machine.
A local-first computer-vision workbench: import video, extract frames, annotate them, build immutable dataset versions, train YOLO models on a local GPU, evaluate the results and run inference — with no cloud storage, hosted GPU or external annotation service anywhere in the loop.

- Industry
- Computer Vision Tooling
- Role
- Architect & Full-Stack Engineer
- Type
- Desktop App
- Year
- 2026
01
Overview
Annotation Maker is the whole detection pipeline — Import → Extract → Annotate → Dataset → Train → Evaluate → Infer → Export — running as one FastAPI process on a single machine. It was built because the alternative for a small team is stitching together a hosted annotation tool, a cloud bucket and a rented GPU, then paying for all three and handing over the footage.
The interesting engineering is in the job model. Frame extraction, dataset builds, training and inference are launched by a supervisor inside the API but execute as separate OS processes with all state in SQLite. That makes cancellation real — a training loop cannot be interrupted from another thread — keeps a CUDA out-of-memory error from taking the API down with it, and lets an interrupted job resume after a restart. There is no message broker to install.
Two rules hold the data model honest. Dataset versions are immutable: a version copies annotation geometry at snapshot time rather than referencing live rows, so editing a box later cannot silently change what a model was trained on. And predictions are never ground truth — model output is stored separately from annotations, and promoting a prediction is an explicit action.
GPU use is explicit rather than best-effort: asking for CUDA when CUDA is unavailable fails immediately with a diagnostic instead of quietly falling back to CPU, because a training run that drops to CPU wastes hours before anyone notices.
02
Business challenge
Building a detection model meant renting three services — a hosted annotation tool, cloud storage and a GPU — and uploading the source footage to all of them. For work on private or client-owned video that is both a recurring cost and a disclosure problem, and the round trip between labelling and training made iteration slow.
What discovery showed
Traced where time actually went on a real detection project and found it was not labelling — it was the hand-offs: exporting a dataset, uploading it, configuring a training job, then pulling results back to decide what to label next.
03
Solution
Collapsed the entire loop into one local application. A FastAPI process serves the built React SPA and owns a SQLite database of metadata and paths; large assets stay on disk, dataset versions hardlink frames instead of copying them, and training runs against the local GPU. Labelling, training and evaluation are three clicks apart, so the next labelling decision is informed by the last run.
What it does
04
Product screenshots
Screens from the application itself. Select any image to view it full size.
05
Architecture
How it fits together
- 01FastAPI serves both the REST API and the built React/Vite SPA — one process runs the whole application
- 02SQLite (WAL) holds metadata and paths only; video, frames, weights and exports live on the filesystem
- 03Job supervisor inside the API launches each long task as a separate OS process, with all state in SQLite
- 04Ultralytics YOLO on PyTorch with an explicit CUDA device policy — no silent CPU fallback
- 05Immutable dataset versions snapshot annotation geometry and hardlink frames into versioned directories
- 06Alembic migrations over a workspace directory that is separate from the code checkout
06
Engineering challenges
- Challenge
- A training loop cannot be interrupted from another thread, so in-process jobs made cancellation a lie and a CUDA out-of-memory error took the API down with it.
- How it was solved
- Moved every long job into its own OS process supervised by the API, with all state in SQLite. Cancellation became a real kill, a crashing job stopped taking the server with it, and an interrupted run could resume after a restart — without adding a message broker.
- Challenge
- Editing an annotation after training silently changed what a past model had been trained on, making runs impossible to compare or reproduce.
- How it was solved
- Made dataset versions immutable — a version copies annotation geometry at snapshot time rather than referencing live rows, and every training run records the exact dataset version hash it used.
- Challenge
- Frames extracted from the same video are near-identical, so a random train/validation split leaks the validation set into training and reports a wildly optimistic mAP.
- How it was solved
- Split video-aware: all frames from one source video land on the same side of the split, so validation scores reflect unseen footage.
07
Technology stack
- Frontend
- ReactViteTypeScript
- Backend
- FastAPIPython 3.13
- Data
- SQLite
- AI & Processing
- Ultralytics YOLO
- Also used
- PyTorchCUDAAlembicFFmpeg
08
Business impact
- 8
- Pipeline Stages
- Local GPU
- Runs On
- None
- Data Leaves Machine
The full detect-model loop — import, label, version, train, evaluate, infer — runs on one machine with no cloud dependency and no per-seat or per-GPU-hour cost. Footage never leaves the device, which makes the tool usable on client-owned and private video that could not be uploaded at all.
09
Project status
What I took from it
- Process isolation, not threads, is what makes cancellation and crash recovery real for GPU work.
- Immutability at the dataset boundary is the cheapest way to make ML experiments reproducible.
- Failing loudly on a missing GPU beats a fallback that silently wastes hours.
Next step
Need a similar solution for your business?
Tell me what you are trying to build or fix. I will come back with how I would approach it, what it realistically takes and where the risks are.


