Skip to content
AMCase studyComputer Vision · MLOpsDelivered

Annotation Maker — Local CV Workbench

Takes a folder of video to a trained detection model on your own GPU, without a single byte leaving the machine.

A local-first computer-vision workbench: import video, extract frames, annotate them, build immutable dataset versions, train YOLO models on a local GPU, evaluate the results and run inference — with no cloud storage, hosted GPU or external annotation service anywhere in the loop.

Annotation Maker — Local CV Workbench interface
Industry
Computer Vision Tooling
Role
Architect & Full-Stack Engineer
Type
Desktop App
Year
2026

01

Overview

Annotation Maker is the whole detection pipeline — Import → Extract → Annotate → Dataset → Train → Evaluate → Infer → Export — running as one FastAPI process on a single machine. It was built because the alternative for a small team is stitching together a hosted annotation tool, a cloud bucket and a rented GPU, then paying for all three and handing over the footage.

The interesting engineering is in the job model. Frame extraction, dataset builds, training and inference are launched by a supervisor inside the API but execute as separate OS processes with all state in SQLite. That makes cancellation real — a training loop cannot be interrupted from another thread — keeps a CUDA out-of-memory error from taking the API down with it, and lets an interrupted job resume after a restart. There is no message broker to install.

Two rules hold the data model honest. Dataset versions are immutable: a version copies annotation geometry at snapshot time rather than referencing live rows, so editing a box later cannot silently change what a model was trained on. And predictions are never ground truth — model output is stored separately from annotations, and promoting a prediction is an explicit action.

GPU use is explicit rather than best-effort: asking for CUDA when CUDA is unavailable fails immediately with a diagnostic instead of quietly falling back to CPU, because a training run that drops to CPU wastes hours before anyone notices.

02

Business challenge

Building a detection model meant renting three services — a hosted annotation tool, cloud storage and a GPU — and uploading the source footage to all of them. For work on private or client-owned video that is both a recurring cost and a disclosure problem, and the round trip between labelling and training made iteration slow.

What discovery showed

Traced where time actually went on a real detection project and found it was not labelling — it was the hand-offs: exporting a dataset, uploading it, configuring a training job, then pulling results back to decide what to label next.

03

Solution

Collapsed the entire loop into one local application. A FastAPI process serves the built React SPA and owns a SQLite database of metadata and paths; large assets stay on disk, dataset versions hardlink frames instead of copying them, and training runs against the local GPU. Labelling, training and evaluation are three clicks apart, so the next labelling decision is informed by the last run.

What it does

Video import with FFmpeg probing and in-place referencing (no copy)
Frame extraction at a chosen stride, with thumbnails and a scrub timeline
Annotation canvas with class management, review queue and keyboard flow
Model-assisted pre-labelling, with every prediction requiring an explicit accept
Immutable dataset versions — geometry snapshotted, frames hardlinked not copied
Video-aware train/validation splits so near-identical frames cannot leak across
YOLO training on a local GPU with live loss and mAP curves
Evaluation with confusion matrix and side-by-side ground-truth vs prediction review
Inference on new footage plus model export
Background jobs as separate OS processes — real cancellation, crash isolation, resume

04

Product screenshots

Screens from the application itself. Select any image to view it full size.

05

Architecture

How it fits together

  1. 01FastAPI serves both the REST API and the built React/Vite SPA — one process runs the whole application
  2. 02SQLite (WAL) holds metadata and paths only; video, frames, weights and exports live on the filesystem
  3. 03Job supervisor inside the API launches each long task as a separate OS process, with all state in SQLite
  4. 04Ultralytics YOLO on PyTorch with an explicit CUDA device policy — no silent CPU fallback
  5. 05Immutable dataset versions snapshot annotation geometry and hardlink frames into versioned directories
  6. 06Alembic migrations over a workspace directory that is separate from the code checkout

06

Engineering challenges

Challenge
A training loop cannot be interrupted from another thread, so in-process jobs made cancellation a lie and a CUDA out-of-memory error took the API down with it.
How it was solved
Moved every long job into its own OS process supervised by the API, with all state in SQLite. Cancellation became a real kill, a crashing job stopped taking the server with it, and an interrupted run could resume after a restart — without adding a message broker.
Challenge
Editing an annotation after training silently changed what a past model had been trained on, making runs impossible to compare or reproduce.
How it was solved
Made dataset versions immutable — a version copies annotation geometry at snapshot time rather than referencing live rows, and every training run records the exact dataset version hash it used.
Challenge
Frames extracted from the same video are near-identical, so a random train/validation split leaks the validation set into training and reports a wildly optimistic mAP.
How it was solved
Split video-aware: all frames from one source video land on the same side of the split, so validation scores reflect unseen footage.

07

Technology stack

Frontend
ReactViteTypeScript
Backend
FastAPIPython 3.13
Data
SQLite
AI & Processing
Ultralytics YOLO
Also used
PyTorchCUDAAlembicFFmpeg

08

Business impact

8
Pipeline Stages
Local GPU
Runs On
None
Data Leaves Machine

The full detect-model loop — import, label, version, train, evaluate, infer — runs on one machine with no cloud dependency and no per-seat or per-GPU-hour cost. Footage never leaves the device, which makes the tool usable on client-owned and private video that could not be uploaded at all.

09

Project status

Delivered2026

What I took from it

  • Process isolation, not threads, is what makes cancellation and crash recovery real for GPU work.
  • Immutability at the dataset boundary is the cheapest way to make ML experiments reproducible.
  • Failing loudly on a missing GPU beats a fallback that silently wastes hours.

Next step

Need a similar solution for your business?

Tell me what you are trying to build or fix. I will come back with how I would approach it, what it realistically takes and where the risks are.

Related work

All projects