Data & ML · Intermediate
Custom Image Classifier, Shipped
Fine-tune a vision model on your own labeled photos and serve real-time predictions from a web app with confidence scores.
You collect and label a small dataset of images, fine-tune a pretrained CNN or ViT backbone using timm and PyTorch, export the model to ONNX for fast inference, and wrap everything in a FastAPI endpoint deployed to Hugging Face Spaces or Modal. The result is a live URL that accepts an image upload and returns a ranked list of predicted classes with confidence scores. Building this end-to-end forces you to confront the real pain points of ML in production: data quality, class imbalance, latency constraints, and model versioning. It is a portfolio-ready artifact that demonstrates the full loop from raw photos to a callable API.
What you build
- Fine-tunes a pretrained EfficientNet or ViT backbone on a custom labeled image dataset using timm
- Applies data augmentation (random crop, flip, color jitter) via torchvision transforms to reduce overfitting on small datasets
- Exports the trained model to ONNX format and runs inference through ONNX Runtime for CPU-friendly production serving
- Serves predictions via a FastAPI endpoint that accepts multipart image uploads and returns class labels with confidence scores
- Hosts the full stack on Hugging Face Spaces (Gradio or Docker) or Modal so anyone can hit the API without local setup
- Displays a ranked top-3 prediction bar chart in the web UI so confidence scores are immediately readable
What it teaches
- Transfer learning mechanics: freezing backbone layers, replacing the classification head, and unfreezing for fine-tuning
- ONNX export and runtime inference, including input/output naming and dynamic batch axes
- FastAPI request handling for binary file uploads, input preprocessing pipelines, and structured JSON responses
- Practical data augmentation strategies for small datasets and how to measure their effect on validation accuracy
- Deploying a Python ML service to a serverless or container-based cloud target with cold-start considerations
How it works
- 1
Labeled Dataset
- Folder per class
- 50-200 imgs/class
↓ loads into
- 2
Fine-Tune
- timm backbone
- PyTorch trainer
- Val accuracy log
↓ exports as
- 3
ONNX Model
- Frozen graph
- CPU-optimized
↓ loaded by
- 4
FastAPI Server
- Image upload route
- Preprocessing pipeline
↓ hosted on
- 5
Cloud Deploy
- HF Spaces / Modal
- Docker container
↓ returns
- 6
Predictions UI
- Top-3 classes
- Confidence scores
Sign in to open the build guide
Free account. Get the step-by-step build and every resource link.
Take it further
- Add an active learning loop: log low-confidence predictions to a review queue, let you label them in a simple UI, and retrigger fine-tuning automatically.
- Implement Grad-CAM visualization so the web response overlays a heatmap on the image showing which pixels drove the top prediction.
- Add model versioning with MLflow or a simple JSON manifest so you can roll back to any prior checkpoint from the API without redeploying.
