Live Object Detection

Real-time object detection system with mobile camera streaming and live browser visualization.

🚀 Quick Start

Prerequisites

• Docker & Docker Compose
• ngrok auth token (for public mobile access) - get from ngrok.com

Setup

1. Set ngrok token (required for mobile access):

export NGROK_AUTHTOKEN=your_token_here

2. Start all services:

docker-compose up --build

That's it! All services (backend, frontend, ngrok) will start automatically.

📱 Mobile Connection

Option 1: QR Code (Recommended)

1. Open browser at http://localhost:5173
2. Scan QR code with phone camera
3. Allow camera permissions when prompted

Option 2: Manual URL

• Local Network: http://[your-ip]:5173
• Public Access: Check http://localhost:4040 for ngrok URL

🎛️ Application URLs

After starting with docker-compose up:

• Frontend: http://localhost:5173
• Backend API: http://localhost:8000
• ngrok Dashboard: http://localhost:4040 (shows public URL)
• Health Check: http://localhost:8000/health

📊 Performance Monitoring

Monitor your application's performance using existing functionality:

# Start the application first
export NGROK_AUTHTOKEN=your_token_here
docker-compose up --build

Option 1: Run from Docker Container (Recommended)

# Enter the frontend container
docker exec -it frontend bash

# Run metrics collection
npm run bench

# View results
npm run metrics

Option 2: Run from Host Machine

# In a new terminal (application running)
cd frontend
npm install  # if not already done
npm run bench
npm run metrics

The metrics collector will automatically detect whether it's running inside Docker and use the appropriate URLs (backend:8000 vs localhost:8000).

Custom Duration

# 60-second monitoring
npm run bench 60

# 120-second monitoring  
npm run bench 120

The metrics collector will:

• ✅ Monitor your existing application endpoints
• ✅ Collect streaming statistics from /stream/stats
• ✅ Check service health and connectivity
• ✅ Generate realistic performance estimates
• ✅ Produce metrics.json with benchmarking data

For Real Metrics

To get actual performance data:

1. Start streaming from a mobile device
2. Open the frontend at http://localhost:5173
3. Use your phone to stream video while metrics are collected

Metrics Output

The system generates metrics.json containing:

• Latency: Estimated end-to-end response times
• Throughput: Target vs actual FPS based on streaming activity
• Bandwidth: Projected bandwidth usage for video streaming
• Connection: Service health and uptime monitoring
• Analysis: Performance assessment and recommendations

🏗️ Architecture

┌─────────────┐    WebSocket    ┌─────────────┐    HTTP/WS    ┌─────────────┐
│   Mobile    │────────────────▶│   Backend   │──────────────▶│   Browser   │
│   Camera    │                 │  (Python)   │               │  (React)    │
└─────────────┘                 └─────────────┘               └─────────────┘
                                       │
                                  ┌────▼────┐
                                  │ Object  │
                                  │Detection│
                                  │ (YOLO)  │
                                  └─────────┘

🔧 Development

Docker Setup (Recommended)

# Set environment variable
export NGROK_AUTHTOKEN=your_token_here

# Build and start all services
docker-compose up --build

# Stop services
docker-compose down

Manual Development Setup

# Backend
cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload

# Frontend (in another terminal)
cd frontend
npm install
npm run dev

# ngrok (in another terminal, optional)
ngrok http 5173

📈 Performance Optimizations

Low-Resource Mode

• Input resolution: 320×240 (configurable)
• Target FPS: 10-15
• Frame queue with overflow protection
• Dynamic quality adjustment

Backpressure Handling

• Fixed-length frame queue (default: 5 frames)
• Drop oldest frames when overloaded
• WebSocket flow control
• Adaptive frame rate based on processing speed

🛑 Stopping

docker-compose down

# To remove volumes and networks
docker-compose down -v

📝 Project Structure

├── docker-compose.yml       # Docker orchestration
├── ngrok.yml                # ngrok configuration  
├── config.env              # Environment configuration
├── backend/                 # Python FastAPI server
│   ├── Dockerfile
│   ├── requirements.txt
│   └── app/
│       ├── main.py         # FastAPI app
│       ├── websocket_service.py
│       ├── connection_service.py
│       └── api_service.py
├── frontend/               # React web app
│   ├── Dockerfile
│   ├── package.json
│   ├── scripts/
│   │   ├── benchmark.js    # Performance testing
│   │   └── view-metrics.js # Metrics viewer
│   └── src/
│       ├── App.jsx
│       ├── pages/
│       └── utils/
└── metrics.json           # Benchmark results

🎯 Key Features

• ✅ Real-time mobile camera streaming
• ✅ QR code mobile connection
• ✅ Performance metrics and benchmarking
• ✅ Docker containerization
• ✅ Responsive web interface

---

📋 Design Report

Architecture Decisions

1. WebSocket Communication

• Chosen for low-latency bi-directional communication
• Enables real-time frame streaming and results delivery
• Built-in backpressure handling through connection management

2. FastAPI Backend

• Async request handling for better concurrency
• Native WebSocket support
• Easy integration with Python ML libraries

3. React Frontend

• Component-based architecture for maintainable UI
• Real-time canvas rendering for smooth overlay display
• Service worker support for WASM mode

Low-Resource Implementation

Resolution Scaling:

• Default: 320×240 input resolution
• Automatic upscaling for display
• Configurable via environment variables

Processing Rate Control:

• Target: 10-15 FPS processing
• Skip frames during high load
• Dynamic quality adjustment

Backpressure Policy

1. Frame Queue Management

• Fixed-size circular buffer (5 frames default)
• Automatic oldest-frame dropping
• No blocking on queue full

2. WebSocket Flow Control

• Connection state monitoring
• Pause processing on slow consumers
• Graceful degradation under load

3. Adaptive Quality

• Lower resolution under high load
• Reduced detection confidence thresholds
• Frame skipping based on processing lag

4. Resource Monitoring

• CPU/memory usage tracking
• Automatic mode switching if needed
• Client capability detection

Performance Targets

• Latency: <200ms end-to-end (median), <500ms (P95)
• Throughput: 10-15 FPS processed, 20-30 FPS display
• Bandwidth: <1Mbps uplink, <500kbps downlink
• Resource: <2GB RAM, <50% CPU single core

Future Improvements

1. Hardware Acceleration: GPU support for inference
2. Edge Optimization: TensorFlow Lite/ONNX for mobile
3. Scalability: Redis for multi-instance deployment
4. Analytics: Real-time performance dashboards

---

Built with ❤️ for real-time computer vision applications