Computer Vision
DriveAware
Vision-based driver distraction detection from three in-car cameras, with four model pipelines compared on drivers the model never saw.
- Role
- Researcher & Developer
DriveAware is my honours year project: a vision-based, multi-cue system that detects driver distraction from in-car video. Instead of relying on a single signal, it watches three: what the driver is doing (body camera), where they are looking (face camera) and their hands on the wheel (hands camera). A model is trained for each cue on the Driver Monitoring Dataset (DMD), and the three decisions are combined by decision-level fusion.
The system runs end to end: an Angular dashboard, a FastAPI service backed by PostgreSQL, and analysis of uploaded or live video with any of the four trained pipelines.
Four pipelines, one shared setup
Every pipeline sees the same input (16 RGB frames from a 2-second window, 224 x 224) and the same split by participant (10 training drivers, 2 validation, 2 test), so the differences come from the architecture.
| Pipeline | Architecture | Parameters |
|---|---|---|
| 1. Manual NumPy CNN | Compact residual CNN; every layer, the backward pass and the Adam optimiser written by hand in NumPy | about 177k |
| 2. PyTorch CNN | The same network in PyTorch, used to validate the manual implementation | about 177k |
| 3. Pretrained CNN | ImageNet ResNet-18 fine-tuned end to end, averaged over the 16 frames | 11.3M |
| 4. Pretrained CNN + Transformer | The Pipeline 3 frame encoder plus a 2-layer temporal transformer (8 attention heads) | 12.4M |
Results on drivers the model never saw
Test accuracy on the two held-out participants:
| Pipeline | Action | Gaze | Wheel |
|---|---|---|---|
| 1. Manual NumPy CNN | 39.62% | 42.31% | 41.22% |
| 2. PyTorch CNN | 17.97% | 48.72% | 45.55% |
| 3. Pretrained CNN | 81.22% | 77.56% | 67.43% |
| 4. Pretrained CNN + Transformer | 80.19% | 78.85% | 79.39% |
| Majority-class baseline | 42.7% | 14.1% | 50.4% |
- The hand-written network matches the framework. Pipelines 1 and 2 reach the same validation accuracy on gaze (35.81%), which supports the correctness of the manual backpropagation.
- Training from scratch does not generalise. With only 10 training drivers, the small networks learn those drivers rather than the task.
- Transfer learning is the largest single improvement, raising action accuracy from 17.97% to 81.22%.
- Temporal modelling helps where motion matters. The transformer lifts wheel accuracy from 67.43% to 79.39% (macro-F1 0.639 to 0.763).
- Pipeline 4 is the most consistent model, at 78.85% to 80.19% on all three cues, and is the one recommended for the system.
Limitations
A gap to unseen drivers remains (85 to 98% training accuracy against 67 to 81% on test). The validation and test sets hold only two drivers each, so scores move noticeably between epochs. Rare actions, such as reaching to the backseat, are recognised less reliably than the headline accuracy suggests (action macro-F1 about 0.68).
