Autonomous Traffic-Aware Robot Car

The SRH Heidelberg BIP challenge gave us a few days and a SunFounder PiCar-X with one goal: make it drive a track completely on its own. It needed to follow a lane, read a traffic sign, turn correctly, complete the loop, and stop at a STOP sign - with bonus points for voice commands and obstacle handling. I owned the voice-control system and the turn-execution logic.

We designed a split brain. The Pi handles everything real-time - lane following, steering, speed, voice, and its ultrasonic sensor - while a YOLO traffic-sign model runs on a GPU laptop. The car streams frames over Wi-Fi and receives detections as JSON. Driving never depends on the network: if Wi-Fi drops, the car keeps following the lane because detection is advice, not a requirement.

My offline Vosk voice system listens on a separate thread from the drive loop and keeps a fast local parser available whenever the language model is slow or unavailable. Safety shaped the implementation more than the happy path.

• “Stop” fires instantly from the partial local transcript and never waits for a network request.
• A two-second timeout and generation counter discard stale language-model replies.
• Negated or ambiguous commands are ignored, while an RMS gate and fuzzy matching improve reliability.
• Detected signs arm a maneuver; the car waits for all three floor sensors to confirm the intersection before turning.

By the demo, autonomous lane following, sign response, voice control, and obstacle stopping all worked. Steering around an obstacle was the honest gap we scoped out in favor of a reliable stop.

This was my first time writing code that drives physical hardware in real time. On a screen, a bug is a wrong answer; on a robot, it can mean driving off a table. Designing safe behavior first - instant local stop, stale-command rejection, and explicit intersection confirmation - changed how I think about what “done” means.