Why I Ditched 'Just Let the LLM Handle It' for a State Machine

A developer building an AI-powered technical interviewer discovered that relying solely on LLMs to manage complex, multi-step workflows leads to unpredictable behavior, such as premature interview termination or repetitive questioning. The author argues that LLMs are effective at generating content but lack the structural memory required to act as reliable state trackers. By implementing a finite state machine (FSM) to handle the interview's flow, the developer successfully decoupled the logic of phase transitions from the LLM's generative tasks. This architectural shift ensured that the model only performs specific actions—like generating questions or hints—within defined states, rather than deciding the overall progression of the interview. The result was a more robust, testable, and reliable product that allowed the LLM to focus on high-quality output without the risk of hallucinating phase transitions, ultimately improving the user experience and reducing technical debt.