My local 7B thinks "kill a Python process" is a violent crime — and my regex beat it

A developer testing AI safety protocols discovered that a 7-billion-parameter local model, Qwen 2.5 Coder, frequently misclassified benign technical queries as violent content. By using contrast pairs—homonyms like "kill a Python process" versus actual threats—the author demonstrated that the LLM struggled with context, flagging harmless phrases as "UNSAFE." Surprisingly, a simple, hand-written regex script with an exception list outperformed the LLM in these specific classification tasks. The author argues that current AI safety benchmarks often measure the entire "scaffold"—the parsers and filters surrounding the model—rather than the intelligence of the model itself. This experiment highlights the limitations of relying solely on large models for safety filtering and suggests that traditional, rule-based systems can still provide superior accuracy for specific, well-defined tasks, especially when running on limited local hardware.