Why I run smoke tests directly in production and how they once broke my analytics

In this article, the author explores the practice of running automated smoke tests directly in a production environment. The focus is on using synthetic traffic to simulate real user scenarios within messaging applications. The author shares a personal experience where test bot activity was mistakenly identified as real user behavior, leading to significant distortions in key product metrics such as DAU, WAU, and MAU. The article discusses strategies to prevent such issues, including methods for filtering technical traffic and implementing 'soft delete' mechanisms. It offers practical recommendations for engineers looking to adopt production testing without compromising the accuracy of business analytics or the quality of collected user behavior data.