Why we chose RAG over fine-tuning for incident analysis

Selectel security engineer Anton Dyatlov shares his experience in automating initial incident analysis within a SOC. The core issue is the high volume of routine work involved in processing false positives, which distracts analysts from critical tasks. Instead of the labor-intensive process of fine-tuning an LLM, the team opted to implement a RAG (Retrieval-Augmented Generation) architecture using a local language model and an additional validation step. This approach allows for enriching alerts with context and automating documentation without sending sensitive data to external services. The article details the system's design, the advantages of choosing RAG over model fine-tuning, and the challenges encountered during operation. The solution significantly reduced the workload on L1 analysts while maintaining data control and the flexibility to tailor the system to the specific needs of the company's infrastructure.