A 25-verifier panel measured an effective size of 1.00

The open-source research project IDKMesh has released findings challenging the assumption that adding more reviewers to a verification gate increases the reliability of the process. In a controlled experiment using 25 distinct program-based verifiers, the researchers discovered that the panel's effective size was only 1.00, meaning the collective output was no more reliable than a single verifier. The study highlights that error correlation between verifiers is significantly higher than expected, even when they operate across different domains. The author notes that common heuristics for calculating effective reviewer counts often overestimate performance, leading to a false sense of security in automated or human-led review gates. The project, which is currently in a research preview phase, provides tools for developers to audit their own review panels and encourages further investigation into whether these correlation patterns persist in AI-driven review systems.