Pre-registering Side Projects: A Method to Avoid Confirmation Bias

In a recent post, a developer discusses the importance of pre-registering side projects to maintain research integrity. By documenting success criteria, failure definitions, and minimum sample sizes in a timestamped Git commit before starting experiments, the author aims to eliminate confirmation bias. This practice prevents the subconscious shifting of goalposts when data results are disappointing. The author applied this method to a research project involving AI coding agents and Google's Gemma 4 models. Despite having a favorite hypothesis that ultimately failed, the pre-registration process ensured the negative result was reported honestly rather than abandoned. The author argues that while pre-registration is often associated with formal academic research, it is a valuable tool for side projects to ensure that negative results are treated as valid, publishable findings, ultimately preventing the common trap of only reporting data that aligns with initial expectations.