I trained my own vision-language model with less than 1 billion parameters

In a recent Habr article, the author shares their experience building a compact vision-language model (VLM) from scratch on a limited budget. Using a single rented GPU, the developer integrated a small language model with a visual encoder. The process covered all stages, from architectural design to training and quality assessment. The resulting model, with 628 million parameters, is capable of describing images, with the total cost of the experiment amounting to approximately $200. The author provides a detailed breakdown of technical decisions, errors encountered during initial runs, and the final system performance. This project demonstrates that modern neural network training methods allow individual developers with limited resources to create effective multimodal solutions. The article is useful for those interested in the practical application of LLMs and computer vision, as well as those looking to understand how to optimize neural network training for accessible hardware.