The translated data: OpenLM is a PyTorch codebase designed for training medium-sized language models, achieving remarkable results with the OpenLM-1B and OpenLM-7B models. OpenLM-1B excels in zero-shot text classification and multiple-choice tasks, while OpenLM-7B continuously enhances its zero-shot performance through ongoing training. Future work includes supporting multimodal models, mixtures of experts, and dataset combinations, as well as expanding OpenLM to support the training of larger models.