Lemur is an open-source language model designed to optimize natural language and coding capabilities for language agents. It balances both natural language and coding skills, enabling agents to follow instructions, reason about tasks, and take real-world actions. Lemur combines the strengths of natural language and coding through a two-stage training process, resulting in state-of-the-art performance on various language and coding benchmarks. It surpasses other available open-source models and narrows the gap between open-source and commercial models in agent capabilities.