MoA (Mixture of Agents) is a novel approach that leverages the collective strengths of multiple large language models (LLMs) to improve performance, achieving state-of-the-art results. Employing a hierarchical architecture with multiple LLM agents per layer, MoA surpasses the 57.5% score achieved by GPT-4 Omni on AlpacaEval 2.0, reaching a score of 65.1% while utilizing only open-source models.