Confident AI is an open-source evaluation infrastructure that provides confidence for Language Models (LLMs). Users can assess their LLM applications by writing and executing test cases and leverage a rich set of open-source metrics to measure their performance. By defining expected outputs and comparing them to actual outputs, users can determine if their LLM is meeting expectations and identify areas for improvement. Confident AI also offers advanced diff tracking capabilities to help users optimize LLM configurations. Furthermore, users can utilize comprehensive analytics to identify key focus areas for use cases, enabling confident deployment of LLMs. Confident AI also provides powerful features to help users confidently deploy LLMs into production, including A/B testing, evaluation, output classification, reporting dashboards, dataset generation, and detailed monitoring.