Agenta-AI / agenta

The Open-source LLMOps Platform Build reliable LLM applications faster with integrated prompt management, evaluation, and observability. --- Documentation • Website • Agenta Cloud --- --- What is Agenta? Agenta is a platform for building production-grade LLM applications. It helps **engineering** and **product teams** create reliable LLM apps faster through integrated prompt management, evaluation, and observability. Core Features 🧪 Prompt Management & Prompt Engineering Collaborate with Subject Matter Experts (SMEs) on prompt engineering and make sure nothing breaks in production. • **Interactive LLM Playground**: Compare prompts side by side against your test cases • **Multi-Model Support**: Experiment with 50+ LLM models or bring-your-own models • **Version Control**: Version prompts and configurations with branching and environments • **Complex Configurations**: Enable SMEs to collaborate on complex configuration schemas beyond simple prompts Explore prompt management → 📊 LLM Evaluation Evaluate your LLM applications systematically with both human and automated feedback. • **Flexible Testsets**: Create testcases from production data, playground experiments, or upload CSVs • **Pre-built and Custom Evaluators**: Use LLM-as-judge, one of our 20+ pre-built evaluators, or you custom evaluators • **UI and API Access**: Run evaluations via UI (for SMEs) or programmatically (for engineers) • **Human Feedback Integration**: Collect and incorporate expert annotations Explore evaluation frameworks → 📡 LLM Observability Get visibility into your LLM applications in production. • **Cost & Performance Tracking**: Monitor spending, latency, and usage patterns • **LLM Tracing**: Debug complex workflows with detailed traces • **Open Standards**: OpenTelemetry native tracing compatible with OpenLLMetry, and OpenInference • **Integrations**: Comes with pre-built integrations for most models and frameworks Learn about observability → 📸 Screenshots 🚀 Getting Started Agenta Cloud (Recommended): The easiest way to get started is through Agenta Cloud. Free tier available with no credit card required. Self-hosting Agenta • Clone Agenta: • Copy configuration: Before starting the services, create the environment file from the example: • Start Agenta services: • Access Agenta at . For deploying on a remote host, or using different ports refers to our self-hosting and remote deployment documentation. 💬 Community Find help, explore resources, or get involved: 🧰 Support • **📚 Documentation** – Full guides and API reference • **📋 Changelog** – Track recent updates • **💬 Slack Community** – Ask questions and get support 🤝 Contribute We welcome contributions of all kinds, from filing issues and sharing ideas to improving the codebase. Check out our CONTRIBUTING.md and: • **🐛 Report bugs** – Help us by reporting problems you encounter • **💡 Share ideas and feedback** – Suggest features or vote on ideas • **🔧 Contribute to the codebase** – Read the guide and open a pull request ⭐ Star Agenta **Consider giving us a star!** It helps us grow our community and gets Agenta in front of more developers. Contributors ✨ Thanks goes to these wonderful people (emoji key): Sameh Methnani 💻 📖 Suad Suljovic 💻 🎨 🧑‍🏫 👀 burtenshaw 💻 Abram 💻 📖 Israel Abebe 🐛 🎨 💻 Master X 💻 corinthian 💻 🎨 Pavle Janjusevic 🚇 Kaosi Ezealigo 🐛 💻 Alberto Nunes 🐛 Maaz Bin Khawar 💻 👀 🧑‍🏫 Nehemiah Onyekachukwu Emmanuel 💻 💡 📖 Philip Okiokio 📖 Abhinav Pandey 💻 Ramchandra Warang 💻 🐛 Biswarghya Biswas 💻 Uddeepta Raaj Kashyap 💻 Nayeem Abdullah 💻 Kang Suhyun 💻 Yoon 💻 Kirthi Bagrecha Jain 💻 Navdeep 💻 Rhythm Sharma 💻 Osinachi Chukwuja _...truncated for preview_