volcengine / OpenViking

OpenViking: The Context Database for AI Agents English / 中文 Website · GitHub · Issues · Docs [![][release-shield]][release-link] [![][github-stars-shield]][github-stars-link] [![][github-issues-shield]][github-issues-shield-link] [![][github-contributors-shield]][github-contributors-link] [![][license-shield]][license-shield-link] [![][last-commit-shield]][last-commit-shield-link] 👋 Join our Community 📱 Lark Group · WeChat · Discord · X --- Overview Challenges in Agent Development In the AI era, data is abundant, but high-quality context is hard to come by. When building AI Agents, developers often face these challenges: • **Fragmented Context**: Memories are in code, resources are in vector databases, and skills are scattered, making them difficult to manage uniformly. • **Surging Context Demand**: An Agent's long-running tasks produce context at every execution. Simple truncation or compression leads to information loss. • **Poor Retrieval Effectiveness**: Traditional RAG uses flat storage, lacking a global view and making it difficult to understand the full context of information. • **Unobservable Context**: The implicit retrieval chain of traditional RAG is like a black box, making it hard to debug when errors occur. • **Limited Memory Iteration**: Current memory is just a record of user interactions, lacking Agent-related task memory. The OpenViking Solution **OpenViking** is an open-source **Context Database** designed specifically for AI Agents. We aim to define a minimalist context interaction paradigm for Agents, allowing developers to completely say goodbye to the hassle of context management. OpenViking abandons the fragmented vector storage model of traditional RAG and innovatively adopts a **"file system paradigm"** to unify the structured organization of memories, resources, and skills needed by Agents. With OpenViking, developers can build an Agent's brain just like managing local files: • **Filesystem Management Paradigm** → **Solves Fragmentation**: Unified context management of memories, resources, and skills based on a filesystem paradigm. • **Tiered Context Loading** → **Reduces Token Consumption**: L0/L1/L2 three-tier structure, loaded on demand, significantly saving costs. • **Directory Recursive Retrieval** → **Improves Retrieval Effect**: Supports native filesystem retrieval methods, combining directory positioning with semantic search to achieve recursive and precise context acquisition. • **Visualized Retrieval Trajectory** → **Observable Context**: Supports visualization of directory retrieval trajectories, allowing users to clearly observe the root cause of issues and guide retrieval logic optimization. • **Automatic Session Management** → **Context Self-Iteration**: Automatically compresses content, resource references, tool calls, etc., in conversations, extracting long-term memory, making the Agent smarter with use. --- Quick Start Prerequisites Before starting with OpenViking, please ensure your environment meets the following requirements: • **Python Version**: 3.10 or higher • **Go Version**: 1.22 or higher (Required for building AGFS components) • **C++ Compiler**: GCC 9+ or Clang 11+ (Required for building core extensions) • **Operating System**: Linux, macOS, Windows • **Network Connection**: A stable network connection is required (for downloading dependencies and accessing model services) • Installation Python Package Rust CLI (Optional) Or build from source: • Model Preparation OpenViking requires the following model capabilities: • **VLM Model**: For image and content understanding • **Embedding Model**: For vectorization and semantic retrieval Supported VLM Providers OpenViking supports three VLM providers: | Provider | Description | Get API Key | |----------|-------------|-------------| | | Volcengine Doubao Models | Volcengine Console | | | OpenAI Official API | OpenAI Platform | | | Unified access to various third-party models (Anthropic, DeepSeek, Gemini, vLLM, Ollama, etc.) | See LiteLLM Providers | > 💡 **Tip**: > - supports unified access to various models. The field must follow the LiteLLM format specification > - The system auto-detects common models (e.g., , , , , , etc.). For other models, use the full prefix according to LiteLLM format Provider-Specific Notes Volcengine (Doubao) Volcengine supports both model names and endpoint IDs. Using model names is recommended for simplicity: You can also use endpoint IDs (found in Volcengine ARK Console: OpenAI Use OpenAI's official API: You can also use a custom OpenAI-compatible endpoint: LiteLLM (Anthropic, DeepSeek, Gemini, Qwen, vLLM, Ollama, etc.) LiteLLM provides unified access to various models. The field should follow LiteLLM's naming convention. Here we use Claude and Qwen as examples: **Anthropic:** **Qwen (DashScope):** > 💡 **Tip for Qwen**: > - For **China/Beijing** region, use : > - For **International** region, use : **Common model formats:** | Provider | Model Example | Notes | |----------|---------------|-------| | Anthropic | | Auto-detected, uses | | DeepSeek | | Auto-detected, uses | | Gemini | | Auto-detected, uses | | Qwen | | Set based on region (see above) | | OpenRouter | | Full prefix required | | vLLM | | Set to vLLM server | | Ollama | | Set to Ollama server | **Local Models (vLLM / Ollama):** For complete model support, see LiteLLM Providers Documentation. • Environment Configuration Server Configuration Template Create a configuration file , remove the comments before copy: > **Note**: For embedding models, currently (Doubao), , and providers are supported. For VLM models, we support three providers: , , and . The provider supports various models including Anthropic (Claude), DeepSeek, Gemini, Moonshot, Zhipu, DashScope, MiniMax, vLLM, Ollama, and more. Server Configuration Examples 👇 Expand to see the configuration example for your model service: Example 1: Using Volcengine (Doubao Models) Example 2: Using O…