mindcraft-bots / mindcraft

🧠mindcraft⛏️ Crafting minds for Minecraft with LLMs and Mineflayer! FAQ | Discord Support | Video Tutorial | Blog Post | Contributor TODO | Paper Website | MineCollab > [!Caution] Do not connect this bot to public servers with coding enabled. This project allows an LLM to write/execute code on your computer. The code is sandboxed, but still vulnerable to injection attacks. Code writing is disabled by default, you can enable it by setting to in . Ye be warned. Getting Started Requirements • Minecraft Java Edition (up to v1.21.6, recommend v1.21.6) • Node.js Installed (Node v18 or v20 LTS recommended. Node v24+ may cause issues with native dependencies) • At least one API key from a supported API provider. See supported APIs. OpenAI is the default. > [!Important] > If installing node on windows, ensure you check > > If you encounter errors on macOS, see the FAQ for troubleshooting native module build issues Install and Run • Make sure you have the requirements above. • Download the latest release and unzip it, or clone the repository. • Rename to and fill in your API keys (you only need one). The desired model is set in or other profiles. For other models refer to the table below. • In terminal/command prompt, run from the installed directory • Start a minecraft world and open it to LAN on localhost port • Run from the installed directory If you encounter issues, check the FAQ or find support on discord. We are currently not very responsive to github issues. To run tasks please refer to Minecollab Instructions Configuration Model Customization You can configure project details in . See file. You can configure the agent's name, model, and prompts in their profile like . The model can be specified with the field, with values like . You will need the correct API key for the API provider you choose. See all supported APIs below. ⭐ VIEW SUPPORTED APIs ⭐ | API Name | Config Variable| Docs | |------|------|------| | | | docs | | | | docs | | | | docs | | | | docs | | | | docs | | (local) | n/a | docs | | | | Intl./cn | | | | docs | | | | docs | | (not grok) | | docs | | | | docs | | | | docs | | | | docs | | | | docs | | | | docs | | | n/a | n/a | | | | docs | | | | docs | For more comprehensive model configuration and syntax, see Model Specifications. For local models we support ollama and we provide our own finetuned models for you to use. To install our models, install ollama and run the following terminal command: Online Servers To connect to online servers your bot will need an official Microsoft/Minecraft account. You can use your own personal one, but will need another account if you want to connect too and play with it. To connect, change these lines in : > [!Important] > The bot's name in the profile.json must exactly match the Minecraft profile name! Otherwise the bot will spam talk to itself. To use different accounts, Mindcraft will connect with the account that the Minecraft launcher is currently using. You can switch accounts in the launcher, then run , then switch to your main account after the bot has connected. Tasks Tasks automatically start the bot with a prompt and a goal item to aquire or blueprint to construct. To run a simple task that involves collecting 4 oak_logs run Here is an example task json format: The is what the bot will have at the start of the episode, refers to the target item and refers to the number of target items the agent needs to collect to successfully complete the task. If you want more optimization and automatic launching of the minecraft world, you will need to follow the instructions in Minecollab Instructions Docker Container If you intend to , it is a good idea to run the app in a docker container to reduce risks of running unknown code. This is strongly recommended before connecting to remote servers, although still does not guarantee complete safety. or simply When running in docker, if you want the bot to join your local minecraft server, you have to use a special host address to call your localhost from inside your docker container. Put this into your settings.js: To connect to an unsupported minecraft version, you can try to use viaproxy Bot Profiles Bot profiles are json files (such as ) that define: • Bot backend LLMs to use for talking, coding, and embedding. • Prompts used to influence the bot's behavior. • Examples help the bot perform tasks. Model Specifications LLM models can be specified simply as , or more specifically with , like . See all supported APIs here. The field can be a string or an object. A model object must specify an , and optionally a , , and additional . You can also use different models/providers for chatting, coding, vision, embedding, and voice synthesis. See the example below. is used for chat, is used for newAction coding, is used for image interpretation, is used to embed text for example selection, and is used for voice synthesis. will be used by default for all other models if not specified. Not all APIs support embeddings, vision, or voice synthesis. All apis have default models and urls, so those fields are optional. The field is optional and can be used to specify additional parameters for the model. It accepts any key-value pairs supported by the api. Is not supported for embedding models. Embedding Models Embedding models are used to embed and efficiently select relevant examples for conversation and coding. Supported Embedding APIs: , , , , If you try to use an unsupported model, then it will default to a simple word-overlap method. Expect reduced performance. We recommend using supported embedding APIs. Voice Synthesis Models Voice synthesis models are used to narrate bot responses and specified with . This field is parsed differently than other models and only supports strings formatted as , like . We only support and for voice synthesis. Specifying Profiles via Command Line By default, the program will use the profiles specified in . You can specify…