PacktPublishing / LLM-Engineers-Handbook

👷 LLM Engineer's Handbook Official repository of the LLM Engineer's Handbook by Paul Iusztin and Maxime Labonne Find the book on Amazon or Packt 🌟 Features The goal of this book is to create your own end-to-end LLM-based system using best practices: • 📝 Data collection & generation • 🔄 LLM training pipeline • 📊 Simple RAG system • 🚀 Production-ready AWS deployment • 🔍 Comprehensive monitoring • 🧪 Testing and evaluation framework You can download and use the final trained model on Hugging Face. > [!IMPORTANT] > The code in this GitHub repository is actively maintained and may contain updates not reflected in the book. **Always refer to this repository for the latest version of the code.** 🔗 Dependencies Local dependencies To install and run the project locally, you need the following dependencies. | Tool | Version | Purpose | Installation Link | |------|---------|---------|------------------| | pyenv | ≥2.3.36 | Multiple Python versions (optional) | Install Guide | | Python | 3.11 | Runtime environment | Download | | Poetry | >= 1.8.3 and [!NOTE] > If you are experiencing issues while installing and running the repository, consider checking the Issues GitHub section for other people who solved similar problems or directly asking us for help. • Clone the Repository Start by cloning the repository and navigating to the project directory: Next, we have to prepare your Python environment and its adjacent dependencies. • Set Up Python Environment The project requires Python 3.11. You can either use your global Python installation or set up a project-specific version using pyenv. Option A: Using Global Python (if version 3.11 is installed) Verify your Python version: Option B: Using pyenv (recommended) • Verify pyenv installation: • Install Python 3.11.8: • Verify the installation: • Confirm Python version in the project directory: > [!NOTE] > The project includes a file that automatically sets the correct Python version when you're in the project directory. • Install Dependencies The project uses Poetry for dependency management. • Verify Poetry installation: • Set up the project environment and install dependencies: This will: • Configure Poetry to use Python 3.11 • Install project dependencies (excluding AWS-specific packages) • Set up pre-commit hooks for code verification • Activate the Environment As our task manager, we run all the scripts using Poe the Poet. • Start a Poetry shell: • Run project commands using Poe the Poet: 🔧 Troubleshooting Poe the Poet Installation Alternative Command Execution If you're experiencing issues with , you can still run the project commands directly through Poetry. Here's how: • Look up the command definition in • Use with the underlying command Example: Instead of: Use the direct command from pyproject.toml: Note: All project commands are defined in the [tool.poe.tasks] section of pyproject.toml Now, let's configure our local project with all the necessary credentials and tokens to run the code locally. • Local Development Setup After you have installed all the dependencies, you must create and fill a  file with your credentials to appropriately interact with other services and run the project. Setting your sensitive credentials in a file is a good security practice, as this file won't be committed to GitHub or shared with anyone else. • First, copy our example by running the following: • Now, let's understand how to fill in all the essential variables within the file to get you started. The following are the mandatory settings we must complete when working locally: OpenAI To authenticate to OpenAI's API, you must fill out the env var with an authentication token. → Check out this tutorial to learn how to provide one from OpenAI. Hugging Face To authenticate to Hugging Face, you must fill out the env var with an authentication token. → Check out this tutorial to learn how to provide one from Hugging Face. Comet ML & Opik To authenticate to Comet ML (required only during training) and Opik, you must fill out the env var with your authentication token. → Check out this tutorial to learn how to get started with Opik. You can also access Opik's dashboard using 🔗this link. • Deployment Setup When deploying the project to the cloud, we must set additional settings for Mongo, Qdrant, and AWS. If you are just working locally, the default values of these env vars will work out of the box. Detailed deployment instructions are available in Chapter 11 of the LLM Engineer's Handbook. MongoDB We must change the env var with the URL pointing to your cloud MongoDB cluster. → Check out this tutorial to learn how to create and host a MongoDB cluster for free. Qdrant Change to , with the URL point to your cloud Qdrant cluster, and with its API key. → Check out this tutorial to learn how to create a Qdrant cluster for free AWS For your AWS set-up to work correctly, you need the AWS CLI installed on your local machine and properly configured with an admin user (or a user with enough permissions to create new SageMaker, ECR, and S3 resources; using an admin user will make everything more straightforward). Chapter 2 provides step-by-step instructions on how to install the AWS CLI, create an admin user on AWS, and get an access key to set up the and environment variables. If you already have an AWS admin user in place, you have to configure the following env vars in your file: AWS credentials are typically stored in . You can view this file directly using or similar commands: > [!IMPORTANT] > Additional configuration options are available in settings.py. Any variable in the class can be configured through the file. 🏗️ Infrastructure Local infrastructure (for testing and development) When running the project locally, we host a MongoDB and Qdrant database using Docker. Also, a testing ZenML server is made available through their Python package. > [!WARNING] > You need Docker installed (>= v27.1.1) For ease of use,…