Best Open Source llm inference Libraries

A curated list of the most popular GitHub repositories tagged with llm inference. Select any project to visualize its architecture and dive into the codebase using RepoMind's AI engine.

#1nomic-ai/gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

77,149C++

Analyze Code

#2ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

41,416Python

Analyze Code

#3gitleaks/gitleaks

Find secrets with Gitleaks 🔑

25,020Go

Analyze Code

#4liguodongiot/llm-action

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

23,177HTML

Analyze Code

#5Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

13,174Python

Analyze Code

#6openvinotoolkit/openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

9,732C++

Analyze Code

#7Tiiny-AI/PowerInfer

High-speed Large Language Model Serving for Local Deployment

8,714C++

Analyze Code