llama.cpp
摘要
llama.cpp 官方站点介绍：这是一个完全本地运行的开源 AI 推理引擎，无需 API 密钥、无遥测、无使用限制，模型与对话数据完全归用户所有。页面提供一键安装命令（curl 脚本），也支持 Brew、Winget 或源码编译。可与本地编码代理 Pi 配合：运行 llama serve 并安装 pi-llama 插件即可自动发现本地模型，文件与请求均不离开本机。性能针对各类硬件优化，同一二进制可跑在笔记本到集群上，并列出 RTX 5090、H100、MI300、A100、T4、B200 等支持型号。页面还推荐首批可运行的模型：Qwen 3.6（多模态推理，含稠密与 MoE 变体）、Gemma 4（基于 Gemini 3 技术，支持多模态与 140+ 语言）、GPT - OSS（OpenAI 首批开放权重模型，支持函数调用与工具使用）及 Gemma 3。
荐读理由
llama.cpp 现在提供一键安装脚本和本地编码助手插件，你可以在自己的机器上跑前沿模型，数据不出本机，省去配 API 和担心隐私的功夫
原文
llama.cpp
AI that lives on your computer. Open-source, private & always local.
Run frontier AI entirely on your machine. No API keys, no telemetry, no limits. Own your models and conversation data.
curl -LsSf https://llama.app/install.sh | sh
Prefer Brew or Winget? Package managers · Rather build from source? Follow instructions
Pair it with a local coding agent.
Run llama serve, install the pi-llama plugin and launch Pi. It will automatically discover your local model. No config, no API keys. Files stay on your machine, requests never leave it.
llama serve
pi install git:github.com/huggingface/pi-llama
pi
Optimized for any hardware.
From your laptop to a cluster, llama.cpp runs on whatever you have. Same binary, same models, same hand-tuned kernels for every GPU and CPU.
RTX 5090
H100
MI300
RTX 4090
A100
T4
B200
RTX 3090
Run your first model
Qwen 3.6
Alibaba's next-gen natively multimodal reasoning models. Dense and MoE variants that rival models many times their size on coding and vision tasks.
Gemma 4
Google's most capable open models, built from Gemini 3 technology. Supports multimodal reasoning, agentic workflows, and 140+ languages.
GPT-OSS
OpenAI's first open-weight models since GPT-2. Built for reasoning, agentic tasks, and developer use with function calling and tool use capabilities.
Gemma 3
Google's multimodal models built from Gemini technology. Supports 140+ languages, vision, and text tasks with up to 128K context for edge to cloud deployment.
这条对你有帮助吗？