Skip to content

LiteLLM model connector for ADK agents

ADK 智能体的 LiteLLM 模型连接器

Supported in ADKPython v0.1.0

LiteLLM is a Python library that acts as a translation layer for models and model hosting services, providing a standardized, OpenAI-compatible interface to over 100+ LLMs. ADK provides integration through the LiteLLM library, allowing you to access a vast range of LLMs from providers like OpenAI, Anthropic (non-Vertex AI), Cohere, and many others. You can run open-source models locally or self-host them and integrate them using LiteLLM for operational control, cost savings, privacy, or offline use cases.

LiteLLM 是一个 Python 库,作为模型和模型托管服务的转换层,为超过 100+ 个 LLM 提供标准化的 OpenAI 兼容接口。ADK 通过 LiteLLM 库提供集成,允许您访问来自 OpenAI、Anthropic (非 Vertex AI)、Cohere 和许多其他提供商的大量 LLM。您可以在本地运行开源模型或自托管它们,并使用 LiteLLM 集成它们,以实现操作控制、成本节约、隐私保护或离线用例。

You can use the LiteLLM library to access remote or locally hosted AI models:

您可以使用 LiteLLM 库访问远程或本地托管的 AI 模型:

  • Remote model host: Use the LiteLlm wrapper class and set it as the model parameter of LlmAgent.
  • 远程模型托管: 使用 LiteLlm 包装类并将其设置为 LlmAgent 的 model 参数。
  • Local model host: Use the LiteLlm wrapper class configured to point to your local model server. For examples of local model hosting solutions, see the Ollama or vLLM documentation.
  • 本地模型托管: 使用 LiteLlm 包装类配置为指向您的本地模型服务器。有关本地模型托管解决方案的示例,请参阅 Ollama 或 vLLM 文档。
Windows Encoding with LiteLLM

When using ADK agents with LiteLLM on Windows, you might encounter a UnicodeDecodeError. This error occurs because LiteLLM may attempt to read cached files using the default Windows encoding (cp1252) instead of UTF-8. Prevent this error by setting the PYTHONUTF8 environment variable to 1. This forces Python to use UTF-8 for all file I/O.

Example (PowerShell): 

# Set for the current session
$env:PYTHONUTF8 = "1"

# Set persistently for the user
[System.Environment]::SetEnvironmentVariable('PYTHONUTF8', '1', [System.EnvironmentVariableTarget]::User)

Windows 与 LiteLLM 的编码问题

在 Windows 上使用 LiteLLM 的 ADK 智能体时,您可能会遇到 UnicodeDecodeError。此错误发生是因为 LiteLLM 可能尝试使用默认 Windows 编码 (cp1252) 而不是 UTF-8 读取缓存文件。通过将 PYTHONUTF8 环境变量设置为 1 来防止此错误。这将强制 Python 对所有文件 I/O 使用 UTF-8。

示例 (PowerShell): 

# 为当前会话设置
$env:PYTHONUTF8 = "1"

# 为用户持久设置
[System.Environment]::SetEnvironmentVariable('PYTHONUTF8', '1', [System.EnvironmentVariableTarget]::User)

Setup

设置

  1. Install LiteLLM: 
    pip install litellm

  2. 安装 LiteLLM: 

    pip install litellm

  3. Set Provider API Keys: Configure API keys as environment variables for the specific providers you intend to use.

  4. 设置提供商 API 密钥: 为您打算使用的特定提供商将 API 密钥配置为环境变量。

    • Example for OpenAI:

    • OpenAI 示例:

      export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

    • Example for Anthropic (non-Vertex AI):

    • Anthropic (非 Vertex AI) 示例:

      export ANTHROPIC_API_KEY="YOUR_ANTHROPIC_API_KEY"

    • Consult the LiteLLM Providers Documentation for correct environment variable names for other providers.

    • 请参阅 LiteLLM 提供商文档,了解其他提供商的正确环境变量名称。

Example implementation

示例实现

from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm

# --- Example Agent using OpenAI's GPT-4o ---
# (Requires OPENAI_API_KEY)
agent_openai = LlmAgent(
    model=LiteLlm(model="openai/gpt-4o"), # LiteLLM model string format
    name="openai_agent",
    instruction="You are a helpful assistant powered by GPT-4o.",
    # ... other agent parameters
)

# --- Example Agent using Anthropic's Claude Haiku (non-Vertex) ---
# (Requires ANTHROPIC_API_KEY)
agent_claude_direct = LlmAgent(
    model=LiteLlm(model="anthropic/claude-3-haiku-20240307"),
    name="claude_direct_agent",
    instruction="You are an assistant powered by Claude Haiku.",
    # ... other agent parameters
)