mirror of
https://github.com/instructkr/claude-code.git
synced 2026-06-05 20:06:43 +00:00
feat: add native Ollama provider support via OLLAMA_HOST env var
- OLLAMA_HOST takes priority over OPENAI_BASE_URL for local Ollama instances - No API key required; placeholder token used for Authorization header - Model names like 'qwen3:8b' bypass strict provider/model syntax validation - detect_provider_kind() checks OLLAMA_HOST first in routing cascade - ProviderClient dispatch uses from_ollama_env() when OLLAMA_HOST is set - Updated USAGE.md and docs with OLLAMA_HOST as preferred env var - Added OLLAMA_CONFIG constant and from_ollama_env() to openai_compat - Added test_ollama_host_bypasses_model_validation unit test - Supersedes PR #3213 (which had a duplicate if-let bug in mod.rs)
This commit is contained in:
11
USAGE.md
11
USAGE.md
@@ -245,6 +245,7 @@ export ANTHROPIC_AUTH_TOKEN="anthropic-oauth-or-proxy-bearer-token"
|
|||||||
| `sk-ant-*` API key | `ANTHROPIC_API_KEY` | `x-api-key: sk-ant-...` | [console.anthropic.com](https://console.anthropic.com) |
|
| `sk-ant-*` API key | `ANTHROPIC_API_KEY` | `x-api-key: sk-ant-...` | [console.anthropic.com](https://console.anthropic.com) |
|
||||||
| OAuth access token (opaque) | `ANTHROPIC_AUTH_TOKEN` | `Authorization: Bearer ...` | an Anthropic-compatible proxy or OAuth flow that mints bearer tokens |
|
| OAuth access token (opaque) | `ANTHROPIC_AUTH_TOKEN` | `Authorization: Bearer ...` | an Anthropic-compatible proxy or OAuth flow that mints bearer tokens |
|
||||||
| OpenRouter key (`sk-or-v1-*`) | `OPENAI_API_KEY` + `OPENAI_BASE_URL=https://openrouter.ai/api/v1` | `Authorization: Bearer ...` | [openrouter.ai/keys](https://openrouter.ai/keys) |
|
| OpenRouter key (`sk-or-v1-*`) | `OPENAI_API_KEY` + `OPENAI_BASE_URL=https://openrouter.ai/api/v1` | `Authorization: Bearer ...` | [openrouter.ai/keys](https://openrouter.ai/keys) |
|
||||||
|
| Ollama local instance | `OLLAMA_HOST` | no auth header (Ollama requires none) | local Ollama server at `http://127.0.0.1:11434` |
|
||||||
|
|
||||||
**Why this matters:** if you paste an `sk-ant-*` key into `ANTHROPIC_AUTH_TOKEN`, Anthropic's API will return `401 Invalid bearer token` because `sk-ant-*` keys are rejected over the Bearer header. The fix is a one-line env var swap — move the key to `ANTHROPIC_API_KEY`. Recent `claw` builds detect this exact shape (401 + `sk-ant-*` in the Bearer slot) and append a hint to the error message pointing at the fix.
|
**Why this matters:** if you paste an `sk-ant-*` key into `ANTHROPIC_AUTH_TOKEN`, Anthropic's API will return `401 Invalid bearer token` because `sk-ant-*` keys are rejected over the Bearer header. The fix is a one-line env var swap — move the key to `ANTHROPIC_API_KEY`. Recent `claw` builds detect this exact shape (401 + `sk-ant-*` in the Bearer slot) and append a hint to the error message pointing at the fix.
|
||||||
|
|
||||||
@@ -305,18 +306,18 @@ cd rust
|
|||||||
### Ollama
|
### Ollama
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export OPENAI_BASE_URL="http://127.0.0.1:11434/v1"
|
export OLLAMA_HOST="http://127.0.0.1:11434"
|
||||||
unset OPENAI_API_KEY
|
|
||||||
|
|
||||||
cd rust
|
cd rust
|
||||||
./target/debug/claw --model "llama3.2" prompt "summarize this repository in one sentence"
|
./target/debug/claw --model "llama3.2" prompt "summarize this repository in one sentence"
|
||||||
```
|
```
|
||||||
|
|
||||||
For Ollama tags with punctuation (for example `qwen2.5-coder:7b`), `OPENAI_BASE_URL` selects the local OpenAI-compatible route even when `OPENAI_API_KEY` is unset:
|
`OLLAMA_HOST` is the preferred env var. Claw routes all models to the local Ollama endpoint automatically, and no API key is needed. The older `OPENAI_BASE_URL` + `OPENAI_API_KEY` workaround is also supported.
|
||||||
|
|
||||||
|
For Ollama tags with punctuation (for example `qwen2.5-coder:7b`), both approaches work:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export OPENAI_BASE_URL="http://127.0.0.1:11434/v1"
|
export OLLAMA_HOST="http://127.0.0.1:11434"
|
||||||
unset OPENAI_API_KEY
|
|
||||||
|
|
||||||
cd rust
|
cd rust
|
||||||
./target/debug/claw --model "qwen2.5-coder:7b" prompt "reply with ready"
|
./target/debug/claw --model "qwen2.5-coder:7b" prompt "reply with ready"
|
||||||
|
|||||||
@@ -57,11 +57,12 @@ ollama serve
|
|||||||
In another shell:
|
In another shell:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export OPENAI_BASE_URL="http://127.0.0.1:11434/v1"
|
export OLLAMA_HOST="http://127.0.0.1:11434"
|
||||||
unset OPENAI_API_KEY
|
|
||||||
claw --model "qwen3:latest" prompt "Reply exactly HELLO_WORLD_123"
|
claw --model "qwen3:latest" prompt "Reply exactly HELLO_WORLD_123"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`OLLAMA_HOST` is the preferred env var for Ollama. Claw routes all models to the local OpenAI-compatible endpoint automatically when this is set, and no API key is needed. The older `OPENAI_BASE_URL` + `OPENAI_API_KEY` workaround is also supported for existing setups.
|
||||||
|
|
||||||
If Ollama is running without auth, `unset OPENAI_API_KEY` is acceptable. Use a placeholder token rather than a real cloud API key if your local server requires an Authorization header.
|
If Ollama is running without auth, `unset OPENAI_API_KEY` is acceptable. Use a placeholder token rather than a real cloud API key if your local server requires an Authorization header.
|
||||||
|
|
||||||
## llama.cpp server
|
## llama.cpp server
|
||||||
|
|||||||
@@ -32,16 +32,25 @@ impl ProviderClient {
|
|||||||
OpenAiCompatConfig::xai(),
|
OpenAiCompatConfig::xai(),
|
||||||
)?)),
|
)?)),
|
||||||
ProviderKind::OpenAi => {
|
ProviderKind::OpenAi => {
|
||||||
// DashScope models (qwen-*) also return ProviderKind::OpenAi because they
|
// OLLAMA_HOST takes priority: local Ollama needs no API key
|
||||||
// speak the OpenAI wire format, but they need the DashScope config which
|
// and ignores DashScope/OpenAI env-based dispatch.
|
||||||
// reads DASHSCOPE_API_KEY and points at dashscope.aliyuncs.com.
|
if std::env::var_os("OLLAMA_HOST").is_some() {
|
||||||
let config = match providers::metadata_for_model(&resolved_model) {
|
Ok(Self::OpenAi(
|
||||||
Some(meta) if meta.auth_env == "DASHSCOPE_API_KEY" => {
|
openai_compat::OpenAiCompatClient::from_ollama_env()
|
||||||
OpenAiCompatConfig::dashscope()
|
.expect("from_ollama_env always returns Some"),
|
||||||
}
|
))
|
||||||
_ => OpenAiCompatConfig::openai(),
|
} else {
|
||||||
};
|
// DashScope models (qwen-*) also return ProviderKind::OpenAi because they
|
||||||
Ok(Self::OpenAi(OpenAiCompatClient::from_env(config)?))
|
// speak the OpenAI wire format, but they need the DashScope config which
|
||||||
|
// reads DASHSCOPE_API_KEY and points at dashscope.aliyuncs.com.
|
||||||
|
let config = match providers::metadata_for_model(&resolved_model) {
|
||||||
|
Some(meta) if meta.auth_env == "DASHSCOPE_API_KEY" => {
|
||||||
|
OpenAiCompatConfig::dashscope()
|
||||||
|
}
|
||||||
|
_ => OpenAiCompatConfig::openai(),
|
||||||
|
};
|
||||||
|
Ok(Self::OpenAi(OpenAiCompatClient::from_env(config)?))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -351,6 +351,11 @@ fn looks_like_local_openai_model(model: &str) -> bool {
|
|||||||
|
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn detect_provider_kind(model: &str) -> ProviderKind {
|
pub fn detect_provider_kind(model: &str) -> ProviderKind {
|
||||||
|
// OLLAMA_HOST takes priority: if set, route all models through the local
|
||||||
|
// OpenAI-compatible endpoint regardless of model name or other env vars.
|
||||||
|
if std::env::var_os("OLLAMA_HOST").is_some() {
|
||||||
|
return ProviderKind::OpenAi;
|
||||||
|
}
|
||||||
let resolved_model = resolve_model_alias(model);
|
let resolved_model = resolve_model_alias(model);
|
||||||
if let Some(metadata) = metadata_for_model(&resolved_model) {
|
if let Some(metadata) = metadata_for_model(&resolved_model) {
|
||||||
return metadata.provider;
|
return metadata.provider;
|
||||||
|
|||||||
@@ -49,6 +49,14 @@ const XAI_MAX_REQUEST_BODY_BYTES: usize = 52_428_800; // 50MB
|
|||||||
const OPENAI_MAX_REQUEST_BODY_BYTES: usize = 104_857_600; // 100MB
|
const OPENAI_MAX_REQUEST_BODY_BYTES: usize = 104_857_600; // 100MB
|
||||||
const DASHSCOPE_MAX_REQUEST_BODY_BYTES: usize = 6_291_456; // 6MB (observed limit in dogfood)
|
const DASHSCOPE_MAX_REQUEST_BODY_BYTES: usize = 6_291_456; // 6MB (observed limit in dogfood)
|
||||||
|
|
||||||
|
pub const OLLAMA_CONFIG: OpenAiCompatConfig = OpenAiCompatConfig {
|
||||||
|
provider_name: "Ollama",
|
||||||
|
api_key_env: "OLLAMA_HOST",
|
||||||
|
base_url_env: "OLLAMA_HOST",
|
||||||
|
default_base_url: "http://127.0.0.1:11434/v1",
|
||||||
|
max_request_body_bytes: 104_857_600,
|
||||||
|
};
|
||||||
|
|
||||||
impl OpenAiCompatConfig {
|
impl OpenAiCompatConfig {
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub const fn xai() -> Self {
|
pub const fn xai() -> Self {
|
||||||
@@ -149,6 +157,22 @@ impl OpenAiCompatClient {
|
|||||||
};
|
};
|
||||||
Ok(Self::new(api_key, config).with_base_url(base_url))
|
Ok(Self::new(api_key, config).with_base_url(base_url))
|
||||||
}
|
}
|
||||||
|
/// Create an Ollama client from `OLLAMA_HOST` env var.
|
||||||
|
/// Ollama requires no API key; a placeholder is used for the Authorization header.
|
||||||
|
pub fn from_ollama_env() -> Option<Self> {
|
||||||
|
let host =
|
||||||
|
std::env::var("OLLAMA_HOST").unwrap_or_else(|_| "http://127.0.0.1:11434".to_string());
|
||||||
|
let base_url = format!("{}/v1", host.trim_end_matches('/'));
|
||||||
|
Some(Self {
|
||||||
|
http: build_http_client_or_default(),
|
||||||
|
api_key: "ollama".to_string(),
|
||||||
|
config: OLLAMA_CONFIG,
|
||||||
|
base_url,
|
||||||
|
max_retries: DEFAULT_MAX_RETRIES,
|
||||||
|
initial_backoff: DEFAULT_INITIAL_BACKOFF,
|
||||||
|
max_backoff: DEFAULT_MAX_BACKOFF,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn with_base_url(mut self, base_url: impl Into<String>) -> Self {
|
pub fn with_base_url(mut self, base_url: impl Into<String>) -> Self {
|
||||||
|
|||||||
@@ -2903,6 +2903,14 @@ fn resolve_model_alias_with_config(model: &str) -> String {
|
|||||||
/// Rejects: empty, whitespace-only, strings with spaces, or invalid chars.
|
/// Rejects: empty, whitespace-only, strings with spaces, or invalid chars.
|
||||||
fn validate_model_syntax(model: &str) -> Result<(), String> {
|
fn validate_model_syntax(model: &str) -> Result<(), String> {
|
||||||
let trimmed = model.trim();
|
let trimmed = model.trim();
|
||||||
|
// Ollama models use names like "qwen3:8b" that don't match provider/model
|
||||||
|
// syntax. Skip strict validation when OLLAMA_HOST is configured.
|
||||||
|
if std::env::var_os("OLLAMA_HOST").is_some() {
|
||||||
|
if trimmed.is_empty() {
|
||||||
|
return Err("invalid model syntax: model string cannot be empty.\nUsage: --model <model-name> e.g. --model qwen3:8b".to_string());
|
||||||
|
}
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
if trimmed.is_empty() {
|
if trimmed.is_empty() {
|
||||||
return Err("invalid model syntax: model string cannot be empty.\nUsage: --model <provider/model> e.g. --model anthropic/claude-opus-4-7".to_string());
|
return Err("invalid model syntax: model string cannot be empty.\nUsage: --model <provider/model> e.g. --model anthropic/claude-opus-4-7".to_string());
|
||||||
}
|
}
|
||||||
@@ -19689,4 +19697,16 @@ mod alias_resolution_tests {
|
|||||||
assert_eq!(resolve_model_alias_with_config(model), model);
|
assert_eq!(resolve_model_alias_with_config(model), model);
|
||||||
assert!(validate_model_syntax(model).is_ok());
|
assert!(validate_model_syntax(model).is_ok());
|
||||||
}
|
}
|
||||||
|
#[test]
|
||||||
|
fn test_ollama_host_bypasses_model_validation() {
|
||||||
|
// Safety: test sets and clears env var within the test.
|
||||||
|
std::env::set_var("OLLAMA_HOST", "http://127.0.0.1:11434");
|
||||||
|
// Ollama model names with colons pass
|
||||||
|
assert!(validate_model_syntax("qwen3:8b").is_ok());
|
||||||
|
assert!(validate_model_syntax("gemma4:e2b").is_ok());
|
||||||
|
assert!(validate_model_syntax("qwen3.6:27b-nvfp4").is_ok());
|
||||||
|
// Empty model still rejected
|
||||||
|
assert!(validate_model_syntax("").is_err());
|
||||||
|
std::env::remove_var("OLLAMA_HOST");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user