feat: 添加vLLM适配器、修复SSE流式输出和中间件问题
- 新增 vLLM 适配器 (internal/adapter/vllm.go),支持 OpenAI 兼容 API - 修复 responseWriter 未实现 http.Flusher 导致 SSE 流式输出 500 错误 - 调整中间件顺序:BodyLimit 移至 Auth 之前,提前拒绝超大请求 - BodyLimit 增强:检查 Content-Length header - 导出 Server.Authenticator() 方法供测试使用 - 配置更新:使用本地 deepseek-r1:1.5b 模型,新增 vllm-chat 逻辑模型 - 修复 handlers.go 中未使用的 target 参数 lint 警告
This commit is contained in:
+12
-2
@@ -36,7 +36,7 @@ context:
|
||||
models:
|
||||
general-chat:
|
||||
provider: ollama
|
||||
actual_model: qwen2.5:0.5b
|
||||
actual_model: deepseek-r1:1.5b
|
||||
endpoint: http://127.0.0.1:11434
|
||||
context_window: 32768
|
||||
max_output_tokens: 4096
|
||||
@@ -46,7 +46,7 @@ models:
|
||||
|
||||
fast-chat:
|
||||
provider: ollama
|
||||
actual_model: qwen2.5:0.5b
|
||||
actual_model: deepseek-r1:1.5b
|
||||
endpoint: http://127.0.0.1:11434
|
||||
context_window: 16384
|
||||
max_output_tokens: 2048
|
||||
@@ -54,6 +54,16 @@ models:
|
||||
residency: on_demand
|
||||
idle_unload_seconds: 600
|
||||
|
||||
vllm-chat:
|
||||
provider: vllm
|
||||
actual_model: deepseek-r1:1.5b
|
||||
endpoint: http://127.0.0.1:8000
|
||||
context_window: 32768
|
||||
max_output_tokens: 4096
|
||||
max_concurrency: 4
|
||||
residency: always
|
||||
cancel_supported: true
|
||||
|
||||
routing:
|
||||
sensitive_data_local_only: true
|
||||
allow_cloud_fallback_by_default: false
|
||||
|
||||
Reference in New Issue
Block a user