Support Local LLM | Voters

Support Local LLM

complete

Support Local LLM like Ollama,LM Studio or anything locally hosted.

An option to customize OpenAI API endpoint URI would be a good start.

Daniel Nguyen

marked this post as

complete

I've added this in version 1.16.0.

Daniel Nguyen

marked this post as

in progress

Daniel Nguyen

Z you can already customize the endpoint in Settings > Models > OpenAI > Advanced Settings

Daniel Nguyen: Tried setting it to localhost (which runs LiteLLM proxying to Ollama) and I can see

GET /v1/models

coming through locally but when chatting with the docs it seems to still hits OpenAI instead.

Ah ignore. It is now working with the latest version 1.4 (I was on 1.1)

Daniel Nguyen

Good idea. I will add this in the next versions