How to Deploy Your Own Private ChatGPT with Open WebUI and Ollama

Open WebUI gives your team a ChatGPT-like interface that runs entirely on your own server. Connect it to Ollama for local model inference, or to any OpenAI-compatible API for cloud models. Your team gets a familiar chat experience. Your data stays on your hardware. Here is how to deploy it with Docker Compose.

Step 1: Install Ollama

Ollama is the model runner that loads and runs AI models on your hardware. Install it first:

curl -fsSL https://ollama.com/install.sh | sh

Pull a model. For a balanced start with good performance on consumer hardware, use Mistral 7B:

ollama pull mistral:7b

Test that Ollama is working:

ollama run mistral:7b "Hello, are you working?"

If you see a response, Ollama is running and the model is loaded.

Step 2: Create the Docker Compose File for Open WebUI

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:8080"
    volumes:
      - open-webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
      - WEBUI_SECRET_KEY=your_long_random_secret_key_here
    extra_hosts:
      - "host.docker.internal:host-gateway"

volumes:
  open-webui:

The environment variable OLLAMA_BASE_URL tells Open WebUI where to find your Ollama instance. If Ollama is running on the same server, use host.docker.internal as shown. If Ollama is on a different server, use its URL instead.

Step 3: Start Open WebUI

docker compose up -d

Configure your reverse proxy. In Nginx Proxy Manager, add a proxy host for ai.yourdomain.com pointing to open-webui:8080. Enable WebSocket support. Request an SSL certificate and enable Force SSL.

Step 4: Configure Access and Users

Open https://ai.yourdomain.com. On first visit, create the admin account. This account manages all users and settings. From the admin panel:

• Create user accounts for your team, or configure OIDC to connect to Authelia for single sign-on
• Set default model to mistral:7b so every user starts with the right model selected
• Configure usage limits if you want to prevent any single user from overwhelming the server

Your team logs in, selects the model, and starts chatting. The interface is nearly identical to ChatGPT: a chat list on the left, the conversation in the centre, and a prompt box at the bottom.

Step 5: Add Documents for RAG (Retrieval Augmented Generation)

Open WebUI supports RAG: upload documents, and the AI can answer questions based on their content. In the Workspace, go to Documents. Upload PDFs, Word documents, or plain text files. The AI indexes them automatically. When you ask a question in chat, type # and select the document to ground the AI's response in that specific source.

This is the most powerful feature for business use. Upload your employee handbook, and the AI answers policy questions. Upload product documentation, and the AI answers customer questions. Upload client contracts, and the AI summarises key terms. The responses cite specific sections of the source document, so you can verify accuracy.

Step 6: Add More Models

Ollama supports hundreds of models. Pull additional models as needed:

ollama pull llama3.1:8b      # Meta's latest
ollama pull codellama:7b     # Specialised for code
ollama pull nomic-embed-text  # For better RAG embeddings

Users can switch between models in Open WebUI from a dropdown menu in the chat interface. Different tasks benefit from different models: use Mistral for general chat, CodeLlama for programming questions, and Llama for longer-form writing.

What This Replaces

With Open WebUI and Ollama deployed, you have replaced:

• ChatGPT (general chat and Q&A)
• ChatGPT with file upload (document Q&A via RAG)
• ChatGPT Team plan (multi-user, admin controls)

All running on your hardware, with zero per-user fees, and your data never leaving your server. The model quality is not GPT-4 level, but for everyday business writing, document queries, and internal knowledge base access, 7B to 13B models perform well.

If deploying AI models and configuring Docker networks is not how you want to spend your weekend, VPS1 deploys Open WebUI and Ollama for you. We handle the hardware sizing, model selection, document import, and integration with your SSO. Your team gets private AI. You get your weekend back.