LibreChat Multi-Provider

Deploy LibreChat on Clore.ai — a powerful, self-hosted ChatGPT alternative with multi-provider AI support, conversation branching, and plugin ecosystem on affordable GPU cloud infrastructure.

Overview

LibreChat is an enhanced, open-source ChatGPT-like interface with 22K+ GitHub stars. It faithfully reimagines the ChatGPT experience while adding features the original lacks — multi-provider switching within the same conversation, conversation branching/forking, a rich plugin system, file uploads with vision, and a full code interpreter sandbox.

Why run LibreChat on Clore.ai?

• True multi-provider in one UI — Switch between GPT-4, Claude 3.5, Gemini Pro, Mistral, and local Ollama models mid-session.

• No GPU needed for the app — LibreChat is a Node.js application; it only needs compute for inference if you attach a local LLM backend.

• Cost-effective self-hosting — Clore.ai pricing starts at fractions of a cent per minute, ideal for running a personal AI hub.

• Persistent conversations — MongoDB stores your full chat history server-side, unlike browser-local solutions.

• Team-friendly — Multi-user support with individual API key management.

Key Features

Feature	Description
Multi-provider	OpenAI, Anthropic, Google, Azure, Mistral, Ollama, OpenRouter
Conversation branching	Fork and explore alternative responses
Plugins	Bing search, Zapier, WolframAlpha, custom tools
File uploads	Images, PDFs, documents with vision analysis
Code interpreter	Execute Python in an isolated sandbox
Artifacts	Render HTML, React, and Markdown outputs
Presets	Save and share custom model configurations

---

Requirements

Server Specifications

Component	Minimum	Recommended	Notes
GPU	None required	RTX 3090 (if adding Ollama)	Only for local LLM inference
VRAM	—	24 GB	For local models via Ollama
CPU	2 vCPU	4 vCPU	Node.js + MongoDB
RAM	4 GB	8 GB	MongoDB benefits from more RAM
Storage	20 GB	50+ GB	File uploads, model cache if local

Clore.ai Pricing Reference

Server Type	Approx. Cost	Use Case
CPU-focused (4 vCPU, 8 GB RAM)	~$0.05–0.10/hr	LibreChat + external API providers
RTX 3090 (24 GB VRAM)	$0.07–0.21/hr	LibreChat + Ollama local inference
RTX 4090 (24 GB VRAM)	$0.14–0.42/hr	LibreChat + faster Ollama/vLLM
A100 80 GB	bare metal	LibreChat + large 70B+ models

💡 Cost tip: If you only use LibreChat to route API calls to OpenAI/Anthropic/Google, you only pay for the Clore.ai server compute (cheap), not the inference hardware. Budget ~$0.05–0.15/hr for a reliable LibreChat host.

Prerequisites

• Clore.ai server with SSH access

• Docker + Docker Compose (pre-installed on Clore.ai)

• Git (pre-installed on Clore.ai)

• At least one LLM API key or a local Ollama/vLLM backend

---

Quick Start

Method 1: Docker Compose (Official — Recommended)

LibreChat's official deployment uses Docker Compose with MongoDB and MeiliSearch for full functionality.

Step 1: Connect to your Clore.ai server

ssh root@<your-clore-server-ip> -p <ssh-port>

Step 2: Clone the repository

git clone https://github.com/danny-avila/LibreChat.git
cd LibreChat

Step 3: Configure environment

cp .env.example .env
nano .env

Set at minimum:

# In .env — critical settings
MONGO_URI=mongodb://mongodb:27017/LibreChat
JWT_SECRET=your-random-64-char-secret-here
JWT_REFRESH_SECRET=another-random-64-char-secret-here
CREDS_KEY=your-random-32-char-key-here
CREDS_IV=your-random-16-char-iv-here

# API Keys (add whichever you use)
OPENAI_API_KEY=sk-your-openai-key
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
GOOGLE_KEY=your-google-gemini-key

Generate secrets quickly:

# Generate random secrets
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

Step 4: Start the stack

docker compose up -d

This starts:

• LibreChat — main application on port 3080

• MongoDB — conversation and user storage

• MeiliSearch — fast conversation search

Step 5: Verify and access

docker compose ps
docker compose logs librechat --tail 30

Open in browser:

http://<your-clore-server-ip>:3080

Register a new account on the login page.

---

Method 2: Pre-built Docker Image (Fastest)

If you want to skip building from source:

mkdir -p ~/librechat && cd ~/librechat

# Download just the docker-compose files
curl -o docker-compose.yml https://raw.githubusercontent.com/danny-avila/LibreChat/main/docker-compose.yml
curl -o .env https://raw.githubusercontent.com/danny-avila/LibreChat/main/.env.example

# Edit configuration
nano .env

# Start
docker compose up -d

---

Method 3: Single-Container Quick Test

For a rapid proof-of-concept without MongoDB (limited functionality):

docker run -d \
  --name librechat \
  --restart unless-stopped \
  -p 3080:3080 \
  -e OPENAI_API_KEY=sk-your-key \
  -e JWT_SECRET=your-jwt-secret-here \
  -e MONGO_URI=mongodb://host-gateway:27017/LibreChat \
  --add-host=host-gateway:host-gateway \
  ghcr.io/danny-avila/librechat-dev:latest

⚠️ This method requires a separate MongoDB instance. Use Method 1 for a complete setup.

---

Configuration

Adding AI Providers

Edit librechat.yaml (create it in the project root) for advanced provider configuration:

cat > librechat.yaml << 'EOF'
version: 1.1.5
cache: true

endpoints:
  openAI:
    models:
      default: ["gpt-4o", "gpt-4o-mini", "gpt-4-turbo", "gpt-3.5-turbo"]
      fetch: true

  anthropic:
    models:
      default: ["claude-opus-4-5", "claude-sonnet-4-5", "claude-3-haiku-20240307"]
      fetch: false

  google:
    models:
      default: ["gemini-1.5-pro", "gemini-1.5-flash", "gemini-pro"]
      fetch: false

  ollama:
    # Points to Ollama running on the same Clore.ai server
    baseURL: http://host-gateway:11434/v1
    apiKey: ollama
    models:
      default: ["llama3.2", "mistral", "codellama"]
      fetch: true

  custom:
    - name: "OpenRouter"
      apiKey: "${OPENROUTER_API_KEY}"
      baseURL: "https://openrouter.ai/api/v1"
      models:
        default: ["meta-llama/llama-3.1-8b-instruct:free"]
        fetch: true
      titleConvo: true
      titleModel: "meta-llama/llama-3.1-8b-instruct:free"
EOF

Mount this file in your docker-compose.yml:

services:
  LibreChat:
    volumes:
      - ./librechat.yaml:/app/librechat.yaml

Environment Variables Reference

Variable	Description	Example
MONGO_URI	MongoDB connection string	mongodb://mongodb:27017/LibreChat
JWT_SECRET	JWT signing secret (64+ chars)	Random hex string
OPENAI_API_KEY	OpenAI key	sk-...
ANTHROPIC_API_KEY	Anthropic key	sk-ant-...
GOOGLE_KEY	Google Gemini key	AI...
ALLOW_REGISTRATION	Enable public signup	true / false
ALLOW_EMAIL_LOGIN	Enable email/password login	true
DEBUG_LOGGING	Verbose logs	true
SEARCH	Enable MeiliSearch	true
MEILI_MASTER_KEY	MeiliSearch API key	Random string

Restricting Registration

For private use, disable public registration after creating your account:

# In .env
ALLOW_REGISTRATION=false

Then restart: docker compose restart LibreChat

Enabling Code Interpreter

# In .env
CODE_INTERPRETER_ENABLED=true

The code interpreter runs Python in an isolated Docker container. Ensure Docker socket is accessible.

File Upload Configuration

# In .env
# Max file size in MB
FILE_UPLOAD_SIZE_LIMIT=100

# Enable image uploads for vision models
VISION_ENABLED=true

---

GPU Acceleration

LibreChat does not use GPU directly — it's a routing layer. GPU acceleration applies to any local inference backend you connect to it.

Connecting to Ollama (Same Server)

If running Ollama on the same Clore.ai server (see Ollama Guide):

# Start Ollama with GPU support
docker run -d \
  --name ollama \
  --gpus all \
  --restart unless-stopped \
  -p 11434:11434 \
  -v ollama_models:/root/.ollama \
  ollama/ollama

# Pull models
docker exec ollama ollama pull llama3.2
docker exec ollama ollama pull codellama:13b

# In librechat.yaml, set:
# baseURL: http://172.17.0.1:11434/v1

Connecting to vLLM (High Throughput)

For high-concurrency deployments (see vLLM Guide):

# Start vLLM on an A100 Clore.ai instance
docker run -d \
  --name vllm \
  --gpus all \
  --restart unless-stopped \
  -p 8000:8000 \
  -v hf_cache:/root/.cache/huggingface \
  -e HF_TOKEN=your-hf-token \
  vllm/vllm-openai:latest \
  --model meta-llama/Llama-3.1-70B-Instruct \
  --tensor-parallel-size 2 \
  --max-model-len 8192

In librechat.yaml:

  custom:
    - name: "Local vLLM"
      apiKey: "not-needed"
      baseURL: "http://172.17.0.1:8000/v1"
      models:
        default: ["meta-llama/Llama-3.1-70B-Instruct"]
        fetch: true

GPU Sizing for Local Models

Model Size	Min VRAM	Recommended Clore GPU	Approx. Cost
7–8B (Q4)	6 GB	RTX 3090	$0.07–0.21/hr
13B (Q4)	10 GB	RTX 3090	$0.07–0.21/hr
34B (Q4)	24 GB	RTX 4090	$0.14–0.42/hr
70B (Q4)	48 GB	2× RTX 3090	$0.14–0.42/hr
70B (FP16)	80 GB	A100 80GB	bare metal

---

Tips & Best Practices

Cost Management on Clore.ai

# Snapshot your configuration before stopping the server
docker compose exec mongodb mongodump --out /tmp/backup
docker cp librechat-mongodb-1:/tmp/backup ./mongo-backup-$(date +%Y%m%d)

# Stop all containers when not in use
docker compose stop

# Or scale down to zero cost by pausing the Clore.ai instance from the dashboard

Backup Strategy

# Automated daily backup script
cat > /root/backup-librechat.sh << 'EOF'
#!/bin/bash
cd ~/LibreChat
docker compose exec -T mongodb mongodump --archive | \
  gzip > ~/backups/librechat-$(date +%Y%m%d-%H%M).mongo.gz
# Keep only last 7 days
find ~/backups -name "*.mongo.gz" -mtime +7 -delete
EOF

chmod +x /root/backup-librechat.sh
# Add to crontab: 0 2 * * * /root/backup-librechat.sh

Restoring from Backup

# Restore MongoDB dump
gunzip < ~/backups/librechat-20240101-0200.mongo.gz | \
  docker compose exec -T mongodb mongorestore --archive

Securing LibreChat

• Always set strong, unique values for JWT_SECRET and CREDS_KEY

• Disable registration after initial user creation: ALLOW_REGISTRATION=false

• Use a reverse proxy (nginx/Caddy) with HTTPS for production

• Regularly update the Docker image: docker compose pull && docker compose up -d

Nginx Reverse Proxy (Optional)

cat > /etc/nginx/sites-available/librechat << 'EOF'
server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:3080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        client_max_body_size 100M;
    }
}
EOF
ln -s /etc/nginx/sites-available/librechat /etc/nginx/sites-enabled/
nginx -t && systemctl reload nginx

---

Troubleshooting

Port 3080 not accessible

# Check if container is running
docker compose ps

# Check port binding
ss -tlnp | grep 3080

# View application logs
docker compose logs librechat --tail 50 -f

# Check Clore.ai firewall — ensure port 3080 is in your port mapping

MongoDB connection refused

# Check MongoDB status
docker compose ps mongodb
docker compose logs mongodb --tail 20

# Verify MONGO_URI in .env matches the service name
# Should be: mongodb://mongodb:27017/LibreChat (not localhost)

# Test connection manually
docker compose exec LibreChat node -e "
const mongoose = require('mongoose');
mongoose.connect(process.env.MONGO_URI)
  .then(() => console.log('Connected!'))
  .catch(e => console.error(e));
"

JWT / Authentication errors

# Regenerate secrets in .env
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

# Update JWT_SECRET and JWT_REFRESH_SECRET
# Then restart: docker compose restart LibreChat

Ollama models not appearing

# Test Ollama API from LibreChat container
docker compose exec LibreChat \
  curl -s http://172.17.0.1:11434/v1/models | python3 -m json.tool

# Ensure Ollama is listening on 0.0.0.0, not just localhost
docker exec ollama ollama serve  # Check startup logs for bind address

Out of disk space

# Check disk usage
df -h
docker system df

# Clean up Docker resources
docker system prune -f
docker volume prune -f  # WARNING: removes unused volumes

# Check LibreChat uploads directory
du -sh ~/LibreChat/client/public/uploads

Update to latest version

cd ~/LibreChat
git pull origin main
docker compose pull
docker compose up -d --build

---

Further Reading

• LibreChat Documentation — complete configuration reference

• LibreChat GitHub — source, issues, changelog

• LibreChat Docker Hub — image tags

• Running Ollama on Clore.ai — local LLM backend

• Running vLLM on Clore.ai — high-throughput inference

• GPU Comparison Guide — choosing the right GPU tier

• LibreChat Config File Reference — librechat.yaml schema