Private GenAI chat with Librechat.ai - Part 4

This is the fourth and final part of the series about building a private GenAI chat with Librechat.ai. Part 3 of the series is here. In this post, I will show you how to install and run Librechat.ai on your Linux server using Docker and NGINX.

Let’s recall the solution diagram from the Introduction:

Diagram

Prerequisites

To access your own Librechat.ai, you need to have:

  • A Linux server with public internet access (public IP address).
    • I use V-Linux server with 2 CPUs and 4 GB of RAM, with Debian 12 (Bookworm) OS.
    • Large enough storage for your chat logs, vector databases and uploaded files (documents and images). My V-Linux server has 100 GB of storage, but as of now less than 35 GB is used.
  • The following packages shall be installed:
  • Domain name for your Librechat instance.
    • I purchased my domain name from Namecheap and I use their DNS servers.

VPS (Virtual Private Server) Providers

You can use any provider of your choice. I personally use Strato, but here are few honorable mentions, where you can get a VPS for a reasonable price:

This list is not exhaustive. Please do your research and find a provider that suits your needs, budget and which is available in your country.

Installation steps

I will follow steps described in the Ubuntu Docker Deployment Guide documentation of Librechat.ai. But I will add more details and explanations, so you can understand what is going on.

Step 1: Clone the repository

I recommend you to create a separate directory where you will keep not only the Librechat repository, but any other repositories you may want to clone and use on your server. Let’s create a directory called sources in your home directory and clone the repository there.

1mkdir ~/sources
2cd ~/sources
3git clone https://github.com/danny-avila/LibreChat.git

Step 2: Creation of configuration files

To launch your Librechat instance with your API keys and other settings, you need to have the following files:

  • .env file with the environment variables. Most of the variables are the values of different API keys, endpoints, etc. This file is used by docker-compose to set up the containers.
  • deploy-compose.yml file with the configuration of the containers. This file is used by docker-compose to set up all Librechat containers.
  • librechat.yml file with some configuration settings for the Librechat instance. This file is used by the Librechat application itself to set up application capabilities and features.
  • Configuration files for NGINX. You need to create a site specific configuration file in the /etc/nginx/sites-available directory and create a symbolic link to it in the /etc/nginx/sites-enabled directory.

Step 2.1: The .env file

In the project directory, you will find a file named .env.example. You need to copy it and rename it to .env:

1cd ~/sources/LibreChat
2cp .env.example .env

In this file you need to set up your API keys and other environment variables. This is what you shall set up to use AWS Bedrock and Google APIs (which we set up in the previous part of this series):

  1. In the AWS Bedrock section:
    • BEDROCK_AWS_DEFAULT_REGION e.g. us-east-1, us-west-2, etc. You shall set this value according to the region where you enabled access to AWS Bedrock Models.
      • e.g. BEDROCK_AWS_DEFAULT_REGION=us-east-1
    • BEDROCK_AWS_ACCESS_KEY_ID - your AWS Access Key ID
      • e.g. BEDROCK_AWS_ACCESS_KEY_ID=AKIAEXAMPLE
    • BEDROCK_AWS_SECRET_ACCESS_KEY
      • e.g. BEDROCK_AWS_SECRET_ACCESS_KEY=EXAMPLESECRET
    • BEDROCK_AWS_MODELS - comma separated list of AWS Bedrock model IDs you want to use. You can find the list of supported models here.
      • e.g. BEDROCK_AWS_MODELS=us.anthropic.claude-3-5-haiku-20241022-v1:0,amazon.nova-pro-v1:0,us.deepseek.r1-v1:0
      • For models which are marked as cross-Region inference, you need to specify the region of model inference before the model ID. For example, the documentation page you will see that the model ID for Anthropic Claude 3.5 Haiku is anthropic.claude-3-5-haiku-20241022-v1:0, but in the us-east-1 and us-west-2 regions this model is marked as cross-Region inference. Therefore, the model ID you shall use in the parameter is us.anthropic.claude-3-5-haiku-20241022-v1:0.
  2. In the RAG (Retrieval Augmented Generation) section we also need to set up variables for Amazon Bedrock. In you AWS account you shall enable access to the Amazon Titan Embed Text v1 model.
    • EMBEDDINGS_PROVIDER variable must be set to bedrock.
    • EMBEDDINGS_MODEL variable is set amazon.titan-embed-text-v1.
    • AWS_DEFAULT_REGION variable is set to the same region as the one you set in BEDROCK_AWS_DEFAULT_REGION.
    • AWS_ACCESS_KEY_ID variable is set to the same value as BEDROCK_AWS_ACCESS_KEY_ID.
    • AWS_SECRET_ACCESS_KEY variable is set to the same value as BEDROCK_AWS_SECRET_ACCESS_KEY.
  3. In the Google section:
    • GOOGLE_SEARCH_API_KEY - your Google Search API key.
    • GOOGLE_CSE_ID - your Google Custom Search Engine ID.
  4. In the YOUTUBE section:
    • YOUTUBE_API_KEY - your YouTube Data API key.
  5. In the Registration and Login section. For my own use I disabled registration features, because I manually add users to instance via built in script.
    • ALLOW_EMAIL_LOGIN - set to true.
    • All other registration related variables I set to false.
    • If you want to enable registration, please check this documentation page for more details: Authentication and configure the variables accordingly.

Step 2.2: The librechat.yaml file

In the project directory there is a file named librechat.example.yaml. You need to copy it to a librechat.yaml with this command

1cp librechat.example.yaml librechat.yml

And then you need to modify it. Example file contains many options and features, which you may not need, such as:

  1. registrations: - this section is used to configure the registration and login features. It enables logins with social media accounts, such as Facebook. Apple, GitHub, etc. If you want to disable it completely, you just comment out this section. (Which I did). Otherwise, check this chapter of the Librechat documentation: Social Authentication.
  2. actions:, mpcServers:, custom: section under endpoints:.

Here is my librechat.yaml file.

Step 2.3: The deploy-compose.yml file

The Librechat project folder contains a file named deploy-compose.yml. This is the file you shall use to set up the containers. Because it is going to be modified, it is better to make a backup copy of it first:

1cp deploy-compose.yml deploy-compose.yml.bak

I recommend applying the following changes to the file:

  1. Add configuration for logging, to limit the size of the log files and the number of log files to keep. Otherwise there is a risk that the container log files will consume a lot of disk space. This configuration shall be applied to all services in the deploy-compose.yml file. To avoid repeating of the same code block, use Extensions feature of the Docker Compose file:
 1x-logging: &logging
 2  driver: "json-file"
 3  options:
 4    max-size: "10m"
 5    max-file: "3"
 6
 7services:
 8  api:
 9    <<: *logging
10    ...
  1. Because your Librechat instance will be deployed behind NGINX, it is better to change the listening port for the Librechat’s own NGINX container. I mapped port 80 to 10080 and port 443 to 10443. This way you can run NGINX on the same server, without port conflicts. You can change the ports in the deploy-compose.yml file:
 1...
 2services
 3...
 4
 5  client:
 6    image: nginx:1.27.0-alpine
 7    container_name: LibreChat-NGINX
 8    ports:
 9      - 10080:80
10      - 10443:443

For the reference here is my deploy-compose.yml file.

Done! Now you can start the containers.

Step 3. Start the containers

To start the containers, run the following command in the project directory:

1docker-compose -f deploy-compose.yml up -d

Initial run of the containers may take a while, because the images will be downloaded from the Docker Hub. Next time, when you run the command, it will be much faster, because the images will be already downloaded.

Step 4. Configure NGINX and Let’s Encrypt

Step 4.1: NGINX configuration

I assume you already have NGINX installed and configured and you know how to add new sites to NGINX. If you don’t, please check the NGINX documentation. Because I use my own domain name on my server and I dedicated it to Librechat, I created a new configuration file in the /etc/nginx/sites-available directory, and then created a symbolic link to it in the /etc/nginx/sites-enabled directory.

Key points of the configuration file are the following:

  1. Follow the best security hardening practices and use the latest TLS versions. I use this guide from SSL Labs. And my configuration file enables only TLS 1.2 and TLS 1.3 protocols.
 1server {
 2    listen 80;
 3    listen [::]:80;
 4
 5    server_name <your_domain>;
 6    autoindex off;
 7
 8    return 301 https://<your_domain>$request_uri;
 9
10    return 404; # managed by Certbot
11}
12
13server {
14
15    listen 443 ssl http2; # managed by Certbot
16    listen [::]:443 ssl http2; # managed by Certbot
17
18    server_name <your_domain>;
19
20    ssl_protocols                   TLSv1.2 TLSv1.3;
21    ssl_prefer_server_ciphers       off;
22    ...
23}
  1. Configure redirection of traffic to the Librechat NGINX container port, which we configured above (in my case port 10080). Because HTTPS encrypton is handled by my NGINX server, I do not need to enable SSL in the Librechat NGINX container.
  2. Increase client_max_body_size parameter from the default value of 1 MB to the same value as in the librechat.yaml configuration file of the parameter fileSizeLimit (default value is 25 MB).
  3. Disable autoindex for security reasons. This way web-site visitors will not be able to see the directory listing of your web server.
  4. In the location / block, set up the proxy to the Librechat NGINX container. And add the following headers to the request:
 1server {
 2
 3    listen 443 ssl http2; # managed by Certbot
 4    listen [::]:443 ssl http2; # managed by Certbot
 5
 6    server_name <your_domain>;
 7    ...
 8    ...
 9    client_max_body_size    25m;
10    autoindex off;
11    location / {
12        autoindex off;
13        proxy_pass                              http://127.0.0.1:10080/; # the port of the Librechat NGINX container
14        proxy_set_header Host                   $http_host;
15        proxy_set_header X-Real-IP              $remote_addr;
16        proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
17        proxy_set_header X-Forwarded-Proto      $scheme;
18        satisfy                                 any;
19    }
20
21}

Step 4.2: Let’s Encrypt configuration (optional)

This step is optional, if you do not want to use Let’s Encrypt for SSL certificate. You can use any other SSL certificate provider of your choice. Use this DigitalOcean guide to set up Let’s Encrypt on your server. The guide is for Ubuntu 20.04, but it should work for Ubuntu 22.04, Ubuntu 24.04 and Debian 12 (Bookworm) as well.

Step 5. Managing users

This section of the Librechat documentation explains how to add and delete users with management scripts in the container: User Management Scripts.

Adding users 

1docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run create-user"

You will be prompted to enter the email address of the user, the password and confirm if the e-mail address is verified.

Or you can pass the e-mail, user name and if e-mail address is verified parameters directly in the command. E.g. to add user “Joe Doe” with the e-mail “joe.doe@example.com”:

1docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run create-user joe.doe@example.com joe.doe --email-verified=true"

And you will be prompted to enter the password.

Deleting users

To delete a user, use his/her e-mail address:

1docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run delete-user joe.doe@example.com"

Congratulations!

You have successfully installed and configured your own Librechat instance on your Linux server. You can access it in your browser.

  • Build agents for your use cases and freely select the best model for purpose.
  • Upload image and documents for analysis without worrying about privacy.
  • Keep your chat logs private and secure.
  • Share your chat logs with your friends and colleagues when you want to.
  • Integrate other tools, innovate, create, experiment and have fun!