TL;DR: I’ve deployed Discourse on Kubernetes (K8s) for my company’s internal discussion platform. Because I couldn’t find a simple tutorial, I documented my steps to help other developers do it too.

Why would I want to deploy Discourse on Kubernetes?

1. Our company already has a Kubernetes cluster for random tools and staging deployment, so it is cheaper to deploy on the existing cluster for an internal Discourse usage.
2. As a founder, I don’t get many chances to code anymore. I wanted to learn how to use Kubernetes because my team has been using it a lot lately.

A quick overview of this tutorial

The tutorial and the sample config below shows how to deploy a single Discourse web-server. This server needs to connect to a PostgreSQL and Redis server. We are using Google Cloud Registry and gcePersistentDisk for storage.

So let’s begin:

Create Discourse app Docker image

We will “misuse” the launcher provided by discourse_docker to create the docker image for the Discourse web server. And by “misuse” I mean that we have over-used the launcher script to create docker images for production use.

1.Clone from https://github.com/discourse/discourse_docker to your local environment.

2.Setup a temporary Redis server and PostgresSQL database in the local environment.

3.Create a containers/web_only.yml (as shown below)

• The env var is not relevant to K8s. It’s just for building the local image, so let’s fill in something that works for your local environment.
• Determine the plugins you want to install with your Discourse setting here.

4.Tip: The local Redis instances might be in protected-mode, and won’t allow a Docker guest to host the connection. For this case, you should start your Redis server with the protected-mode turned off: redis-server --protected-mode no

5.Create the Docker images and upload the images to your K8s Docker registry. In this case, we are using Google Cloud Registry:

• Create Docker image with Discourse’s launcher: ./launcher bootstrap web_only
• Verify that the image is created: docker images. You should see the Discourse image in the list if successful.
• Upload the image to the registry with this command:

docker tag local_discourse/web_only gcr.io/**my-cluster**/discourse:latestgcloud docker — push gcr.io/**my-cluster**/discourse:latest

view raw
gistfile1.txt
hosted with ❤ by GitHub

Sample config in web_only.yml :

‍

templates:
 – "templates/web.template.yml"
 – "templates/web.ratelimited.template.yml"
env:
 LANG: en_US.UTF-8
 UNICORN_WORKERS: 2
 DISCOURSE_DB_USERNAME: chpapa
 DISCOURSE_DB_PASSWORD: ''
 DISCOURSE_DB_HOST: docker.for.mac.localhost
 DISCOURSE_DB_NAME: chpapa
 DISCOURSE_DEVELOPER_EMAILS: 'bencheng@oursky.com'
 DISCOURSE_HOSTNAME: 'localhost'
 DISCOURSE_REDIS_HOST: docker.for.mac.localhost
hooks:
 after_code:
   – exec:
       cd: $home/plugins
       cmd:
         – mkdir -p plugins
         – git clone https://github.com/discourse/docker_manager.git
         – git clone https://github.com/discourse/discourse-solved.git
         – git clone https://github.com/discourse/discourse-voting.git
         – git clone https://github.com/discourse/discourse-slack-official.git
         – git clone https://github.com/discourse/discourse-assign.git
run:
 – exec:
     cd: /var/www/discourse
     cmd:
       – sed -i 's/GlobalSetting.serve_static_assets/true/' config/environments/production.rb
       – bash -c "touch -a /shared/log/rails/{sidekiq,puma.err,puma}.log"
       – bash -c "ln -s /shared/log/rails/{sidekiq,puma.err,puma}.log log/"
       – sed -i 's/default \$scheme;/default https;/' /etc/nginx/conf.d/discourse.conf

‍

view raw
web_only.yml
hosted with ❤ by GitHub

Now we’re ready to deploy to K8s

1. Prepare a persistent volume

We will need a persistent volume as the database to store the user info and the discussion items. We are using GCEPersistentDisk for the persistent disk on the K8s cluster. Now, let’s create two 10GB disks for the app and database respectively. You may consider your Discourse usage to adjust the disk size configuration.

gcloud compute disks create –size=**10GB** –type=**pd-ssd** –zone=**us-east1-b** **discourse**
gcloud compute disks create –size=**10GB** –type=**pd-ssd** –zone=**us-east1-b** **discourse-pgsql**

‍

view raw
gcloud_disksettings.sh
hosted with ❤ by GitHub

2. Deploy to Kubernetes

Next, we will configure the deployment settings of the K8s cloud. Customize the sample K8s file. Here are some variables you probably want to tweak:

volumes.yaml

• For both persistent volumes:
  • metadata.name
  • spec.capacity.storage
  • spec.gcePersistentDisk.pdName (for the persistent disk name above)
  • spec.claimRef.namespace (for the namespace you’re using in K8s)
• The sample file here assumes you are using gcePersistentDisk. volumes.yaml needs to change heavily depending on what type of persistent disk you plan to use.

redis.yaml

• Redis Deployment:
  • spec.template.spec.containers.resources.* (CPU and Memory resources for cache server)

pgsql.yaml

• PersistentVolumeClaim (pgsql-pv-claim):
  • spec.resources.requests.storage (storage of DB server)

discourse.yaml

• PersistentVolumeClaim (discourse-pv-claim)
  • spec.resources.requests.storage (storage of web-server disk for logs and backups)
• Deployment (web-server)
  –spec.template.spec.containers.image (Set the URL to point to your Docker image)
  –spec.template.spec.containers.env

DISCOURSE_DEVELOPER_EMAILS
DISCOURSE_HOSTNAME
DISCOURSE_SMTP_ADDRESS
DISCOURSE_SMTP_PORT

• spec.template.spec.containers.resources.* (CPU and Memory resources for your web-server)

ingress.yaml

• spec.rules.host
• spec.tls.hosts

Recommended: From there, you might want to create your own namespace for the deployment. Also assume you have set the right context to run the kubectl command in the namespace. (For details, read Kubernetes documentation). Otherwise, you should rename most of the names in the config files above to unique names and add some labels.

Apply secrets. dbUsername and dbPassword can be anything you want. Please set the right smtpUsername and smtpPassword for the mail delivery services you use.

Another note on HTTPS for Ingress: you should read this document and the Ingress controllers specific to your cluster and update ingress.yamlaccordingly.

apiVersion: v1
kind: Secret
metadata:
 name: secret
type: Opaque
data:
 dbUsername:
 dbPassword:
 smtpUsername:
 smtpPassword:

‍

view raw
secret.yaml
hosted with ❤ by GitHub

Apply all config files:

kubectl apply -f secret.yamlkubectl apply -f volumes.yamlkubectl apply -f redis.yamlkubectl apply -f pgsql.yaml

view raw
apply_configs.sh
hosted with ❤ by GitHub

Before starting the app, run the following on PostgreSQL instance to initialize the database properly. You can find your pod name by running kubectl get pods.

kubectl exec **pgsql** — su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
kubectl exec **pgsql** — su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
kubectl exec **pgsql** — su postgres -c 'psql **discourse** -c "create extension if not exists hstore;"'
kubectl exec **pgsql** — su postgres -c 'psql **discourse** -c "create extension if not exists pg_trgm;"'

‍

view raw
init_pgsql.sh
hosted with ❤ by GitHub

Create Discourse deployment and Ingress with these commands:

kubectl apply -f discourse.yamlkubectl apply -f ingress.yaml

view raw
create_deployment.sh
hosted with ❤ by GitHub

From here, your Discourse instance should be up and running. Below are some useful commands in case things don’t work and require debugging:

# Check logs of k8s pod
kubectl logs –since=1h –tail=50 -lapp=web-server

# Open an interactive terminal into the web-server / pgsql server:
kubectl exec -it **web-server** — /bin/bash
kubectl exec -it **pg-sql** — /bin/bash

# You might want to do the following:
# * Delete / create the pgsql database
# * Check logs under /shared/log/rails
# * Start stop unicorn by sv stop unicorn
# * Run bundle exec rake db:migrate / admin:create in case something went wrong under /var/www/discourse
# * You can also check the logs with an admin account at /admin/logs of your deployment

‍

view raw
k8s_debugg_message.sh
hosted with ❤ by GitHub

Setup S3 backup and file upload

Discourse can use AWS S3 for backup and file upload. Here are the steps to enable it:

1. Create two S3 buckets: one for backup and one for file upload. Set them as private.
2. Create an IAM user with API access only and attach the AWS inline policy below:

https://gist.github.com/ourskycode/12a1e1f77883fd590c615ed9be73676f#file-aws-s3-json

1. Fill in the Access Key and Key ID in Discourse Setting.

Then Discourse can upload files to the S3 bucket you’ve specified, so you can attach image and file attachments in each post.

That’s it!

I hope this piece is helpful for you to set up your own Discourse platform. It’s also a practical exercise for me to try deploying an app on K8s.

Potential improvement and scaling up:

• It’s possible to run multiple replicas for Discourse web-server for scaling up. It should work, but I haven’t tried yet.
• We could also deploy Master-Slave PostgreSQL for scaling up. We’re using Bitnami’s PostgreSQL docker image and you can read the relevant instructions here.

‍