Preparing the System

Along this de­plo­y­ment we'­ll use a lot of stu­ff. So, lo­ts of stu­ff nee­ds to be ins­ta­lled and con­fi­gu­red be­fo­re we can start de­plo­y­men­t.

1. Get rid of all se­r­­vi­­ces that wi­­ll not be nee­­de­­d. Idea­­lly no­­­thing should be run­­ning on the sys­­tem for whi­­ch you do­­n't ha­­ve a co­m­­pe­­lling rea­­so­­n.

2. In­s­­ta­­ll and se­­tup DBMS. I got MyS­­QL via the Ubun­­tu pa­­cka­­ge.

3. In­s­­ta­­ll other things. In this ca­se, that mean­­t:

   • Vi­­r­­­tua­­­lenv

   • Re­­­dis (U­s­ed to crea­­­te job queues)

   • Ngi­nx (For re­­­ve­­r­­­se pro­­­­­x­­­yi­n­­­g)

   • Ga­­­tling (For sta­­­tic fi­­­le se­­r­­­vi­n­­­g)

   • mo­­­­­nit (For se­­r­­­vi­­­ce mo­­­­­ni­­­to­­­­­ri­n­­­g/­­­res­­­ta­­r­­­ti­n­­­g)

   • Exi­­­m4 (For SM­­­TP se­­r­­­vi­­­ce)

Getting the Code Up There

Sin­ce the pro­ject is hos­ted at Gi­tHu­b, it's ve­ry rea­so­na­ble to just use gi­t. Sin­ce it's pu­re py­tho­n, vir­tua­lenv and pip are good for de­pen­den­cy hand­lin­g.

So, I created a user at the VP­S, and started wri­ting a script that should get eve­r­y­thing in pla­ce, pre­sen­ted he­re wi­th mu­ch mo­re co­m­ment that it has in real li­fe:

#/bin/sh
if [ ! -d nikola-server ]
then
    # This means this is an initial deployment or else I have nuked
    # everything. So start by cloning the repo and creating the
    # virtualenv it will use
    git clone git://github.com/ralsina/nikola-server.git
    virtualenv nikola-server/venv
fi

# Go into the repo and rollback any changes to settings.py
cd nikola-server
git checkout alva/alva/settings.py
# Update from master
git pull
# Override settings.py with the deployment version, which
# has things like the proper DB settings and such.
# Yes I know I could override it using an env. variable
# bu this works too.
cp ../settings.py alva/alva/settings.py

# Activate the venv
. venv/bin/activate

# Enter the django project's folder and install all it requires
cd alva
pip install -r requirements.txt

# These are requirements for deployment which the development
# setup doesn't need (MySQL driver, gunicorn)
pip install mysql-python
pip install gunicorn

# Bring the deployment server's DB up to speed
./manage.py syncdb
# Perform any necessary DB migrations
./manage.py migrate  allauth.socialaccount
./manage.py migrate allauth.socialaccount.providers.twitter
./manage.py migrate blogs

# Put all the static files in the right place so the right
# webserver will pick it up
./manage.py collectstatic --noinput

Web Servers

This de­plo­y­ment in­vol­ves a num­ber of web­ser­ver­s:

Gunicorn, running in localhost:9000

No con­fi­gu­ra­tio­n, it uses the co­m­mand li­ne.

Gatling, running in 184.82.108.14:80 serving static files:

Configured in /etc/default/gatling:

DAEMON_OPTS="-e -v -D -S -F -U -u nobody -c /var/www"
DAEMON_OPTS="-c /srv/www -P 2M -d  -v -p 80 -F -S -i 184.82.108.14"
DAEMON="gatling"

Nginx, running as reverse proxy for Gunicorn in 184.82.108.15:80:

Configured in /e­tc/n­gi­n­x/­si­tes-e­na­ble­d/­de­fault:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        listen 184.82.108.15:80;
        index index.html index.htm;

    server_name donewithniko.la;
    # no security problem here, since / is alway passed to upstream
    root /path/to/test/hello;
    location / {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://localhost:9000/;
    }
}

Running Things

Things should start and stay started. Process should not run as root if not needed. Other than that, I have no strong requirements. Here's the monit configuration (/etc/monit/conf.d/alva.conf) , which should be fairly self-explanatory except for the startstop.sh thing explained in the sidebar:

check process gunicorn with pidfile /home/alva/gunicorn.pid
    start program = "/bin/startstop.sh start gunicorn alva /usr/bin/writelog /home/alva/gunicorn.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py run_gunicorn --bind=127.0.0.1:9000"
    stop program = "/bin/startstop.sh stop gunicorn"
    if failed host 127.0.0.1 port 9000
        protocol http then restart

check process rqworker with pidfile /home/alva/rqworker.pid
    start program = "/bin/startstop.sh start rqworker alva /usr/bin/writelog /home/alva/rqworker.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py rqworker"
    stop program = "/bin/startstop.sh stop rqworker"

check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx start"
    stop program = "/etc/init.d/nginx stop"
    if failed host 184.82.108.15 port 80
        protocol http then restart

check process gatling with pidfile /var/run/gatling.pid
    start program = "/etc/init.d/gatling start"
    stop program = "/etc/init.d/gatling stop"
    if failed host 184.82.108.14 port 80
        protocol http then restart

check process redis with pidfile /var/run/redis.pid
    start program = "/etc/init.d/redis-server start"
    stop program = "/etc/init.d/redis-server stop"

check exim4 with pidfile /var/run/exim4/exim.pid
    start program = "/etc/init.d/exim4 start"
    stop program = "/etc/init.d/exim4 stop"
    if failed host 127.0.0.1 port 25 then restart

check mysql with pidfile /var/run/mysqld/mysqld.pid
    start program = "/etc/init.d/mysql start"
    stop program = "/etc/init.d/mysql stop"
    if failed host 127.0.0.1 port 3306 then restart

So, that star­ts eve­r­y­thing I want run­nin­g, and ho­pe­fu­lly wi­ll keep it run­nin­g.

How To Deploy

First a bit of CLI ac­tio­n:

ssh -l deployer burst.ralsina.me
sh -x deploy.sh

Then, via mo­ni­t, res­tart gu­ni­corn and rqwo­rke­r. And tha­t's it. The­re's sti­ll so­me things that could be do­ne to make it mo­re strea­m­li­ned but it's good enou­gh at this poin­t.