Lightweight Environment

How to set up GOGS, Drone, and Haproxy on a tiny VPS environment in Ubuntu 15

The dust is settling around my development sandbox for 2016. I've got what feels like a good mix of monitoring, scalability, and cost (about $10 / month) - lots of juice for the squeeze.

Here's a guide through most of what I have set up and how you can do it yourself. My primary goal was to have an online presence that is fairly scalable with low costs. I'll talk about how to set up SSH and users, Git-based version control, deploy automation, and load balancing. I'll briefly touch on service management, deploy scripts, and reverse proxies, then we'll call it a day.

VPS environment

I've settled on Digital Ocean, a VPS provider with pretty good pricing and great community articles. Digital Ocean charges per hour, so it won't cost you much to play around.

This guide will use the smallest instance (droplets), weighing in at 512MB of memory and 20GB disk space, running the default Ubuntu 15.04 x64 images. I'll refer to the IP of this instance as 192.0.2.1.

SSH Keys

PuTTYgen is a good tool to make SSH keys in Windows (it's bundled with PuTTY). We'll make two keys: one for user root, and one for user deploy, who will own everything running on the commit and deploy chain.

On Windows, various programs (like Git) will look in your Documents folder for SSH keys, so it's good to save them there.

Droplets and Users

Use PuTTY to SSH into the new droplet:

...and create the deploy user:

The new user should have sudo privileges:

Now, you should be able to make a new profile in PuTTY to SSH in as deploy and run sudo commands without requiring a password.

There's various programs to wrap the PuTTY profile tool into tabbed interfaces with various extras. My current favorite is SuperPuTTY.

Swap

Since you're running a tiny environment with limited memory available, you'll need to set up some swap space to give a little extra capacity to the memory. A good rule of thumb:

Swap should equal 2x physical RAM for up to 2 GB of physical RAM, and then an additional 1x physical RAM for any amount above 2 GB, but never less than 32 MB.

Digital Ocean has a good guide on adding swap. It's worth reading. Here's the distilled content:

Dependencies

There's a few dependencies required for the various software installs below, so knock them out in one fell swoop:

Note that MySQL requires a password to be set for its root user, which will be used later.

Go

Go is required to run GOGS ("Go Git Service").

GOGS

I tried Gitlab earlier this year, and it was too demanding for tiny instances with limited resources. That said, it was a good experience - they have a CI runner built in and a straightforward installation process. It's a great resource for small or medium-size organizations, but not for my one man show.

GOGS is the lightweight go-to solution at the moment. It's an adolescent project, written in Go, that provides a great way to host Git version control. I cobbled together this process from a few install guides.

If everything went well, it's now time to configure GOGS. These settings and others are stored in the custom configuration file (at /home/deploy/go/src/github.com/gogits/gogs/custom/conf/app.ini) and can be manually edited later.

Service Management

At this point, you'll have to kill the GOGS process, and it becomes apparent we need a service. There are several ways to get GOGS up and running as a daemon when the system starts.

Upstart manages scripts in the /etc/init folder, but is being "transitioned" out of Ubuntu. Upstart starts processes greedily, triggered by events, whether they are needed or not. Its replacement, Systemd, starts processes lazily, triggered by dependencies. A third option is Supervisor, which prefers the role of "service manager" rather than "init manager".

Our continuous deployer, Drone, uses either Upstart or Systemd, so we'll use them too. I'll show both here in case you prefer Ubuntu 14. (If you're interested in Supervisor, here's how to do that.)

Systemd (Ubuntu >= 15)

Upstart (Ubuntu <= 14)

Boom - version control. Now to add deployment automation.

Drone

Drone is a lightweight continuous deployment tool that adds Git hooks to watch your commits. It then triggers builds from a build file that is in your project. They recently released Drone 0.4, with new docs and easier installation. It's a great project that's on the up-and-up.

Config

Create the config file at /etc/drone/dronerc, and add the following configuration: 

REMOTE_DRIVER=gogs
REMOTE_CONFIG=http://192.0.2.1:3000
DATABASE_DRIVER=sqlite3
DATABASE_CONFIG=/var/lib/drone/drone.sqlite

Pull the Docker image: docker pull drone/drone:0.4, and fire up the Docker container: 

sudo docker run
	--volume /var/lib/drone:/var/lib/drone
	--volume /var/run/docker.sock:/var/run/docker.sock
	--env-file /etc/drone/dronerc
	--restart=always
	--publish=4000:8000
	--detach=true
	--name=drone
	drone/drone:0.4

Note that in the installation docs, Drone is published to :80 by default. Our load balancer will be listening on that port for all incoming traffic, so Drone has to be served from a different port - we'll use :4000.

You should now be able navigate to 192.0.2.1:4000 and see something like this: Successful Drone install

You can now log in with your GOGS account. Drone will automatically synchronize publically available repos, and you can activate the ones you wish to watch. They're triggered by deployment scripts.

A word on databases

You may be thinking, "If both Drone and GOGS have support for both MySQL and SQLite, why don't we configure them to use the same database software, and get a performance increase?" The answer, for now, is surprisingly clear.

Deploy Scripts

Configuration files are popular at the moment in continuous deployment solutions for good reason:

If you've used Jenkins, you know that anything in the above list takes some doing to accomplish. Tools like Gitlab CI and Travis CI use configs written in YAML. I've found it to be a great approach.

Drone offers a variety of ways to work with its Docker container when the build task runs. A very basic configuration looks like this: 

build:
  image: golang
  commands:
    - echo hello from drone.yml
    - whoami
    - ./your-deploy-script.sh
Once this is run, you'll see your commands being executed in the build output. The last line runs a shell script you can create in the root of your project. Similar to Jenkins, I've found it's easiest to have a shell script to run, mostly because they support variables to hold server IPs or repository URIs.

Dockerfile

Drone uses Docker, so you may need to construct a Dockerfile for your deploy runner image. Digital Ocean has a helpful guide with most of the commands. Here's a simple example Dockerfile: 
FROM ubuntu

RUN apt-get -y --install-recommends --install-suggests update
RUN apt-get -y --install-recommends install git

Here's a few helpful commands to help work with Docker: 

# Use a Dockerfile to build an image with a (t)ag.
sudo docker build -t IMAGENAME /absolute/path/to/dockerfile

# Run Bash that is (i)nteractive and outputs to (t)erminal
sudo docker run -it IMAGENAME /bin/bash

# Output report of all images, filter for untagged, and for each one, remove.
sudo docker rmi $(docker images | grep "^<none>")

# Output report of (a)ll containers (IDs only using the -q flag) and for each one, remove.
sudo docker rm $(docker ps -aq)

HAProxy

This example assumes you have a domain name ("YOURDOMAIN.COM") and it's registered to your droplet IP. You should be able to reach GOGS on YOURDOMAIN.COM:3000.

Reverse proxy-ing

Basically, HAProxy listens for HTTP requests on :80, checks if they match an (a)ccess (c)ontrol (l)ist, and routes to a specific backend or a default.

The config is fairly readable; any unclear commands are explained pretty well in the documentation. You can see that HAProxy works as a reverse proxy server, distributing requests to backends. You can experiment and make addresses for Drone, and the stats monitoring page (search for "Monitoring HAProxy") served by HAProxy.

Most important, you can now add as many servers as you want and the traffic will be balanced across them. Go forth and scale!