Developing and deploying a modulus checking API

Standard

Following my latest post about a Go package to validate UK bank account numbers, I wanted to offer a public API to let people check if a UK bank account number is valid or not. I know that offering a Go package is not ideal for everyone because for the moment Go is not everywhere in the tech ecosystem, and it’s always convenient to have an API you can send requests to, especially in a frontend context. My goal was to offer a JSON API, supporting authentication thanks to a HTTP header and with rate limits. With this, in the future you could adapt rate limits to some API keys, if you want to allow a larger amount of requests for some clients.

Packages I used

I wanted to give cloudflare/service a go because it lets you build quickly JSON APIs with some default endpoints for heartbeat, version information, statistics and monitoring. I used etcinit/speedbump to offer the rate limiting functionality and it was very easy to use. Note that the rate limiting functionality requires a Redis server to store request counts. Finally, I used the famous codegangsta/negroni to create middlewares to handle API authentication and rate limits and keeping my only controller relatively clean.

Deploying behind Nginx

My constraints were the following:

  • The API should only be accessible via HTTPS and HTTP should redirect to HTTPS.
  • The Golang server should run on a port > 1024 and the firewall will block access to everything but ports 22, 80 and 443
  • The only endpoints that should be exposed to the public are /verify, /version and /heartbeat. Statistics and monitoring should be accessible by administrators on localhost through HTTP

I ended up with this Nginx virtual host to suit my needs, I’m not sure if it can be simpler:

geo $is_localhost {
  default 0;
  127.0.0.1/32 1;
}

server {
    listen 80;
    listen 443 ssl;

    server_name modulus.antoine-augusti.fr localhost.antoine-augusti.fr;

    ssl_certificate /etc/nginx/ssl/modulus.antoine-augusti.fr.crt;
    ssl_certificate_key /etc/nginx/ssl/modulus.antoine-augusti.fr.key;

   if ($is_localhost) {
      set $test A;
   }

    if ($scheme = http) {
      set $test "${test}B";
    }
    
    # Redirect to HTTPS if not connecting from localhost
    if ($test = B) {
      return 301 https://$server_name$request_uri;
    }
    
    # Only the following endpoints are accessible to people not on localhost
    location ~ ^/(verify|heartbeat|version)  {
      include sites-available/includes/dispatch-golang-server;
    }

    # Default case
    location / {
      # Not on localhost? End of game
      if ($is_localhost = 0) {
        return 403;
      }
      # Forward request for people on localhost
      include sites-available/includes/dispatch-golang-server;
    }
}

And for sites-available/includes/dispatch-golang-server:

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;

With this, I can still access the reserved endpoints by opening a SSH tunnel first with ssh -L4242:127.0.0.1:80 [email protected] and going to http://localhost.antoine-augusti.fr:4242/stats after.

Note that the Golang server is running on port 8080 and it should be monitored by Supervisor or whatever you want to use.

Grabbing the code and a working example

First of all, the API is available on GitHub under the MIT license so that you can deploy and adapt it yourself. If you want to test it first, you can use the API key foo against the base domain https://modulus.antoine-augusti.fr. Here is a cURL call for the sake of the example:

curl -H "Content-Type: application/json" -H "Api-Key: foo" -X POST -d '{"sort_code": "308037", "account_number": "12345678"}' https://modulus.antoine-augusti.fr/verify

Note that this API key is limited to 5 requests per minute. You’ve been warned 🙂 Looking for more requests per month or SLA, drop me a line.

Go challenge: validating UK bank account numbers

Standard

As I was reading through the SEPA specification, I found that it was not that simple to check if a UK bank account number was valid or not. If you’re not familiar with UK banks, they don’t use IBAN to transfer money within the UK, but a combination of a sort code and an account number. A sort code identifies the bank’s branch and each account has got an account number. A sort code is a 6 digits number and an account number can be between 6 and 11 digits, but most of them are 8 digits long.

For example, here is a valid UK bank account:

  • Sort code: 107999
  • Account number: 88837491

Algorithms to check if a UK bank account is valid

A very common way to check if a number (bank account, credit card, parking ticket…) is valid, is to apply a modulus algorithm. You perform an operation on each digit (addition, multiplication by a weight, substitution…), when you reach the end you divide by a specific number and you check that the remainder of the division is equal to something. Seems easy, right? Well, this is not that simple for UK bank accounts. In fact, if you want to go through the official specification on the Vocalink website, you will see that they use 2 algorithms, but they have also 15 exceptions to take into account (and some of them are weird or tricky to handle!). You will need to adapt the way you compute the modulus value according to a weight table also.

From the specification to a package

Reading the specification was interesting, but what really motivated me to code a Go package to solve this problem was the fact that test cases where provided in the specification! What a dream: the specification offers you 34 test cases, and they cover nearly all the exceptions. I jumped on the opportunity, it’s not that often that you are offered with a way to check that what you have done is actually right. In fact, I followed a Test Driven Developemnt aproach and it really guided me during the development and especially the refactoring.

Getting the code

The code is available on GitHub under the MIT license and should be well documented and tested. As always, pull requests and bug reports are welcome!

Here is an example:

package main

import (
    "fmt"

    "github.com/AntoineAugusti/moduluschecking/models"
    "github.com/AntoineAugusti/moduluschecking/parsers"
    "github.com/AntoineAugusti/moduluschecking/resolvers"
)

func main() {
    // Read the modulus weight table and the sorting
    // code substitution table from the data folder
    parser := parsers.CreateFileParser()

    // The resolver handles the verification of the validity of
    // bank accounts according to the data obtained by the parser
    resolver := resolvers.NewResolver(parser)

    // This helper method handles special cases for
    // bank accounts from:
    // - National Westminster Bank plc (10 or 11 digits with possible presence of dashes, for account numbers)
    // - Co-Operative Bank plc (10 digits for account numbers)
    // - Santander (9 digits for account numbers)
    // - banks with 6 or 7 digits for account numbers
    bankAccount := models.CreateBankAccount("089999", "66374958")

    // Check if the created bank account is valid against the rules
    fmt.Println(resolver.IsValid(bankAccount))
}

Continuous integration and code coverage in Golang

Standard

It took me some time to find the right setup and the right tools to achieve something not that complicated: continuous integration and coverage reports for Golang projects hosted on GitHub.

I’m happy to share my configuration with you, hopefully it will save you some time. I’m using Travis CI for the continuous integration platform and Codecov for code coverage reports. Both are free and easy to setup: you can get just log in using your GitHub account, you will be up and running in under 5 minutes.

Here is the Travis file (.travis.yml) I use:

language: go
before_install:
  - go get golang.org/x/tools/cmd/vet
  - go get github.com/modocache/gover
  - go get github.com/vendor/package/...
script:
  # Vet examines Go source code and reports suspicious construct
  - go vet github.com/vendor/package...
  # Run the unit tests suite
  - go test -v ./...
  # Collect coverage reports
  - go list -f '{{if len .TestGoFiles}}"go test -coverprofile={{.Dir}}/.coverprofile {{.ImportPath}}"{{end}}' ./... | xargs -i sh -c {}
  # Merge coverage reports
  - gover . coverprofile.txt
after_success:
  # Send coverage reports to Codecov
  - bash < (curl -s https://codecov.io/bash) -f coverprofile.txt

Replace github.com/vendor/package with your GitHub URL and you're good to go! You will be protected against yourself or contributors for your package. Unit tests will not break and coverage will not decrease. Or at least you will know when it happens!

Bonus: fancy badges

I like to put at the beginning of every README file a few information:

  • The status of the latest build (green is reassuring)
  • The software license, so that people immediately know if it's okay to use it for their project
  • A link to the GoDoc website, for documentation
  • The percentage of code covered by unit tests

If you want to do the same, here is what you can write at the very top of your README.md file:

# Travis CI for the master branch
[![Travis CI](https://img.shields.io/travis/vendor/package/master.svg?style=flat-square)](https://travis-ci.org/vendor/package)
# Note that this is for the MIT license and it expects a LICENSE.md file
[![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/vendor/package/blob/master/LICENSE.md)
# Link to GoDoc
[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square)](https://godoc.org/github.com/vendor/package)
# Codecov for the master branch
[![Coverage Status](http://codecov.io/github/vendor/package/coverage.svg?branch=master)](http://codecov.io/github/vendor/package?branch=master)

One more time, don’t forget to replace vendor/package (even in URLs) with your details and you’re good to go!

Demo

Head to AntoineAugusti/colors to see what it looks like.

Happy coding!