Feature flags API in golang

Standard

Over the last few months, I’ve been interested in golang (the Go language) but I didn’t know what to build to really try it. Sure, I’ve done the exercises from the online tutorial and I’ve read the awesome website Go by example, but I didn’t have a real use-case yet. Until a few days ago when I decided to build an API related to feature flags!

What are feature flags?

Feature flags let you enable or disable some features of your application, for example when you’re under unexpected traffic or when you want to let some users try a new feature you’ve been working on. They decouple feature release and code deployment, so that you can release features whenever you want, instead of whenever the code happens to ship.

With this package, you can enable the access of a feature for:

  • specific user IDs
  • specific groups
  • a percentage of your user base
  • everyone
  • no one

And you can combine things! You can give access to a feature for users in the group dev or admin and for users 1337 and 42 if you want to.

What I’ve learned

I guess it’s a rather complete project because it involves a storage layer (a key-value store, with bolt), some logic around a simple model (what is a feature? How do we control access to a feature?) and an HTTP layer (with the default HTTP server and gorilla/mux). Moreover I’ve tried to write some tests, and it was really interesting to discover the “Go way” to do it!

Anyway, I’ve learned a lot and I’m fairly happy with the codebase, but if you spot anything that can be improved or that is wrong, please do get in touch with me (GitHub issues and tweets are perfect).

Here is the source code: github.com/AntoineAugusti/feature-flags.

Keeping my brain busy in my free time

Standard

For the last few months, I’ve tried to do sometimes in my free time what is often called “code katas”.

A code kata is an exercise in programming which helps a programmer hone their skills through practice and repetition.

Basically, you try to solve problems by writing algorithms. If you don’t know what to solve, there are a lot of platforms you can look at to find exercises:
CodinGame
Prologin (in French)
Rosalind (bioinformatics challenges)
Kaggle (machine learning competitions)

On most of these platforms, you can “play” alone. You are given a problem with detailed instructions, a dataset and an example of the expected output. Your goal will be to solve this problem. You will know if you have successfully solved the problem when you will submit your solution (usually you can use the language of your choice) thanks to automated unit tests.

I like to solve unusual problems, and most of the time they are more difficult than what I work on on a daily basis (at least when you’ve done enough exercices). This is a great opportunity to discover advanced algorithms or to learn a new language. Because you do this in your time it’s always nice to know than these problems can usually be solved in less than 1 hour, and also that you will not need to maintain your implementation in the future! When you’ve successfully solved a problem, you’re done for real 🙂

If you wanna check what I’ve already done, and maybe to see how I’ve done it, I’ve got a public GitHub repository dedicated to this: github.com/AntoineAugusti/katas.

Learning English as a French guy

Standard

I have been really interested in the English language since I was 14 or 15 years old and I was reading technical documentation, mostly about web technologies and the MaNGOS project.

It became critical to be able to understand and to speak English during my Computer Science engineering studies, because it’s an important part of the job and also because it is required to get a minimal score to the TOEIC exam in order to have the Engineering degree.

Here is a list of things I’ve tried to do over the last 5 years to improve my English level:

  • TV shows. An amazing way to do something enjoyable while learning. You will get used to hear English and you will also discover new everyday words, words or expressions you cannot learn at school. Begin with subtitles in French, move to English subtitles and then remove subtitles! If you wanna track which series you are watching and when new episodes are available, use Betaseries. Try to watch at least 1 or 2 hours of series everyday, but stay focused while watching episodes 😉
  • Read in English about topics you care about. Like English Facebook pages, follow Twitter accounts, subscribe to RSS feeds about topics you care and you read daily about: your work, life tips, sports, music, whatever. Instead of reading it in your native language, the goal is to read it in English. Don’t just say “I am going to read 1 or 2 articles everyday in English” / “I am going to spend 45 minutes everyday to read articles English”. No, you should see English articles coming your way. In fact, you should read more articles in English than in your native language if you want to progress.
  • Be known as an English-speaking person. If you’ve got a Twitter account, tweet in English. If your Facebook friends are open-minded, post in English on Facebook also. Write your résumé in English on LinkedIn, have your personal website available in English. You need to write often in English, it should become a habit!
  • Use electronic devices in English. Switch your smartphone, laptop, tablet, watch, fridge, online services accounts, social media accounts, etc. in English. First of all you will discover new technical words and you will no longer suffer from poor translations in your native language!
  • Search on search engines in English. Trust me, you will find more content in English and this is another part of your training “I am going to write and to read everything in English”.
  • Travel or work in an English speaking country. This one is an obvious one, but this is is the ultimate thing you can do to improve your English: live in an English-speaking country for a few months. While living here, get out of your comfort zone and interact with people as much as you can. Embrace your differences and discover a new culture!
  • Train for the TOEIC exam. Okay, this test is definitely not perfect, but at least it says that you should be able to work in an English environment. The maximum score is 990, the closer to this, the better! A good score to this test immediately reflects on your résumé that you’ve a decent English level. If you look online, you will find training exercises. Get used to the format of the test so that you will not be surprised when you will pass it for real, under a lot of stress. Be warned: this is difficult to stay really focused during 1h30, with no pauses.

Decorator pattern and repositories

Standard

My use case

Lately I’ve been using a lot the decorator pattern with repositories on Teen Quotes. My use case is somewhat simple: I use a relational database (MySQL) but sometimes I what to cache the results of some queries in a key-value store (Redis / Memcached for instance). With something like that, I don’t need to hit my database for queries that are always run or are slow to run, I’ll hit my key-value store instead. It’ll reduce the pressure on my database and will give some results faster for the application.

The decorator pattern

If you’re not familiar with the decorator pattern yet, it’s quite simple to use and I’m sure you’ll love it in no time. Basically, the decorator pattern allows behavior to be added to an individual object, either statically or dynamically, without affecting the behavior of other objects from the same class. As Richard Bagshaw said, the idea is that you take an object, and you wrap this object in another object which provides additional functionality, and you keep wrapping extra classes repeatedly for each additional requirement.

If you want to see some real world examples, continue to read this blog post or go directly to Laracasts.

Some code

I’ll show you something I’ve been working on last week: the ability to add tags to quotes. A “tag” is like a category for a “quote” (a post, an article, whatever you want to call it). I’m using Laravel with Eloquent for my relational database. I’ve created an interface called TagRepository.

namespace TeenQuotes\Tags\Repositories;

use TeenQuotes\Tags\Models\Tag;
use TeenQuotes\Quotes\Models\Quote;

class DbTagRepository implements TagRepository {

  /**
   * Create a new tag
   *
   * @param  string $name
   * @return \TeenQuotes\Tags\Models\Tag
   */
  public function create($name)
  {
    return Tag::create(compact('name'));
  }

  /**
   * Get a tag thanks to its name
   *
   * @param  string $name
   * @return \TeenQuotes\Tags\Models\Tag|null
   */
  public function getByName($name)
  {
    return Tag::whereName($name)->first();
  }

  /**
   * Add a tag to a quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @param  \TeenQuotes\Tags\Models\Tag $t
   */
  public function tagQuote(Quote $q, Tag $t)
  {
    $q->tags()->attach($t);
  }

  /**
   * Remove a tag from a quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @param  \TeenQuotes\Tags\Models\Tag $t
   */
  public function untagQuote(Quote $q, Tag $t)
  {
    $q->tags()->detach($t);
  }

  /**
   * Get a list of tags for a given quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @return array
   */
  public function tagsForQuote(Quote $q)
  {
    return $q->tags()->lists('name');
  }

  /**
   * Get the total number of quotes having a tag
   *
   * @param  \TeenQuotes\Tags\Models\Tag $t
   * @return int
   */
  public function totalQuotesForTag(Tag $t)
  {
    return $t->quotes()->count();
  }
}

Pretty simple stuff, I’m sure you’ve seen this multiple times. Let’s move on to the interesting part: the caching layer. We will create a new class CachingTagRepository implementing the same interface TagRepository. The key thing is that we’ll require a TagRepository class to be given in the constructor of this new class. Ultimately, we will pass the DB layer here.

 
namespace TeenQuotes\Tags\Repositories;

use Cache;
use TeenQuotes\Tags\Models\Tag;
use TeenQuotes\Quotes\Models\Quote;

class CachingTagRepository implements TagRepository {

  /**
   * @var \TeenQuotes\Tags\Repositories\TagRepository
   */
  private $tags;

  public function __construct(TagRepository $tags)
  {
    // The key thing is here: we assume we've already
    // a class that is implementing the interface.
    // We can rely on that!
    $this->tags = $tags;
  }

  /**
   * Create a new tag
   *
   * @param  string $name
   * @return \TeenQuotes\Tags\Models\Tag
   */
  public function create($name)
  {
    return $this->tags->create($name);
  }

  /**
   * Get a tag thanks to its name
   *
   * @param  string $name
   * @return \TeenQuotes\Tags\Models\Tag|null
   */
  public function getByName($name)
  {
    $callback = function() use ($name)
    {
      return $this->tags->getByName($name);
    };

    return Cache::rememberForever('tags.name-'.$name, $callback);
  }

  /**
   * Add a tag to a quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @param  \TeenQuotes\Tags\Models\Tag $t
   */
  public function tagQuote(Quote $q, Tag $t)
  {
    Cache::forget($this->cacheNameForListTags($q));

    $keyTotal = $this->cacheNameTotalQuotesForTag($t);

    if (Cache::has($keyTotal))
      Cache::increment($keyTotal);

    return $this->tags->tagQuote($q, $t);
  }

  /**
   * Remove a tag from a quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @param  \TeenQuotes\Tags\Models\Tag $t
   */
  public function untagQuote(Quote $q, Tag $t)
  {
    Cache::forget($this->cacheNameForListTags($q));

    $keyTotal = $this->cacheNameTotalQuotesForTag($t);

    if (Cache::has($keyTotal))
      Cache::decrement($keyTotal);

    return $this->tags->untagQuote($q, $t);
  }

  /**
   * Get a list of tags for a given quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @return array
   */
  public function tagsForQuote(Quote $q)
  {
    $key = $this->cacheNameForListTags($q);

    $callback = function() use($q)
    {
      return $this->tags->tagsForQuote($q);
    };

    return Cache::remember($key, 10, $callback);
  }

  /**
   * Get the total number of quotes having a tag
   *
   * @param  \TeenQuotes\Tags\Models\Tag $t
   * @return int
   */
  public function totalQuotesForTag(Tag $t)
  {
    $key = $this->cacheNameTotalQuotesForTag($t);

    $callback = function() use ($t)
    {
      return $this->tags->totalQuotesForTag($t);
    };

    return Cache::remember($key, 10, $callback);
  }

  /**
   * Get the key name when we list tags for a quote
   *
   * @param  \TeenQuotes\Quotes\Models\Quote $q
   * @return string
   */
  private function cacheNameForListTags(Quote $q)
  {
    return 'tags.quote-'.$q->id.'.list-name';
  }

  /**
   * Get the key name to have the number of quotes
   * having a tag
   *
   * @param  \TeenQuotes\Tags\Models\Tag $t
   * @return string
   */
  private function cacheNameTotalQuotesForTag(Tag $t)
  {
    return 'tags.tag-'.$t->name.'.total-quotes';
  }
}

You see, we do some things before (or after) calling the initial implementation, to add some functionalities (here a caching layer). Sometimes we directly defer to the initial implementation (see the create method).

Bonus: registering that in the IoC container

Let’s bind our TagRepository interface to the caching layer and the storage layer in a service provider!

namespace TeenQuotes\Tags;

use Illuminate\Support\ServiceProvider;
use TeenQuotes\Tags\Repositories\CachingTagRepository;
use TeenQuotes\Tags\Repositories\DbTagRepository;
use TeenQuotes\Tags\Repositories\TagRepository;

class TagsServiceProvider extends ServiceProvider {

  /**
   * Bootstrap the application events.
   *
   * @return void
   */
  public function boot()
  {
      //
  }

  /**
   * Register the service provider.
   *
   * @return void
   */
  public function register()
  {
      $this->registerBindings();
  }

  private function registerBindings()
  {
      $this->app->bind(TagRepository::class, function()
      {
          return new CachingTagRepository(new DbTagRepository);
      });
  }
}

Et voilà ! Happy coding!

Why Twitter was down for 5 hours. It’s all about calendars.

Standard

Twitter was down for many users this Monday, between midnight and 5 AM, CET. Android app users were logged out and the API was not working during a few hours. A bug in a line of code caused the service to think that it was 29 December, 2015.

Gregorian VS ISO calendars

The ISO week numbering system uses the YYYY format for the year instead of the Gregorian calendar’s yyyy. It then looks at which week of the year it is, and then uses a date digit with 1 starting on Monday. So, for example, Tuesday of the 50th week of 2014 would have been 2014-W50-2 in ISO week format. The problem comes in when January 1 of the new year ends up falling on a date that doesn’t get along well with the ISO week format. The first day of 2015 will start on a Thursday, whereas the ISO standard expects the first week of the year to start on “the Monday that contains the first Thursday in January.” In 2014-2015, that would be January 1st, 2015. That’s why Twitter believed that we were in 2015 on Monday, because “the first Thursday in January” is in fact this week.

Afterwards, some parts of the system believed we were almost a year later, while other were perfectly aware that we were still in 2014. Users were not able to log in (and were logged out) because tokens have a limited lifetime and therefore are not valid a year later.

Dates are hard to handle

Dates are really hard to handle when programming, and it’s very easy to make a mistake without breaking a thing until the day it will eventually explode everything because calendars or timezones are different. The same kind of mistake happened two years ago with the “Do Not Disturb” feature on iPhones, and it isn’t the last time it will happen for sure.

Using JavaScript / jQuery to detect if someone is using AdBlock

Standard

Let’s face it: ads are not cool. I know, but sometimes you have no other choice than to use them to pay your server / your coffee / whatever. And if you are using ads, you know that some people will not see your ads because they are using a plugin that blocks ads on your website. You want to reach these visitors and just say

“Hey, we know ads are not cool, but please make an exception for us or give us a small amount of money, just to support our work”

On Teen Quotes I am showing a friendly message in place of the only ad to say what I have to say. Here is what people with an ad blocker are seeing:
adblock

But let’s get back to the point: detect visitors that are using an ad blocker with JavaScript.

Some code

Let’s write some code. You’re going to write the most simple JS file you have ever written. Ready? Here it is:

var isNaughtyVisitor = false;

BOOM! That was fast, isn’t it? In fact, we don’t care about the variable name or its value, we just want to define a new variable. But this variable needs to be defined in a file called ads.js that you will include in your HTML. Why so? This is were it gets funny. Ad blockers are a little dumb, so when they see a file named something like ad.js or ads.js, they will block the request. And if the request was blocked… your variable will not be defined!

The second step is just to test in your regular JS file of your application if the variable was defined. This file needs to be include after the previous file, ads.js. You can add something like this:

$(document).ready(function() {
    // The div that will add to the DOM if the visitor is 
    // using an ad blocker
    var div = 'Your amazing HTML block here';
    
    // The script was never called, probably using an ad blocker
    if (typeof(isNaughtyVisitor) == "undefined") {
        // Insert the div wherever you want
        $("#footer").before(div);
        // Send the event to Google Analytics if you want to track
        ga('send', 'event', 'ads', 'hidden');
    // A friendly user!
    } else {
        // Send the event to Google Analytics if you want to track
        ga('send', 'event', 'ads', 'displayed');
    }
});

Pretty darn simple, and really effective. The downside is that friendly users will have to make an extra HTTP request. Yep, I know, it’s not fair for them.

Using ImageMagick to crop images

Standard

ImageMagick is a PHP library for image processing and image generation. It is a requirement for a popular library, stojg/crop that offers various cropping techniques. The library can be installed easily via Composer, but it requires ImageMagick to work.

Installing ImageMagick on Ubuntu / Debian

You are lucky, there is a package for that. It is just a one line command:

sudo apt-get install php5-imagick

It will grab ImageMagick, build the PHP extension and enable it in your php.ini file. You may have to restart your web server or PHP-FPM to reflect the changes.

Using ImageMagick on Travis CI

If you are running a continuous integration test suite, you may be interested to test some methods that are using the ImageMagick library under the hood. In order to do that, the ImageMagick library needs to be installed on the Travis’ VM. At the time, the ImageMagick library is already pulled in for every VM, but the PHP extension is not enabled by default. We will still try to grab the library before enabling it, just to be cautious. You’ll have to edit your travis.yml file and add the following lines to your before_script:

before_script:
  - sudo apt-get update -q
  - sudo apt-get install -y imagemagick
  - printf "\n" | pecl install imagick

The last line is a very dirty hack: when running the command pecl install imagick, you have to hit “Enter” to autodetect some settings. That’s why you need to pipe a new line to this command. Dirty, but it works, you’ll be able to test methods using ImageMagick afterwards.

Fighting against suicide on Teen Quotes

Standard

Teen Quotes lets teenagers share their daily thoughts and feelings. Since people speak their mind, we receive happy quotes about friendship and love, regular life inspiring quotes and sad quotes. And sometimes, unfortunately, really really sad quotes.

At Teen Quotes, we have a very strict publishing policy. Every quote gets reviewed by a human before being published. If we don’t like a quote, we can edit it a little bit to make it publishable or we can refuse it.

How to manage refused quotes

In the past, when we received a quote that we didn’t want to publish, here is what happened:

  • The submitted quote goes through moderation and gets refused. As an immediate result, it will never be published on Teen Quotes.
  • The author of the quote receives an email telling that its quote was refused with a few hints to improve: verify the English, be sure that a similar quote wasn’t published before and try to write an original quote.

This type of email is sent a lot because we refuse around 75 % of the submitted quotes. We would be super happy to explain in every email why this specific quote was refused but it would take too much time during moderation.

But I am convinced that sometimes we should contact the author of the quote, in specific situations. One of the possible situation is described in this post’s title: suicide promotion / suicide thoughts in a quote.

Submitting super sad quotes is a real alarm

As already said, users submit quotes on Teen Quotes to inspire others. Most of our visitors come just to get inspired, to relate to others, to put words on feelings. We deeply think that we shouldn’t publish quotes that promote suicide thoughts. Everyone has ups and downs, but suicide should not be an option. So we protect other members from Teen Quotes by refusing such quotes. And today, we are going one step further.

Trying to help authors that need assistance

In the upcoming major version of Teen Quotes, if a user submits a really sad quote, we will send him a personal email to make sure that everything is right and we will include some useful links if he needs to get help from non-profit organisation around the world. As a software engineer, it is usually difficult to have an impact in someone’s life. Moving a button 2 pixels to the left will not make a huge difference. We deeply hope that we will be able to help people with this feature. It really matters to us.

Laravel package for the recommendation system Easyrec

Standard

During a few days in August I have coded a Laravel wrapper for the recommendation system Easyrec. If you want to display most viewed items, best rated items, related items or something like that, a recommendation system is the perfect way to go. If you are familiar with Machine Learning techniques, you now that a recommendation system is something very difficult. If you remember, Netflix offered $ 1,000,000 to improve its collaborative filtering algorithm.

Easyrec provides a REST API that you can call for free. This is something very convenient if don’t have a lot of data and that you still want to use a recommender system for your web service.

Features overview

After registering the service provider and the alias in your app/config/app.php file, it will be super easy to use! Here is a little overview of the available functions:

Easyrec::view(42, 'Post 42', 'http://example.com/posts/42', 200, null, null, 'POST'); // User #200 has viewed post #42
Easyrec::rate(42, 8, 'Book 42', 'http://example.com/books/42', 200, null, null, 'BOOK'); // User #200 has rated 8/10 book #42
Easyrec::mostViewedItems(10, 'WEEK', 'BOOK'); // Retrieves the 10 most viewed books for the last week
Easyrec::bestRatedItems(10, 'MONTH', 'POST'); // Retrieves the 10 best rated posts for the last month

Documentation and download

Of course the package is available via Composer. The full documentation can be found on GitHub: github.com/AntoineAugusti/laravel-easyrec.

Do not hesitate to open up an issue if something is not working as expected. Pull-requests are very welcome also!

Error handlers order in Laravel

Standard

Error handlers are a neat feature of Laravel. They allow you to deal with custom exceptions so that you know what to do when something wrong happens. If you want to play with error handlers, you need to know two vital things:

  1. If an exception handler returns a response, that response will be sent to the browser and no other error handlers will be called ;
  2. Handlers are called based on the type-hint of the Exception they handle. If two handlers handle the same exception, the last one will be called first.

About error handlers order

An example:

App::error(function(MyCustomException $exception, $code)
{
	// This handler will not be called first
});

App::error(function(MyCustomException $exception, $code)
{
	// This handler will be called first
});

Error handlers and responses

If your last handler returns a response, your previous handler will not be called because of the first rule.

App::error(function(MyCustomException $exception, $code)
{
	// This handler will not be called AT ALL
});

App::error(function(MyCustomException $exception, $code)
{
	// This handler will be called first
	$data = ['title' => 'Error occured'];

	// We are returning a response: no other error handlers will be called
	return Response::view('errors.default', $data, $code);
});

Playing with error handlers order

Let’s say you want an error handler to be called at the bottom of the stack. This is very useful for example if you are using a package that registers an error handler and you want one of your error handlers to be called after. Laravel provides the App::pushError() function that registers an error handler at the bottom of the stack.

App::error(function(MyCustomException $exception, $code)
{
	// This handler will be called first
	// If we are returning a response here, no other error handlers will be called
});

App::pushError(function(MyCustomException $exception, $code)
{
	// This handler will be called at the end
	$data = ['title' => 'Error occured'];

	return Response::view('errors.default', $data, $code);
});