Skip to main content

Mariano Guerra's Log

This is my blog, more about me at marianoguerra.github.io

🦋 @marianoguerra.org 🐘 @marianoguerra@hachyderm.io 🐦 @warianoguerra

ameo: Redis compatible GET,SET,DEL and PUBLISH/SUBSCRIBE on riak_core with WebSocket API

Last week I was invited by Erlang Solutions to give a talk at the London's Erlang Meetup and to help at a riak_core themed hackathon at their offices the next day.

The talk slides: Building distributed applications: riak_core vs partisan in case you are interested.

After giving the introduction to riak_core the teams started to work on their projects and since they were pretty busy and didn't need much help I decided to "participate" too by implementing an idea I had in mind for a while.

The result is called ameo and can be described as:

a Redis compatible, distributed in-memory key-value store and pubsub server
implemented using riak_core that exposes the topics via WebSockets.

Why?

Many languages make it hard to provide websocket connections and live connections with clients, they work on a request/response basis and/or make it really hard/expensive to handle multiple persistent connections.

I've seen solutions that involve starting a redis server and putting usually nodejs in the front to expose redis topics via websockets, this involves two moving parts and for many projects, managing nodejs which they may not have experience with.

The solution is to just start one instance of ameo or a cluster of ameos and expose the WebSocket API to clients and the Redis API to the servers.

Servers can use their preferred Redis client library and as long as they only use GET, PUT, DEL, PUBLISH, SUBSCRIBE and UNSUBSCRIBE it will look like they are talking to a Redis server.

On my way back I had some extra time at the airport so I implemented a basic web UI to play with the websocket client and to provide a reference implementation others can use.

You can see the result in this screencast:

Implementation details:

As said earlier I use riak_core for clustering, for the Redis part I took some modules from an Erlang implementation of Redis called edis and I created a library called edis_proto that allows any project to expose a Redis compatible API to their servers with a couple lines of code.

The WebSocket part is implemented using the Cowboy Web Server.