Multi-Paxos with riak_ensemble Part 3
In the previous post I showed how to use riak_ensemble in a rebar3 project, now I will show how to create an HTTP API for the Key/Value store using Cowboy and jsone.
This post assumes that you have erlang and rebar3 installed, I'm using erlang 19.3 and rebar3 3.4.3.
The source code for this post is at https://github.com/marianoguerra/cadena check the commits for the steps.
Dependency Setup
To have an HTTP API we will need an HTTP server, in our case we will use Cowboy 2.0 RC 3, for that we need to:
Add it as a dependency (we will load if from git since it's still a release candidate)
Add it to our list of applications to start when our application starts
Add it to the list of dependencies to include in our release
-
Set up the HTTP listener and routes when our application starts
Read the cowboy documentation for details on how to set up the listener
We setup just one route that is handled by the cadena_h_keys module, it's a plain HTTP handler, no fancy REST stuff for now, there we handle the request on the init/2 function itself, we pattern match against the method field on the request object and handle:
- POST
-
set a key in a given ensemble to the value sent in the JSON request body
- GET
-
get a key in a given ensemble, if not found null will be returned in the value field in the response
- DELETE
-
delete a key in a given ensemble, returns null both if the key existed and if itdidn't
Any other method would get a 405 Method Not Allowed response.
The route has the format /keys/<ensemble>/<key>, for now we only allow the root ensemble to be set in the <ensemble> part of the path.
We also add the jsone library to encode/decode JSON and the lager library to log messages.
We add both to the list of dependencies to include in the release.
We will also need to have a way to override the HTTP port where each instance listens to so we can run a cluster on one computer and each node can listen for HTTP requests on a different port.
The dev and prod releases will listen on 8080 as specified in vars.config.
node1 will listen on port 8081 (override in vars_node1.config)
node2 will listen on port 8082 (override in vars_node2.config)
node3 will listen on port 8083 (override in vars_node3.config)
To avoid having to configure this in sys.config we will define a cuttlefish schema in config.schema that cuttlefish will use to generate a default config file and validation code for us.
We have to replace the variables from variable overrides in our config.schema file for each release before it's processed by cuttlefish itself, for that we use the template directive on an overlay section on the release config.
Build devrel:
Check the configuration file generated for each node at:
_build/node1/rel/cadena/etc/cadena.conf _build/node2/rel/cadena/etc/cadena.conf _build/node3/rel/cadena/etc/cadena.conf
The first part is of interest to us, it looks like this for node1, the port number is different in node2 and node3:
## port to listen to for HTTP API ## ## Default: 8081 ## ## Acceptable values: ## - an integer http.port = 8081 ## number of acceptors to user for HTTP API ## ## Default: 100 ## ## Acceptable values: ## - an integer http.acceptors = 100 ## folder where ensemble data is stored ## ## Default: ./cadena_data ## ## Acceptable values: ## - text data.dir = ./cadena_data
Start 3 nodes in 3 different shells:
Start enseble and join nodes, I created a target called devrel-setup in the Makefile to make it easier:
Let's set key1 in ensemble root to 42 on node1 (port 8081):
Response:
Let's get key1 in ensemble root to 42 on node2 (port 8082):
Response:
Same on node3:
Response:
Overwrite on node1:
Response:
Get on node2:
Let's set key2 in ensemble root to {"number": 42} on node1 (port 8081):
Response:
Get it on node2:
Response:
Delete key2 in ensemble root on node2:
Response:
Check that it was removed by trying to get it again on node2:
Response:
There you go, now you have a Consistent Key Value Store with an HTTP API.