Mariano Guerra's Log (Publicaciones sobre riak_core)

Riak Core Tutorial Part 2: Starting

Mariano Guerra — Thu, 03 May 2018 12:48:26 GMT

This is part of a series, see the previous one at Riak Core Tutorial Part 1: Setup

Creating our Project

Now that we have our tools and our template installed we can start by asking rebar3 to create a new project we will call tanodb using the riak_core template we just installed:

rebar3 new rebar3_riak_core name=tanodb

If it fails saying it can't find rebar3 check that it's in your $PATH environment variable and that you have Erlang activated.

The output should be something like this:

===> Writing tanodb/apps/tanodb/src/tanodb.app.src
===> Writing tanodb/apps/tanodb/src/tanodb.erl
===> Writing tanodb/apps/tanodb/src/tanodb_app.erl
===> Writing tanodb/apps/tanodb/src/tanodb_sup.erl
===> Writing tanodb/apps/tanodb/src/tanodb_console.erl
===> Writing tanodb/apps/tanodb/src/tanodb_vnode.erl
===> Writing tanodb/rebar.config
===> Writing tanodb/.editorconfig
===> Writing tanodb/.gitignore
===> Writing tanodb/README.rst
===> Writing tanodb/Makefile
===> Writing tanodb/config/admin_bin
===> Writing tanodb/priv/01-tanodb.schema
===> Writing tanodb/config/advanced.config
===> Writing tanodb/config/vars.config
===> Writing tanodb/config/vars_dev1.config
===> Writing tanodb/config/vars_dev2.config
===> Writing tanodb/config/vars_dev3.config

Building and Running

Before explaining what the files mean so you get an idea what just happened let's run it!

cd tanodb
make
make console

make runs rebar3 commands to build a release of our project, for that it uses a tool called relx.

The initial build may take a while since it has to fetch all the dependencies and build them.

After the release is built (you can check the result by inspecting the folder _build/default/rel/tanodb/) we can run it.

When we run make console we get some noisy output that should end with something like this:

Eshell V9.3  (abort with ^G)
(tanodb@127.0.0.1)1>

This is the Erlang shell, something like a REPL connected to our system, we now can test our system by calling tanodb:ping() on it.

(tanodb@127.0.0.1)1> tanodb:ping().
{pong,1347321821914426127719021955160323408745312813056}

The response is the atom pong and a huge number that we will explain later, but to make it short, it's the id of the process that replied to us.

Exploring the Template Files

The template created a lot of files and if you are like me, you don't like things that make magic and don't explain what's going on, that's why we will get a brief overview of the files created here.

First this files are created:

apps/tanodb/src/tanodb.app.src
apps/tanodb/src/tanodb.erl
apps/tanodb/src/tanodb_app.erl
apps/tanodb/src/tanodb_sup.erl
apps/tanodb/src/tanodb_console.erl
apps/tanodb/src/tanodb_vnode.erl

Those are the meat of this project, the source code we start with, if you know a little of erlang you will recognice many of them, let's explain them briefly, if you think you need more information I recommend you this awesome book which you can read online: Learn You Some Erlang for great good!

tanodb.app.src: This file is "The Application Resource File", you can read it, it's quite self descriptive. You can read more about it in the Building OTP Applications Section of Learn You Some Erlang or in the man page for app in the Erlang documentation.
tanodb.erl: This file is the main API of our application, here we expose all the things you can ask our application to do, for now it can only handle the ping() command but we will add some more in the future.
tanodb_app.erl: This file implements the application behavior it's a set of callbacks that the Erlang runtime calls to start and stop our application.
tanodb_sup.erl: This file implements the supervisor behavior it's a set of callbacks that the Erlang runtime calls to build the supervisor hierarchy.
tanodb_console.erl: This file is specific to riak_core, it's a set of callbacks that will be called by the tanodb-admin command.
tanodb_vnode.erl: This file is specific to riak_core, it implements the riak_code_vnode behavior, which is a set of callbacks that riak_core will call to accomplish different tasks, it's the main file we will edit to add new features.

Those were the source code files, but the template also created other files, let's review them

rebar.config: This is the file that rebar3 reads to get information about our project like dependencies and build configuration, you can read more about it on the rebar3 documentation
.editorconfig: This file describes the coding style for this project, if your text editor understands editorconfig files then it will change it's setting for this project to the ones described in this file, read more about editor config on the editorconfig website
.gitignore: A file to tell git which files to ignore from the repository.
README.rst: The README of the project
Makefile: A make file with some targets that will make it easier to achieve some complex tasks without copying and pasting too much.
config/admin_bin: A template for the tanodb-admin command.
priv/01-tanodb.schema: The cuttlefish schema file that describes what configuration our application supports, it starts with some example configuration fields that we will use as the application grows.
config/advanced.config: This file is where we configure some advanced things of our application that don't go on our tanodb.config file, here we configure riak_core and our logging library
config/vars.config: This file contains variables used by relx to build a release, you can read more about it in the rebar3 release documentation

The following files are like vars.config but with slight differences to allow running more than one node on the same machine:

config/vars_dev1.config
config/vars_dev2.config
config/vars_dev3.config

Normally when you have a cluster for your application one operating system instance runs one instance of your application and you have many operating system instances, but to test the clustering features of riak_core we will build 3 releases of our application using offsets for ports and changing the application name to avoid collisions.

Those are all the files, follow the links to know more about them.

Playing with Clustering

Before starting to add features, let's first play with clustering so we understand all those config files above work.

Build 3 releases that can run on the same machine:

make devrel

This will build 3 releases of the application using different parameters (the dev1, dev2 and dev3 files we saw earlier) and will place them under:

_build/dev1
_build/dev2
_build/dev3

This is achived by using the profiles feature from rebar3.

Now open 3 consoles and run the following commands one on each console:

make dev1-console
make dev2-console
make dev3-console

This will start the 3 nodes but they won't know about each other, for them to know about each other we need to "join" them, that is to tell one of them about the other two, this is achieved using the tanodb-admin command, here is how you should run it manually (don't run them):

_build/dev2/rel/tanodb/bin/tanodb-admin cluster join tanodb1@127.0.0.1
_build/dev3/rel/tanodb/bin/tanodb-admin cluster join tanodb1@127.0.0.1

We tell dev2 and dev3 to join tanodb1 (dev1), to make this easier and less error prone run the following command:

make devrel-join

Now let's check the status of the cluster:

make devrel-status

You can read the Makefile to get an idea of what those commands do, in this case devrel-status does the following:

_build/dev1/rel/tanodb/bin/tanodb-admin member-status

You should see something like this:

================================= Membership ===============
Status     Ring    Pending    Node
------------------------------------------------------------
joining     0.0%      --      'tanodb2@127.0.0.1'
joining     0.0%      --      'tanodb3@127.0.0.1'
valid     100.0%      --      'tanodb1@127.0.0.1'
------------------------------------------------------------
Valid:1 / Leaving:0 / Exiting:0 / Joining:2 / Down:0

It should say that 3 nodes are joining, now check the cluster plan:

make devrel-cluster-plan

The output should be something like this:

=============================== Staged Changes ==============
Action         Details(s)
-------------------------------------------------------------
join           'tanodb2@127.0.0.1'
join           'tanodb3@127.0.0.1'
-------------------------------------------------------------


NOTE: Applying these changes will result in 1 cluster transition

#############################################################
                         After cluster transition 1/1
#############################################################

================================= Membership ================
Status     Ring    Pending    Node
-------------------------------------------------------------
valid     100.0%     34.4%    'tanodb1@127.0.0.1'
valid       0.0%     32.8%    'tanodb2@127.0.0.1'
valid       0.0%     32.8%    'tanodb3@127.0.0.1'
-------------------------------------------------------------
Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

WARNING: Not all replicas will be on distinct nodes

Transfers resulting from cluster changes: 42
  21 transfers from 'tanodb1@127.0.0.1' to 'tanodb3@127.0.0.1'
  21 transfers from 'tanodb1@127.0.0.1' to 'tanodb2@127.0.0.1'

Now we can commit the plan:

make devrel-cluster-commit

Which should say something like:

Cluster changes committed

Now riak_core started an internal process to join the nodes to the cluster, this involve some complex processes that we will explore in the following chapters.

You should see on the consoles where the nodes are running that some logging is happening describing the process.

Check the status of the cluster again:

make devrel-status

You can see the vnodes transfering, this means the content of some virtual nodes on one tanodb node are being transferred to another tanodb node:

================================= Membership =============
Status     Ring    Pending    Node
----------------------------------------------------------
valid      75.0%     34.4%    'tanodb1@127.0.0.1'
valid       9.4%     32.8%    'tanodb2@127.0.0.1'
valid       7.8%     32.8%    'tanodb3@127.0.0.1'
----------------------------------------------------------
Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

At some point you should see something like this, which means that the nodes are joined and balanced:

================================= Membership ==============
Status     Ring    Pending    Node
-----------------------------------------------------------
valid      34.4%      --      'tanodb1@127.0.0.1'
valid      32.8%      --      'tanodb2@127.0.0.1'
valid      32.8%      --      'tanodb3@127.0.0.1'
-----------------------------------------------------------
Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

When you are bored you can stop them:

make devrel-stop

Building a Production Release

Even when our application doesn't have the features to merit a production release we are going to learn how to do it here since you can later do it at any step and get a full release of the app:

make prod-release

In that command rebar3 runs the release task using the prod profile, which has some configuration differences with the dev profiles we use so that it builds something we can unpack and run on another operating system without installing anything.

Let's package our release:

cd _build/prod/rel
tar -czf tanodb.tgz tanodb tanodb_config
cd -
mv _build/prod/rel/tanodb.tgz .

You can copy it to a clean OS and run:

tar -xzf tanodb.tgz
./tanodb/bin/tanodb console

And it runs!

Wrapping Up

Now you know how to create a riak_core app from a template, how to build a release and run it, how to build releases for a development cluster, run the nodes, join them and inspect the cluster status and how to build a production release and run it on a fresh server.

Riak Core Tutorial Part 1: Setup

Mariano Guerra — Thu, 03 May 2018 12:32:23 GMT

This is the first chapter in a series that will explore how to use riak_core to build a distributed key-value store.

We first need to have Erlang installed and rebar3 installed.

The Easy Way

If you don't have Erlang installed or you don't have problem to install the latest one system wide you can try installing it with your package manager:

For Homebrew on OS X: brew install erlang
For MacPorts on OS X: port install erlang
For Ubuntu and Debian: apt-get install erlang
For Fedora: yum install erlang
For FreeBSD: pkg install erlang

Please check that the package version is 20.x, if not, check for instructions on how to install the Erlang Solutions package for Ubuntu, CentOS, Mac OS X, Debian or Fedora here: https://www.erlang-solutions.com/resources/download.html

Setting up rebar3

Now we have Erlang, we need a build tool, we are going to use rebar3:

# download rebar3 to our bin directory
wget https://s3.amazonaws.com/rebar3/rebar3 -O $HOME/bin/rebar3

# set execution permissions for our user
chmod u+x rebar3

Just in case you have problems running the rebar3 commands with a different version, here's the version I'm using:

rebar3 version

Output:

rebar 3.5.0 on Erlang/OTP 20 Erts 9.2

Install Riak Core Rebar3 Template

To create a Riak Core project from scratch we will use a template called rebar3_template_riak_core.

we need to clone its repo in a place where rebar3 can see it:

mkdir -p ~/.config/rebar3/templates
git clone https://github.com/marianoguerra/rebar3_template_riak_core.git ~/.config/rebar3/templates/rebar3_template_riak_core

The Hard Way: Building Erlang with kerl

If you have or want to have more than one version running side by side you can use kerl, from its github README:

Easy building and installing of Erlang/OTP instances.

Kerl aims to be shell agnostic and its only dependencies, excluding what's
required to actually build Erlang/OTP, are curl and git.

First we need to fetch kerl:

# create bin folder in our home directory if it's not already there
mkdir -p ~/bin

# cd to it
cd ~/bin

# download kerl script
curl -O https://raw.githubusercontent.com/kerl/kerl/master/kerl

# set execution permitions for our user
chmod u+x kerl

You will need to add ~/bin to your PATH variable so your shell can find the kerl script, you can do it like this in your shell:

# set the PATH environment variable to the value it had before plus a colon
# (path separator) and a new path which points to the bin folder we just
# created
PATH=$PATH:$HOME/bin

If you want to make this work every time you start a shell you need to put it it the rc file of your shell of choice, for bash it's ~/.bashrc, for zsh it's ~/.zshrc, you will have to add a line like this:

export PATH=$PATH:$HOME/bin

After this, start a new shell or source your rc file so that it picks up your new PATH variable, you can check that it's set correctly by running:

echo $PATH

And checking in the output if $HOME/bin is there (notice that $HOME will be expanded to the actual path).

Building an Erlang release with kerl

We have kerl installed and available in our shell, now we need to build an Erlang release of our choice, for this we will need a compiler and other tools and libraries needed to compile it:

This are instructions on ubuntu 17.10, check the names for those packages on your distribution.

# required: basic tools and libraries needed
# (compiler, curses for the shell, ssl for crypto)
sudo apt-get -y install build-essential m4 libncurses5-dev libssl-dev

# optonal: if you want odbc support (database connectivity)
sudo apt-get install unixodbc-dev

# optonal: if you want pdf docs you need apache fop and xslt tools and java (fop is a java project)
sudo apt-get install -y fop xsltproc default-jdk

# optional: if you want to build jinterface you need a JDK
sudo apt-get install -y default-jdk

# optional: if you want wx (desktop GUI modules)
sudo apt-get install -y libwxgtk3.0-dev

Now that we have everything we need we can finally build our Erlang release.

First we fetch an updated list of releases:

kerl update releases

The output in my case:

The available releases are:

R10B-0 R10B-10 R10B-1a R10B-2 R10B-3 R10B-4 R10B-5 R10B-6 R10B-7 R10B-8
R10B-9 R11B-0 R11B-1 R11B-2 R11B-3 R11B-4 R11B-5 R12B-0 R12B-1 R12B-2 R12B-3
R12B-4 R12B-5 R13A R13B01 R13B02-1 R13B02 R13B03 R13B04 R13B R14A R14B01
R14B02 R14B03 R14B04 R14B_erts-5.8.1.1 R14B R15B01 R15B02
R15B02_with_MSVCR100_installer_fix R15B03-1 R15B03 R15B
R16A_RELEASE_CANDIDATE R16B01 R16B02 R16B03-1 R16B03 R16B 17.0-rc1 17.0-rc2
17.0 17.1 17.3 17.4 17.5 18.0 18.1 18.2.1 18.2 18.3 19.0 19.1 19.2 19.3
20.0 20.1 20.2 20.3

Let's build the 20.3 version:

# this will take a while
kerl build 20.3 20.3

And install it:

kerl install 20.3 ~/bin/erl-20.3

Now everytime we want to use this version of Erlang we need to run:

. $HOME/bin/erl-20.3/activate

Test that Everything Works

We have installed several tools:

kerl: Let's you install multiple Erlang releases that can live side by side
Erlang 20.3: The version of erlang we are going to be using
Rebar 3: Our build tool
rebar3_template_riak_core: Rebar 3 Template that will make it easy to setup fresh riak_core projects for experimentation

Now we need to check that everything is setup correctly, we will do that by creating a template and building it.

Remember to have $HOME/bin in your $PATH and Erlang 20.3 activated, cd to a folder of your choice to hold this project and run:

rebar3 new rebar3_riak_core name=akv

Output should be similar to this one:

===> Writing akv/apps/akv/src/akv.app.src
===> Writing akv/apps/akv/src/akv.erl
===> Writing akv/apps/akv/src/akv_app.erl
===> Writing akv/apps/akv/src/akv_sup.erl
===> Writing akv/apps/akv/src/akv_console.erl
===> Writing akv/apps/akv/src/akv_vnode.erl
===> Writing akv/rebar.config
===> Writing akv/.editorconfig
===> Writing akv/.gitignore
===> Writing akv/README.rst
===> Writing akv/Makefile
===> Writing akv/config/admin_bin
===> Writing akv/priv/01-akv.schema
===> Writing akv/config/advanced.config
===> Writing akv/config/vars.config
===> Writing akv/config/vars_dev1.config
===> Writing akv/config/vars_dev2.config
===> Writing akv/config/vars_dev3.config

Now let's try to build it:

cd akv
make

Output is to long to list, after it ends, near the end you should see this line:

===> release successfully created!

Now let's try to run it:

make console

Last lines should be:

Eshell V9.3  (abort with ^G)
(akv@127.0.0.1)1>

You can exit with q(). and pressing enter or hitting Ctrl-C twice.

We're ready to start!