Serving Static Files with Erlang, Cowboy and Rebar, (raw material)

Mariano Guerra

2014-08-29 10:09

the "serving static files from a server" market needs some disruption, let's tackle that problem with some erlang.

create the folder:

mkdir disrupt
cd disrupt

get rebar:

wget http://cloud.github.com/downloads/basho/rebar/rebar
chmod u+x rebar

generate app:

./rebar create-app appid=disrupt

add dependencies:

vim rebar.config

the file should contain the following code:

{deps, [
    {cowboy, "1.0.0", {git, "https://github.com/ninenines/cowboy", {tag, "1.0.0"}}}
]}.

this tells to get cowboy 1.0.0 from git as dependency, let's fetch the dependencies:

./rebar get-deps

now let's serve some static files, open src/disrupt_app.erl, change the start function so it looks like this:

start(_StartType, _StartArgs) ->
    Dispatch = cowboy_router:compile([
        {'_', [
            {"/[...]", cowboy_static, {priv_dir, disrupt, "assets",
                [{mimetypes, cow_mimetypes, all}]}}
        ]}
    ]),
    {ok, _} = cowboy:start_http(http, 100, [{port, 8080}], [
        {env, [{dispatch, Dispatch}]}
    ]),
    disrupt_sup:start_link().

the code above is taken from the static example from cowboy adapted to our needs: https://github.com/ninenines/cowboy/tree/master/examples/static_world

docs here: http://ninenines.eu/docs/en/cowboy/1.0/guide/static_handlers/

now make the priv/assets folder:

mkdir -p priv/assets

and put some content in the index file:

echo "hello static world" > priv/assets/index.html

let's make a release for the project, first create the release files:

mkdir rel
cd rel
../rebar create-node nodeid=disrupt
cd ..

edit rel/reltool.config, the line:

{lib_dirs, []},

should change to:

{lib_dirs, ["../deps"]},

add the line:

sub_dirs, ["rel"]}.

to the top of the rebar.config file

if not you get this error:

Command 'generate' not understood or not applicable

yay helpfulness!

now we are one confusing error closer to our goal, but now we get:

ERROR: generate failed while processing /home/mariano/tmp/disrupt/rel: {'EXIT',{{badmatch,{error,"disrupt: Missing application directory."}},
         [{rebar_reltool,generate,2,[]},
          {rebar_core,run_modules,4,[]},
          {rebar_core,execute,5,[]},
          {rebar_core,process_dir0,6,[]},
          {rebar_core,process_dir,4,[]},
          {rebar_core,process_each,5,[]},
          {rebar_core,process_dir0,6,[]},
          {rebar_core,process_dir,4,[]}]}}

only because I've seen some other erlang projects I decided that this may be the solution:

mkdir -p apps/disrupt
cd apps/disrupt
ln -s ../../src
ln -s ../../ebin
ln -s ../../priv
cd ../..

we try again to make the release and we get the error again, nice!

for no apparent reason that the fact that I saw it on some other project, change the line on rel/reltool.config:

{lib_dirs, ["../deps"]},

to:

{lib_dirs, ["../deps", "../apps"]},

trying again:

./rebar compile generate

success!

let's try running it:

./rel/disrupt/bin/disrupt console

and what do we get? well, a crash! \o/:

Exec: /home/mariano/tmp/disrupt/rel/disrupt/erts-5.10.4/bin/erlexec -boot /home/mariano/tmp/disrupt/rel/disrupt/releases/1/disrupt -mode embedded -config /home/mariano/tmp/disrupt/rel/disrupt/releases/1/sys.config -args_file /home/mariano/tmp/disrupt/rel/disrupt/releases/1/vm.args -- console
Root: /home/mariano/tmp/disrupt/rel/disrupt
Erlang R16B03 (erts-5.10.4) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.10.4  (abort with ^G)
(disrupt@127.0.0.1)1>
=INFO REPORT==== 29-Aug-2014::11:49:32 ===
    application: disrupt
    exited: {bad_return,
             {{disrupt_app,start,[normal,[]]},
              {'EXIT',
               {undef,
                [{cowboy_router,compile,
                  [[{'_',
                     [{"/[...]",cowboy_static,
                       {priv_dir,disrupt,"assets",
                        [{mimetypes,cow_mimetypes,all}]}}]}]],
                  []},
                 {disrupt_app,start,2,
                  [{file,"src/disrupt_app.erl"},{line,13}]},
                 {application_master,start_it_old,4,
                  [{file,"application_master.erl"},{line,269}]}]}}}}
    type: permanent
{"Kernel pid terminated",application_controller,"{application_start_failure,disrupt,{bad_return,{{disrupt_app,start,[normal,[]]},{'EXIT',{undef,[{cowboy_router,compile,[[{'_',[{\"/[...]\",cowboy_static,{priv_dir,disrupt,\"assets\",[{mimetypes,cow_mimetypes,all}]}}]}]],[]},{disrupt_app,start,2,[{file,\"src/disrupt_app.erl\"},{line,13}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,269}]}]}}}}}"}

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,disrupt,{bad_return,{{disrupt_app,start,[normal,[]]},{'EXIT',{undef,[{cowboy_router,compile,[[{'_',[{"/[...]",cowboy_static

with all my time reading erlang crashes I see an undef there, it seems the app cant find cowboy_router:compile

but I have it on my deps..

well, let's put it on src/disrupt.app.src applications for no other reason that I've seen people do that, the file should look like this:

{application, disrupt,
 [
  {description, ""},
  {vsn, "1"},
  {registered, []},
  {applications, [
                  kernel,
                  stdlib,
                  cowboy
                 ]},
  {mod, { disrupt_app, []}},
  {env, []}
 ]}.

see the cowboy as last element on the applications list? that's what you should add.

let's try again:

./rebar compile generate

and I get:

<some output removed here>

==> rel (generate)
ERROR: generate failed while processing /home/mariano/tmp/disrupt/rel: {'EXIT',{{badmatch,{error,"Application cowboy is used in release \"disrupt\" and cannot be excluded"}},
         [{rebar_reltool,generate,2,[]},
          {rebar_core,run_modules,4,[]},
          {rebar_core,execute,5,[]},
          {rebar_core,process_dir0,6,[]},
          {rebar_core,process_dir,4,[]},
          {rebar_core,process_each,5,[]},
          {rebar_core,process_dir0,6,[]},
          {rebar_core,process_dir,4,[]}]}}

"Application cowboy is used in release "disrupt" and cannot be excluded"

who told you to exclude it?

let's apply a technic we already used a lot before, let's make up reasons for what it may be failing, it says exclude there and I've seen a lot of exclude and include atoms in rel/reltool.config, maybe it's that?

let's layer some other technic I use a lot, let's try to make that file look as similar as another one I've seen that works, in this case it's the reltool.config file generated by riak_core rebar template:

https://github.com/basho/rebar_riak_core/blob/master/riak_core.reltool.config

ok, that one seems to have less stuff than ours, let's start commenting everything that looks different until we reach this point:

{sys, [
    {lib_dirs, ["../deps", "../apps"]},
        %{erts, [{mod_cond, derived}, {app_file, strip}]},
        %{app_file, strip},
        {rel, "disrupt", "1",
            [
                kernel,
            stdlib,
            sasl,
            disrupt
                ]},
        {rel, "start_clean", "",
            [
                kernel,
            stdlib
                ]},
        {boot_rel, "disrupt"},
        {profile, embedded},
        %{incl_cond, exclude},
        {excl_archive_filters, [".*"]}, %% Do not archive built libs
    {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)",
        "^erts.*/(doc|info|include|lib|man|src)"]},
        {excl_app_filters, ["\.gitignore"]},
        {app, sasl,   [{incl_cond, include}]},
        %{app, stdlib, [{incl_cond, include}]},
        %{app, kernel, [{incl_cond, include}]},
        {app, disrupt, [{incl_cond, include}]}
    ]}.

{target_dir, "disrupt"}.

{overlay, [
    {mkdir, "log/sasl"},
        {copy, "files/erl", "\{\{erts_vsn\}\}/bin/erl"},
        {copy, "files/nodetool", "\{\{erts_vsn\}\}/bin/nodetool"},
        {copy, "files/disrupt", "bin/disrupt"},
        {copy, "files/disrupt.cmd", "bin/disrupt.cmd"},
        {copy, "files/start_erl.cmd", "bin/start_erl.cmd"},
        {copy, "files/install_upgrade.escript", "bin/install_upgrade.escript"},
        {copy, "files/sys.config", "releases/\{\{rel_vsn\}\}/sys.config"},
        {copy, "files/vm.args", "releases/\{\{rel_vsn\}\}/vm.args"}
    ]}.

trying again:

rm -rf rel/disrupt
./rebar compile generate

success!

now let's try running it:

./rel/disrupt/bin/disrupt console

the output we get:

Exec: /home/mariano/tmp/disrupt/rel/disrupt/erts-5.10.4/bin/erlexec -boot /home/mariano/tmp/disrupt/rel/disrupt/releases/1/disrupt -mode embedded -config /home/mariano/tmp/disrupt/rel/disrupt/releases/1/sys.config -args_file /home/mariano/tmp/disrupt/rel/disrupt/releases/1/vm.args -- console
Root: /home/mariano/tmp/disrupt/rel/disrupt
Erlang R16B03 (erts-5.10.4) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.10.4  (abort with ^G)
(disrupt@127.0.0.1)1>

that's what I would call a non crashing system, let's see if it works:

curl http://localhost:8080/index.html

we get:

hello static world

success!

and that's how you serve a static file with erlang, cowboy and rebar :)

Mariano Guerra's Log

Comments