Serving Static Files with Erlang, Cowboy and Rebar, (raw material)
the "serving static files from a server" market needs some disruption, let's tackle that problem with some erlang.
create the folder:
mkdir disrupt cd disrupt
get rebar:
wget http://cloud.github.com/downloads/basho/rebar/rebar chmod u+x rebar
generate app:
./rebar create-app appid=disrupt
add dependencies:
vim rebar.config
the file should contain the following code:
this tells to get cowboy 1.0.0 from git as dependency, let's fetch the dependencies:
./rebar get-deps
now let's serve some static files, open src/disrupt_app.erl, change the start function so it looks like this:
start(_StartType, _StartArgs) -> Dispatch = cowboy_router:compile([ {'_', [ {"/[...]", cowboy_static, {priv_dir, disrupt, "assets", [{mimetypes, cow_mimetypes, all}]}} ]} ]), {ok, _} = cowboy:start_http(http, 100, [{port, 8080}], [ {env, [{dispatch, Dispatch}]} ]), disrupt_sup:start_link().
the code above is taken from the static example from cowboy adapted to our needs: https://github.com/ninenines/cowboy/tree/master/examples/static_world
docs here: http://ninenines.eu/docs/en/cowboy/1.0/guide/static_handlers/
now make the priv/assets folder:
mkdir -p priv/assets
and put some content in the index file:
echo "hello static world" > priv/assets/index.html
let's make a release for the project, first create the release files:
mkdir rel cd rel ../rebar create-node nodeid=disrupt cd ..
edit rel/reltool.config, the line:
{lib_dirs, []},
should change to:
{lib_dirs, ["../deps"]},
add the line:
sub_dirs, ["rel"]}.
to the top of the rebar.config file
if not you get this error:
Command 'generate' not understood or not applicable
yay helpfulness!
now we are one confusing error closer to our goal, but now we get:
ERROR: generate failed while processing /home/mariano/tmp/disrupt/rel: {'EXIT',{{badmatch,{error,"disrupt: Missing application directory."}}, [{rebar_reltool,generate,2,[]}, {rebar_core,run_modules,4,[]}, {rebar_core,execute,5,[]}, {rebar_core,process_dir0,6,[]}, {rebar_core,process_dir,4,[]}, {rebar_core,process_each,5,[]}, {rebar_core,process_dir0,6,[]}, {rebar_core,process_dir,4,[]}]}}
only because I've seen some other erlang projects I decided that this may be the solution:
mkdir -p apps/disrupt cd apps/disrupt ln -s ../../src ln -s ../../ebin ln -s ../../priv cd ../..
we try again to make the release and we get the error again, nice!
for no apparent reason that the fact that I saw it on some other project, change the line on rel/reltool.config:
{lib_dirs, ["../deps"]},
to:
{lib_dirs, ["../deps", "../apps"]},
trying again:
./rebar compile generate
success!
let's try running it:
./rel/disrupt/bin/disrupt console
and what do we get? well, a crash! \o/:
Exec: /home/mariano/tmp/disrupt/rel/disrupt/erts-5.10.4/bin/erlexec -boot /home/mariano/tmp/disrupt/rel/disrupt/releases/1/disrupt -mode embedded -config /home/mariano/tmp/disrupt/rel/disrupt/releases/1/sys.config -args_file /home/mariano/tmp/disrupt/rel/disrupt/releases/1/vm.args -- console Root: /home/mariano/tmp/disrupt/rel/disrupt Erlang R16B03 (erts-5.10.4) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.4 (abort with ^G) (disrupt@127.0.0.1)1> =INFO REPORT==== 29-Aug-2014::11:49:32 === application: disrupt exited: {bad_return, {{disrupt_app,start,[normal,[]]}, {'EXIT', {undef, [{cowboy_router,compile, [[{'_', [{"/[...]",cowboy_static, {priv_dir,disrupt,"assets", [{mimetypes,cow_mimetypes,all}]}}]}]], []}, {disrupt_app,start,2, [{file,"src/disrupt_app.erl"},{line,13}]}, {application_master,start_it_old,4, [{file,"application_master.erl"},{line,269}]}]}}}} type: permanent {"Kernel pid terminated",application_controller,"{application_start_failure,disrupt,{bad_return,{{disrupt_app,start,[normal,[]]},{'EXIT',{undef,[{cowboy_router,compile,[[{'_',[{\"/[...]\",cowboy_static,{priv_dir,disrupt,\"assets\",[{mimetypes,cow_mimetypes,all}]}}]}]],[]},{disrupt_app,start,2,[{file,\"src/disrupt_app.erl\"},{line,13}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,269}]}]}}}}}"} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) ({application_start_failure,disrupt,{bad_return,{{disrupt_app,start,[normal,[]]},{'EXIT',{undef,[{cowboy_router,compile,[[{'_',[{"/[...]",cowboy_static
with all my time reading erlang crashes I see an undef there, it seems the app cant find cowboy_router:compile
but I have it on my deps..
well, let's put it on src/disrupt.app.src applications for no other reason that I've seen people do that, the file should look like this:
{application, disrupt, [ {description, ""}, {vsn, "1"}, {registered, []}, {applications, [ kernel, stdlib, cowboy ]}, {mod, { disrupt_app, []}}, {env, []} ]}.
see the cowboy as last element on the applications list? that's what you should add.
let's try again:
./rebar compile generate
and I get:
<some output removed here> ==> rel (generate) ERROR: generate failed while processing /home/mariano/tmp/disrupt/rel: {'EXIT',{{badmatch,{error,"Application cowboy is used in release \"disrupt\" and cannot be excluded"}}, [{rebar_reltool,generate,2,[]}, {rebar_core,run_modules,4,[]}, {rebar_core,execute,5,[]}, {rebar_core,process_dir0,6,[]}, {rebar_core,process_dir,4,[]}, {rebar_core,process_each,5,[]}, {rebar_core,process_dir0,6,[]}, {rebar_core,process_dir,4,[]}]}}
"Application cowboy is used in release "disrupt" and cannot be excluded"
who told you to exclude it?
let's apply a technic we already used a lot before, let's make up reasons for what it may be failing, it says exclude there and I've seen a lot of exclude and include atoms in rel/reltool.config, maybe it's that?
let's layer some other technic I use a lot, let's try to make that file look as similar as another one I've seen that works, in this case it's the reltool.config file generated by riak_core rebar template:
https://github.com/basho/rebar_riak_core/blob/master/riak_core.reltool.config
ok, that one seems to have less stuff than ours, let's start commenting everything that looks different until we reach this point:
{sys, [ {lib_dirs, ["../deps", "../apps"]}, %{erts, [{mod_cond, derived}, {app_file, strip}]}, %{app_file, strip}, {rel, "disrupt", "1", [ kernel, stdlib, sasl, disrupt ]}, {rel, "start_clean", "", [ kernel, stdlib ]}, {boot_rel, "disrupt"}, {profile, embedded}, %{incl_cond, exclude}, {excl_archive_filters, [".*"]}, %% Do not archive built libs {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)", "^erts.*/(doc|info|include|lib|man|src)"]}, {excl_app_filters, ["\.gitignore"]}, {app, sasl, [{incl_cond, include}]}, %{app, stdlib, [{incl_cond, include}]}, %{app, kernel, [{incl_cond, include}]}, {app, disrupt, [{incl_cond, include}]} ]}. {target_dir, "disrupt"}. {overlay, [ {mkdir, "log/sasl"}, {copy, "files/erl", "\{\{erts_vsn\}\}/bin/erl"}, {copy, "files/nodetool", "\{\{erts_vsn\}\}/bin/nodetool"}, {copy, "files/disrupt", "bin/disrupt"}, {copy, "files/disrupt.cmd", "bin/disrupt.cmd"}, {copy, "files/start_erl.cmd", "bin/start_erl.cmd"}, {copy, "files/install_upgrade.escript", "bin/install_upgrade.escript"}, {copy, "files/sys.config", "releases/\{\{rel_vsn\}\}/sys.config"}, {copy, "files/vm.args", "releases/\{\{rel_vsn\}\}/vm.args"} ]}.
trying again:
rm -rf rel/disrupt ./rebar compile generate
success!
now let's try running it:
./rel/disrupt/bin/disrupt console
the output we get:
Exec: /home/mariano/tmp/disrupt/rel/disrupt/erts-5.10.4/bin/erlexec -boot /home/mariano/tmp/disrupt/rel/disrupt/releases/1/disrupt -mode embedded -config /home/mariano/tmp/disrupt/rel/disrupt/releases/1/sys.config -args_file /home/mariano/tmp/disrupt/rel/disrupt/releases/1/vm.args -- console Root: /home/mariano/tmp/disrupt/rel/disrupt Erlang R16B03 (erts-5.10.4) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.4 (abort with ^G) (disrupt@127.0.0.1)1>
that's what I would call a non crashing system, let's see if it works:
curl http://localhost:8080/index.html
we get:
hello static world
success!
and that's how you serve a static file with erlang, cowboy and rebar :)