How BEAMs are made

Mariano Guerra

Elixir Karlsruhe Meetup

March 2016

Ego Slide

Start the project

gh:marianoguerra/otl to follow each step as a commit

commit 1:

rebar3 new fn_escript name=otl

cd otl

rebar3 escriptize

./_build/default/bin/otl

Implement erl2ast command

Because reading absform is too much work :)

commit 2:

$ ./_build/default/bin/otl erl2ast examples/mymod.erl

{ok,[{attribute,1,file,{"examples/mymod.erl",1}},
     {attribute,1,module,mymod},
     {attribute,2,export,[{main,0}]},
     {function,4,main,0,[{clause,4,[],[],[{atom,4,true}]}]},
     {eof,5}]}

Implement erl2ast2erl

commit 3:

$ ./_build/default/bin/otl erl2ast2erl examples/mymod.erl

-file("examples/mymod.erl", 1).

-module(mymod).

-export([main/0]).

main() -> true.

Implement erl2beam

commit 4:

$ ./_build/default/bin/otl erl2beam examples/mymod.erl .

{ok,[{module_name,mymod},{path,"./mymod.beam"}]}

Implement erl2beam

$ erl

Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10]
[kernel-poll:false]

Eshell V7.0  (abort with ^G)
1> mymod:main().
true
2> q().
ok

Tokenize true token, skip whites

commit 5:

$ cat examples/trues.otl

true true true


$ ./_build/default/bin/otl lex examples/trues.otl

{ok,[{boolean,1,true},{boolean,1,true},{boolean,1,true}],2}

Parse true token to AST

commit 6:

$ ./_build/default/bin/otl ast examples/trues.otl

{ok,[{atom,1,true},{atom,1,true},{atom,1,true}]}

Build Module with generated AST

commit 7:

$ ./_build/default/bin/otl mod examples/trues.otl mymod myfun

{ok,[{attribute,1,file,{"examples/trues.otl",1}},
     {attribute,1,module,mymod},
     {attribute,2,export,[{myfun,0}]},
     {function,3,myfun,0,
               [{clause,4,[],[],
                        [{atom,1,true},{atom,1,true},{atom,1,true}]}]}]}

Compile to BEAM

commit 8:

$ ./_build/default/bin/otl beam examples/trues.otl . mymod myfun

{ok,[{module_name,mymod},{path,"./mymod.beam"}]}

Compile to Erlang Source

commit 9:

$ ./_build/default/bin/otl erl examples/trues.otl mymod myfun

-file("examples/trues.otl", 1).
-module(mymod).
-export([myfun/0]).

myfun() -> true, true, true.

But I don't like your language!

rebar3 efene compile --format=erl

Add eval

commit 10:

$ ./_build/default/bin/otl eval examples/trues.otl

true

Implement a repl

commit 11:

$ ./otl-shell
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]

otl shell (Ctrl+g and then q to quit, Ctrl+g for Job Control Mode)

>>> true
true
>>> true true
true
>>>
>>> false
error: {1,otl_lexer,{illegal,"f"}}
>>> ^G
User switch command
 --> q

Parse Numbers Containing 1

commit 12:

$ ./otl-shell
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]

otl shell (Ctrl+g and then q to quit, Ctrl+g for Job Control Mode)

>>> 1
1
>>> 1 true
true
>>> 1 true 1111
1111
>>> ^G
User switch command
 --> q

Parse Numbers Containing 1

$ ./_build/default/bin/otl erl examples/one_true_language.otl one_true language

-file("examples/one_true_language.otl", 1).

-module(one_true).

-export([language/0]).

language() -> 1, true.

And we are done

We just created the one true language (TM)

Parse other numbers

commit 13:

$ ./otl-shell

>>> false true 1
1
>>> 92
92
>>> 42
42

Support floats

commit 14:

>>> 42
42
>>> 12.4
12.4
>>> true
true
>>> false
false