Some Quotes I had around and wanted to put somewhere.

Navigation implies state. Software that can be navigated is software in which the user can get lost. The more navigation, the more corners to get stuck in. The more manipulable state, the more ways to wander into a “bad mode.” State is the primary reason people fear computers—stateful things can be broken

Most of the time, a person sits down at her personal computer not to create, but to read, observe, study, explore, make cognitive connections, and ultimately come to an understanding. This person is not seeking to make her mark upon the world, but to rearrange her own neurons. The computer becomes a medium for asking questions, making comparisons, and drawing conclusions—that is, for learning.

Many types of context can be naturally expressed in some informative graphical domain, relieving the user from manipulating information-free general-purpose controls.

If the software properly infers as much as possible from history and the environment, it should be able to produce at least a reasonable starting point for the context model. Most of the user’s interaction will then consist of correcting (or confirming) the software’s predictions. This is generally less stressful than constructing the entire context from scratch.

Simplicity. “I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is make it so complicated that there are no obvious deficiencies.” C.A.R. Hoare, The Emperor’s Old Clothes Turing Award lecture (1980), p81. From a practical (and historical) standpoint, we can assume that no complex specification will be implemented exactly. This, in itself, is not a problem. However, multiple, decentralized implementations of a complex specification will be incorrect in different ways. A platform consisting of the union of all possible implementations is thus arbitrarily unreliable—the designer can have no assurance of what a recipient actually receives. For a platform to be reliable, it must either have a single implementation, or be so utterly simple that it can be implemented uniformly. If we assume a practical need for open, freely implementable standards, the only option is simplicity.*

All of our days are numbered, we cannot afford to be idle To act on a bad idea is better than to not act at all. Because the worth of an idea never becomes apparent until you do it.

—nick cave, 20000 days on earth

As usual, brain dump, just instructions, not much content.

download flume from here: https://flume.apache.org/download.html

I'm using this one: http://www.apache.org/dyn/closer.cgi/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz

unpack and put it somewhere.

create a file with the following content, I will name it flume-syslog.conf and place it in ~/tmp/, you should too if you are lazy and don't want to change the commands:

# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# I'll be using TCP based Syslog source
a1.sources.r1.type = syslogtcp
# the port that Flume Syslog source will listen on
a1.sources.r1.port = 7077
# the hostname that Flume Syslog source will be running on
a1.sources.r1.host = localhost

# Describe the sink
a1.sinks.k1.type = logger

# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

Install rsyslog if you don't have it and start it, I'm using fedora 22, change for your distro:

sudo dnf install rsyslog
sudo service rsyslog start

Configure rsyslog with your rule, you can do it directly on /etc/rsyslog.conf or better, check that the following line is uncommented:

$IncludeConfig /etc/rsyslog.d/*.conf

And put your config under /etc/rsyslog.d/50-default.conf (create it if it doesn't exist)

We are going to forward only messages with a given tag, since we are interested on a subset of the logs, in this case we only want log lines with the tag "test", add this to the rsyslog config file:

:syslogtag, isequal, "test:" @@127.0.0.1:7077

Save and restart rsyslog:

sudo service rsyslog start

Start flume with your configuration:

./bin/flume-ng agent --conf conf --conf-file ~/tmp/flume-syslog.conf --name a1 -Dflume.root.logger=INFO,console  -Dorg.apache.flume.lifecycle.LifecycleSuperviso=INFO,console

Now generate a log line with our tag:

logger -t test 'Testing Flume with Syslog!

you should see a line like this:

2015-08-27 18:06:25,096 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{host=ganesha, Severity=5, Facility=1, priority=13, timestamp=1440695180000} body: 74 65 73 74 3A 20 54 65 73 74 69 6E 67 20 46 6C test: Testing Fl }

If you don't see the line check /var/log/messages to see if your message is there:

sudo vim /var/log/messages

Bonus track! sending apache logs to syslog and from there to flume.

for this install apache 2, on fedora:

sudo dnf install httpd
sudo service httpd start
sudo bash -c "echo 'welcome!' > /var/www/html/index.html"

curl localhost

The output should be:

welcome!

Now configure apache to forward logs to syslog, open /etc/httpd/conf.d/welcome.conf and add at the bottom:

CustomLog "|/usr/bin/logger -t test" combined

Restar apache:

sudo service httpd restart

Now open the page or use curl to get a page:

/etc/httpd/conf.d/welcome.conf

You should see a new log on flume.

Where to go from here?

• Put flume on another machine, change the ip address 127.0.0.1 to that address

• change the tag (test) on rsyslog and on welcome.conf to something else

• Buy me a beer

This is a continuation of this post: Enable CORS in Apache Solr but this time for an instance that is running in cloudera.

No idea how it was installed since it was already there, but doing some investigation and avoiding reading the docs at all costs I arrived at this solution.

The idea of this post is to make you avoid reading the docs too!

First I will give names to some things that may be different for you:

CDH=/opt/cloudera/parcels/CDH-5.3.2-1.cdh5.3.2.p0.10/
CDH_USER=cloudera-scm
CDH_GROUP=cloudera-scm

Now do:

cd $CDH/jars/
wget http://repo1.maven.org/maven2/org/eclipse/jetty/jetty-servlets/9.1.5.v20140505/jetty-servlets-9.1.5.v20140505.jar
wget http://repo1.maven.org/maven2/org/eclipse/jetty/jetty-util/9.1.5.v20140505/jetty-util-9.1.5.v20140505.jar

cd $CDH/lib/bigtop-tomcat/lib/
ln -s $CDH/jars/jetty-servlets-9.1.5.v20140505.jar
ln -s $CDH/jars/jetty-util-9.1.5.v20140505.jar

chown $CDH_USER.$CDH_GROUP jetty-servlets-9.1.5.v20140505.jar
chown -h $CDH_USER.$CDH_GROUP jetty-servlets-9.1.5.v20140505.jar

chown $CDH_USER.$CDH_GROUP jetty-util-9.1.5.v20140505.jar
chown -h $CDH_USER.$CDH_GROUP jetty-util-9.1.5.v20140505.jar

Then create $CDH/lib/bigtop-tomcat/bin/setenv.sh with your favorite text editor and put in it the following:

CLASSPATH="$CATALINA_HOME"/lib/jetty-util-9.1.5.v20140505.jar:"$CATALINA_HOME"/lib/jetty-servlets-9.1.5.v20140505.jar:"$CATALINA_HOME"/lib/servlet-api.jar

Open $CDH/etc/solr/tomcat-conf.dist/WEB-INF/web.xml with your text editor and follow the instructions at Enable CORS in Apache Solr

The way to know if it worked is to open the Solr admin panel, if it loads it works, if it doesn't look at the logs, mine are at /var/log/solr/. To be sure that the classpath was set correctly from setenv.sh look in the solr admin page in the "Java Properties" section for the java.class.path variable, it should have the class path you set in setenv.hs plus some extra stuff (mainly bootstrap.jar).

If the admin page doesn't load (tomcat 404) look at the logs, some class loading error may be happening, comment the config you added in web.xml and restart.

The version I'm using of the jetty jars is because newer versions are compiled for java 1.8 and I have 1.7, use older/newer depending on your java version.

Quick post since there's no easy googlable (?) resource to do this.

open the file server/solr-webapp/webapp/WEB-INF/web.xml and add the following XML before the existing filter section:

<filter>
    <filter-name>cross-origin</filter-name>
    <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class>
    <init-param>
         <param-name>allowedOrigins</param-name>
         <param-value>http://localhost*</param-value>
    </init-param>
     <init-param>
         <param-name>allowedMethods</param-name>
         <param-value>GET,POST,DELETE,PUT,HEAD,OPTIONS</param-value>
     </init-param>
     <init-param>
         <param-name>allowedHeaders</param-name>
         <param-value>origin, content-type, cache-control, accept, options, authorization, x-requested-with</param-value>
     </init-param>
    <init-param>
        <param-name>supportsCredentials</param-name>
        <param-value>true</param-value>
    </init-param>
    <init-param>
      <param-name>chainPreflight</param-name>
      <param-value>false</param-value>
    </init-param>
</filter>

<filter-mapping>
  <filter-name>cross-origin</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>

taken from this Stack Overflow response

I need to update the gif from the Event Fabric landing page and I forgot how I did it last time.

So this time I will write it here as a reminder.

First take the screenshots, I do it the good ol' way by using the browser fullscreen and hitting the screeshot key at almost regular intervals.

That leaves me with a set of screenshots I want to resize, so I run:

mogrify -path small -resize 800x450 *.png

This will resize all the *.png files in the current folder to 800x450 and write the results into a folder called small.

Now we go to the small folder and generate the gif:

cd small
convert -delay 100 -loop 0 *.png animation.gif

This will greate a gif that transitions every second from the png images and save it in the animation.gif file.

First we need to pick an ldap server to run, I've heard horror stories about setting up ldap servers locally for testing purposes so I was surpised by how easy it was to setup a ldap server using Apache Directory Studio which is a kind of Visual Tool to manage Apache Directory from an Eclipse based UI.

We start by downloading it from the Apache Directory Download Page

After downloading and unpacking it (snippet for the lazy linux users):

wget http://mirrors.fe.up.pt/pub/apache//directory/studio/dist/2.0.0.v20130628/ApacheDirectoryStudio-linux-x86_64-2.0.0.v20130628.tar.gz

tar -xzf ApacheDirectoryStudio-linux-x86_64-2.0.0.v20130628.tar.gz

Now we start it, in my case I had some segfaults as noted on the download page so I had to add the variable declaration before, it works for fedora, you may have to change the path a little:

GTK2_RC_FILES=/usr/share/themes/Raleigh/gtk-2.0/gtkrc ./ApacheDirectoryStudio

Now that we have it running we will see something like this:

We first create a new server by clicking on new server on the bottom left:

We pick "ApacheDS 2.0.0" just because we like to be in the bleeding edge (?):

If we double click the created server we can see the config, I leave everything as default:

We now start the server by clicking the green run button on the bottom left:

Then we want to create a connection, we do that by right clicking the server and selecting "Create a Connection":

Then we connect:

Now we can see the details of the server on the top left panel, we click the entry named "dc=example,dc=com" (in my case, if you changed the config it may be different for you.

on the pop up menu we pick "New -> New Entry":

On the dialog we select "Create entry from scratch" and click next:

We want to create an organization unit to hold our users so we start typing "org" on the "Available object classes" entry until we see the entry "organizationalUnit":

We select it and click the "Add" button, when the right panel is populated we click "Next >"

On the next step of the dialog we enter "ou" on the RDN entry and "users" on the right side and click "Next >"

On the next step we click "Finish"

Now we want to create a new user under our "users" organization unit, to do that we right click ont he users ou and again select "New -> New Entry" and pick "Create entry from scratch" and then "Next >".

after that we look for the object class "inetOrgPerson" and click "Add" and then "Next >":

On the next step we enter uid as RDN and the username as value on the right side, in my case it will be mariano because I'm egocentric :P

Then we click "Next >" and on the next step we fill cn (Common Name) and sn (Surename) and click "Finish"

by the way, dc is Domain Component and dn is Distinguished Name ;)

Now we have our first user but it doesn't have a password, let's set it by double clicking the user on the top left pannel and clicking the "New Attribute" on the bar at the top of the center panel:

On the dialog that opens we pick "Attribute type" "userPassword" and click "Next >"

We enter a password:

And that's it, we have a user with a password inside an organization unit!

Now that we have the server running and one user we can code some erlang to try it.

Most of the code was taken from the Erlang Central Article "How To Talk LDAP from Erlang"

1> Host = "127.0.0.1".
"127.0.0.1"
2> Port = 10389.
10389
3> {_,S} = eldap:open([Host], [{port, Port}]).
{ok,<0.37.0>}
4> UserRest = "ou=users,dc=example,dc=com".
"ou=users,dc=example,dc=com"
5> Username = "mariano".
"mariano"
6> Password = "secret".
"secret"
7> DN = "uid=" ++ Username ++ "," ++ UserRest.
"uid=mariano,ou=users,dc=example,dc=com"
8>
8> eldap:simple_bind(S, DN, Password).
ok
9> eldap:simple_bind(S, DN, "anothersecret").
{error,invalidCredentials}

and that's the basics, there's some more code in the article were this code was based.

You can also find the docs for the eldap library in the erlang documentation for eldap

create project skeleton:

mkdir eco
cd eco
wget https://github.com/rebar/rebar/wiki/rebar
chmod u+x rebar
./rebar create-app appid=eco

let's add some dependencies, ranch to accept tcp connections and lager for logging, for that open rebar.config with your text editor and enter this:

{deps, [
    {lager, "2.1.0", {git, "https://github.com/basho/lager", {tag, "2.1.0"}}},
    {ranch, "1.1.0", {git, "https://github.com/ninenines/ranch", {tag, "1.1.0"}}}
]}.

{erl_opts, [debug_info, {parse_transform, lager_transform}]}.

now let's try compiling it:

./rebar get-deps
./rebar compile

we can start our app from the shell, which won't be really useful, but why not:

erl -pa ebin/ deps/*/ebin

and we run:

1> application:start(eco).
ok

now let's use ranch and lager for something, first we create a protocol implementation, open a file called eco_protocol.erl and put the following content in it:

-module(eco_protocol).
-behaviour(ranch_protocol).

-export([start_link/4]).
-export([init/4]).

start_link(Ref, Socket, Transport, Opts) ->
    Pid = spawn_link(?MODULE, init, [Ref, Socket, Transport, Opts]),
    {ok, Pid}.

init(Ref, Socket, Transport, _Opts = []) ->
    ok = ranch:accept_ack(Ref),
    loop(Socket, Transport).

loop(Socket, Transport) ->
    case Transport:recv(Socket, 0, 5000) of
        {ok, Data} ->
            lager:info("echoing ~p", [Data]),
            Transport:send(Socket, Data),
            loop(Socket, Transport);
        _ ->
            ok = Transport:close(Socket)
    end.

edit the start function in src/eco_app.erl so it looks like this:

start(_StartType, _StartArgs) ->
    {ok, _} = ranch:start_listener(eco, 1, ranch_tcp, [{port, 1883}],
                                                        eco_protocol, []),
    eco_sup:start_link().

and add the apps we need in eco.app.src by adding ranch and lager to the applications entry like this:

{applications, [
                kernel,
                stdlib,
                ranch,
                lager
               ]},

now let's compile and try again:

./rebar compile

Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.3  (abort with ^G)
1> application:start(eco).
{error,{not_started,ranch}}
2> application:start(ranch).
ok
3> application:start(eco).
{error,{not_started,lager}}
4> application:start(lager).
{error,{not_started,goldrush}}
5> application:start(goldrush).
{error,{not_started,syntax_tools}}
6> application:start(syntax_tools).
ok
7> application:start(goldrush).
{error,{not_started,compiler}}
8> application:start(compiler).
ok
9> application:start(goldrush).
ok
10> application:start(lager).
ok
11> 21:05:52.373 [info] Application lager started on node nonode@nohost
11> application:start(eco).
ok
21:06:09.335 [info] Application eco started on node nonode@nohost

now let's send some data:

telnet localhost 1883

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
asd
asd

(I wrote the first asd, the second is the reply)

in the console you should see this log line:

21:10:05.098 [info] echoing <<"asd\r\n">>

now let's build a release so others can use our server (?):

mkdir rel
cd rel
../rebar create-node nodeid=eco

add the following two lines to rebar.config:

{sub_dirs, ["rel"]}.
{lib_dirs, ["deps"]}.

and edit rel/reltool.config, change the lib_dirs entry to this:

{lib_dirs, ["../deps"]},

add ranch and lager in the rel entry:

{rel, "eco", "1",
 [
  kernel,
  stdlib,
  sasl,
  ranch,
  lager,
  eco
 ]},

and change the app, echo entry to look like this:

{app, eco, [{mod_cond, app}, {incl_cond, include}, {lib_dir, ".."}]}

now let's try to build a release:

./rebar compile
./rebar generate

now let's start our server:

./rel/eco/bin/eco console

you should see some output like this:

Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]


=INFO REPORT==== 5-Feb-2015::22:15:22 ===
inet_parse:"/etc/resolv.conf":4: erroneous line, SKIPPED
21:15:22.393 [info] Application lager started on node 'eco@127.0.0.1'
21:15:22.394 [info] Application eco started on node 'eco@127.0.0.1'
Eshell V6.3  (abort with ^G)
(eco@127.0.0.1)1>

now let's telnet again:

telnet localhost 1883

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
lala!
lala!

on the console again you should see some log like this:

21:16:01.540 [info] echoing <<"lala!\r\n">>

and that's it, now evolve your echo server into an actual server :)

this is a brain dump of something I did and I want documented somewhere.

first install vagrant, I won't go over it.

then do:

vagrant init

then edit the generated Vagrant file, I changed this:

# use ubuntu trusty
config.vm.box = "ubuntu/trusty64"


# 1GB or ram
config.vm.provider "virtualbox" do |vb|
    vb.memory = "1024"
end

# provision all the needed packages, I'm using it to build a phonegap app
# that's why I install all the npm stuff
config.vm.provision "shell", inline: <<-SHELL
  sudo apt-get update
  sudo apt-get install -y ant lib32ncurses5 lib32stdc++6 lib32z1 npm build-essential git nodejs-legacy openjdk-7-jdk
  sudo npm install -g "phonegap@3.5.0-0.21.14" bower grunt-cli
SHELL

then:

vagrant up
vagrant ssh

inside the vm:

cd
wget http://dl.google.com/android/android-sdk_r24.0.2-linux.tgz
tar -xzf android-sdk_r24.0.2-linux.tgz

echo "export PATH=\$PATH:\$HOME/android-sdk-linux/tools:\$HOME/android-sdk-linux/platform-tools:\$HOME/android-sdk-linux/build-tools/19.1.0" >> $HOME/.bashrc
bash

android update sdk -a --no-ui --filter "platform-tools"
android update sdk -a --no-ui --filter "android-19"
android update sdk -a --no-ui --filter "sys-img-armeabi-v7a-android-19"
android update sdk -a --no-ui --filter "build-tools-19.1.0"
android create avd --name myandroid -t "android-19"

I installed the version 19 of the sdk because of project requirements, feel free to install something more up to date, to see what you can install you can run:

android list sdk --extended

this is a reminder for myself in the future, I tend to watch/listen to talks when I'm doing something like cooking, cleaning.

but when I'm outside I want to take my music player and the talks tend to be on youtube, so normally I search for "youtube to mp3" and there's some ad ridden site that does that (sometimes)

but today I wanted to learn how to do it from the cli and here is the post that explains it so I can come back int he future to copy paste the commands.

first install youtube-dl and ffmpeg:

sudo yum install youtube-dl ffmpeg

then download your video:

youtube-dl https://www.youtube.com/watch\?v\=06h21nBqwec

then extract the audio from the video:

ffmpeg -i Music\ Theory\ and\ Performance\ Analysis\ with\ Sebastian\ and\ Czerny-06h21nBqwec.webm -strict experimental -acodec vorbis -ab 128k -vn ./output.ogg

then do whatever you want with the video and the audio

ok, this is a small dump of what I did, I'm surely missing some stuff and surely it won't work as is for you, but maybe it does and you are as happy as I'm right now generating noise (not as happy as anyone close to me at this moment).

all the things I think I installed:

sudo apt install alsa-tools alsaplayer-jack alsa-utils pulseaudio-module-jack supercollider jack-tools fftw3 qjackctl openjdk-8-jdk

yes, you can see the desperation there with some stuff that may not make sense, but at least it works..

you need to install leiningen, it's really easy and it has good instructions on the site so I won't go into details, just follow the instructions here: http://leiningen.org/

then I follow the instructions from overtone's wiki here:

https://github.com/overtone/overtone/wiki/Installing-overtone

before starting our repl and rocking like a hurricane we need to kill pulseaudio and start jack, it sounds easier than it is because pulseaudio just won't stay dead :(

the way I found to make it work was to edit pulseadio client.conf:

sudo vim /etc/pulse/client.conf

uncomment the line (remove the semicolon):

; autospawn = yes

and leaving it like this:

autospawn = no

I added myself to the audio group, not sure if it's required but just in case:

sudo adduser $USER audio

for this to take effect you need to logout and login again, to make sure you have the group, open a terminal and run:

groups

you should see audio between some other groups, if you can't see it try rebooting or replacing $USER with your actual username in the adduser command.

now stop pulseaudio:

pulseaudio --kill

then start jackd, I tried all the combinations I could find on the internet without success, this is the one that worked for me:

jackd -R -d alsa -d hw:1

if that doesn't work try:

jackd -R -d alsa

or try the versions that are recommended on the overtone wiki:

jackd -r -d alsa -r 44100

or:

jackd -r -d alsa -r 44100 -P

you can also try running qjackctl and play with the settings to see if you have luck.

now that we (hopefully) have jackd running, we can start playing with overtone, here is a small dump of a session:

lein new tutorial
cd tutorial

# add [overtone "0.9.1"] to :dependencies
vim project.clj

lein deps
lein repl

# inside the clojure repl
user=> (use 'overtone.live)
user=> (definst foo [] (saw 220))
#<instrument: foo>
user=> (foo)
#<synth-node[loading]: user/foo 35>
user=> (kill 35)

when you are done you can stop jackd from the ui or from the shell however you started it and start again pulseaudio:

pulseaudio --start

some additional note, when I'm using jackd I can't set the volume from the media keys or the sound mixer in the top planel, I run:

alsamixer

press F6, select the output I'm using and change it with the keys (ESC to close).

hope it's useful for someone.