Mariano Guerra's Log (Posts about gae)

PubHubSubBub Developer Tutorial

Mariano Guerra — Tue, 15 Mar 2011 02:22:00 GMT

Note: the project and this document are hosted here: https://github.com/marianoguerra/pshb-example improvements and corrections are welcome!

welcome, this document contains an example application written in python that will help you play with a pshb (pubsubhubbub from now on) hub and an application that publishes its content to it.

this guide explains how to install a pubsubhubbub server in your computer so you can play with it, normally on a web application you would use a pshb compatible server like:

official pshb hub on app engine: http://pubsubhubbub.appspot.com/
superfeedr hub: http://superfeedr.com/

Requirements

python >= 2.5 (I'm using 2.6.1)
git (for the example)
subversion (to download pshb code)
bash or similar shell to run the scripts

Installing

git clone https://github.com/marianoguerra/pshb-example.git
cd pshb-example
bash setup.sh

the first command will fetch the project from github, the third one will get some libraries needed for the example to run.

note: answer "y" to sammy and "n" to all the other questions that the script does (we don't need those libraries)

Running

now we will start the example application called pleinu twice (so we can test the communication using the hub) and we will start a local pshb hub.

open 3 terminals and run one comment on each one:

google_appengine/dev_appserver.py src/ -p 8000 --datastore_path=/tmp/tubes1
google_appengine/dev_appserver.py src/ -p 8001 --datastore_path=/tmp/tubes2
google_appengine/dev_appserver.py pubsubhubbub/hub/

this commands asume that you are at the root of the pshb-example folder.

Playing

open a browser tabs pointing to:

http://localhost:8000/

click the signup link and create a new user.

I will create one called spongebob, you will have to change the username whenever you see it.

after the signup process click the login button and enter the user and password you just entered.

create a message and click send, the message should appear below.

now go to http://localhost:8000/atom/messages/from/spongebob/

you should see an atom feed with the message you just created.

Publishing

now that we have a page that generates information we need to publish it on the hub.

open a tab in your browser pointing to http://localhost:8080/ and click on the publish link near the bottom.

if you get an error remove the s from the https protocol in the address bar and refresh.

on the Topic field enter the url to the atom feed we saw before: http://localhost:8000/atom/messages/from/spongebob/

and click Publish, the page wont change, that's ok.

Subscribing

now we need to subscribe one user from the other site (http://localhost:8001/) to the messages sent by our user.

go to http://localhost:8001/ and create another user, I will call it patrick

in the main page of the hub (http://localhost:8080/) click on the subscribe like near the bottom

enter http://localhost:8001/p/notify/patrick/ on the Callback field (change patrick for your username if you used another one) enter http://localhost:8000/atom/messages/from/spongebob/ on the Topic field (change spongebob for your username if you used another one)

click the "Do it" button, the page won't change, that's ok.

Sending a message

Go to http://localhost:8000/ (login if you closed it) and send a message.

Now go to http://localhost:8001/ and refresh the page, you should see the messages published by the user in the other site.

Note

to make it work and avoid an exception I had to add a return statement at the beginning of the log_message function at google_appengine/google/appengine/tools/dev_appserver.py

if you get that exception like this:

in log_request
    self.requestline, str(code), str(size))
  File "/home/asd/pubsubhubbub/pshb/google_appengine/google/appengine/
tools/dev_appserver.py", line 3314, in log_message
    if self.channel_poll_path_re.match(self.path):
AttributeError: DevAppServerRequestHandler instance has no attribute
'path'

edit the function to look like this:

def log_message(self, format, *args):
  """Redirect log messages through the logging module."""
  return
  if self.channel_poll_path_re.match(self.path):
    logging.debug(format, *args)
  else:
    logging.info(format, *args)

you will have to set write permissions to the file to save it (chmod u+w dev_appserver.py)

you will have to restart the pshb server:

google_appengine/dev_appserver.py pubsubhubbub/hub/

nuevo intento de viejo proyecto

Mariano Guerra — Fri, 12 Mar 2010 14:32:00 GMT

queres probar algo nuevo? tenes algun lugar para recomendar? ayudame a cargar datos http://pleinu.appspot.com (consejos/quejas bienvenidas)

esta vez empieza desde lo basico y se arma con tu feedback

para los nionios como yo, esta hecho en python sobre appengine usando tubes*

* http://a-series-of-tubes.appspot.com/ y http://github.com/marianoguerra/tubes

Pubsubhubbub developer's guide

Mariano Guerra — Sat, 12 Sep 2009 00:00:00 GMT

naming conventions

pshb/PSHB = pubsubhubbub
gae/GAE = Google App Engine
things that start with "$" are bash commands
things that start with "!" are important notes

create directory to hold all the content of this tutorial

$ mkdir pshb
$ cd pshb

download Google App Engine SDK

! pshb trunk needs the latest version of GAE (1.2.4 as of today) to work, but ! the download page display an older version

$ wget http://googleappengine.googlecode.com/files/google_appengine_1.2.4.zip
$ unzip google_appengine_1.2.4.zip

! GAE needs python 2.5 in order to work, if you have an older or newer version of python install python 2.5 (OS and distro dependent, wont be covered here)

download the latest version of pshb

$ svn checkout http://pubsubhubbub.googlecode.com/svn/trunk/ pubsubhubbub

start pshb

$ python2.5 google_appengine/dev_appserver.py pubsubhubbub/hub/

! note the python2.5 command

check that the hub started

type http://localhost:8080 on your browser, you should see something like:

Welcome to the demo PubSubHubbub reference Hub server!

download the example

# we will need to fetch the tubes library and the example that we will be using.
$ git clone git://github.com/marianoguerra/tubes.git

# now we will start the example

$ cd tubes/ihasfriendz/
$ python main.py

we should see something like:

 * Running on http://0.0.0.0:8081/
* Restarting with reloader...

components used by the example

werkzeug: http://werkzeug.pocoo.org
jquery: http://jquery.com
tubes: http://github.com/marianoguerra/tubes/tree/master
pubsubhubbub_publish.py: from pshb
feedformatter.py: http://code.google.com/p/feedformatter/

publish the feed to the hub

go to the following URL on your browser: http://localhost:8080/publish

on the Topic field enter: http://localhost:8081/atom/stream/_MYUSER_ and click publish

! if everything goes OK, then you wont notice anything on the page, that's ok, browsers act that way to 204 responses

! MYUSER is a placeholder for the user you will use to post notices on the test app later (for example http://localhost:8081/atom/stream/marianoguerra)

subscribing to the hub

go to the following URL on your browser: http://localhost:8080/subscribe

on the Callback field enter: http://localhost:8081/callback on the Topic field enter: http://localhost:8081/atom/stream/_MYUSER_ on the Verify token field enter something random like: iwantmahcookie

! MYUSER is a placeholder for the user you will use to post notices on the test app later (for example http://localhost:8081/atom/stream/marianoguerra)

click Do it

! if everything goes OK, then you wont notice anything on the page, that's OK, browsers act that way to 204 responses

create some content

go to the following URL on your browser: http://localhost:8081/files/index.html

post some content on the form, use the user you used as MYUSER

you can check that the item was posted going manually to http://localhost:8081/atom/stream/_MYUSER_, you should see an atom feed there

manually processing the tasks

when running the hub on the dev server we have to run the task queues by hand, to do that go to http://localhost:8080/_ah/admin/queues

on the Tasks in Queue column of the feed-pulls you should see a number different than 0 (that is the number of messages you created since the last execution of that task).

click on the feed-pulls link, there click on the run button.

when we run the feed pulls task, we tell pshb to fetch th e feeds that have new content (the ones that did a post to the hub to inform that there is new content)

! on production hubs this tasks are done autom atically

now we go again to the Task Queues page, th ere the event-delivery queue should have a number different than 0 (the number of messages that are pending to be sent to the subscribers), we click on the event-delivery and then we click on the run button.

when we run the event delivery task, we tell pshb to do a POST on every callback url registered for the feeds that were fetched on the feed-pulls task.

seeing it work

now that we created a feed, informed the hub that we had new content, the hub fetched the content and sent it to the callback, we want to see this content, for this go with your browser to http://localhost:8081/new-notices/, you will see the notices that pshb posted back to you the last time.

! if you refresh the page you will notice that the messages aren't there anymore, that's because the example stores the new messages in a Queue that is flushed when the request for new notices is made, in this way you can see only the new messages.

! the example stores all the information on global variables on main.py (this is to make the example simpler), so every time you change something on main.py and save the server will reload the changes and all the data will disappear.

for lazy people

#!/usr/bin/env sh

# create the example directory
mkdir pshb
cd pshb

CWD=$(pwd)
EXAMPLE=$CWD/tubes/ihasfriendz

wget http://googleappengine.googlecode.com/files/google_appengine_1.2.4.zip
unzip google_appengine_1.2.4.zip

svn checkout http://pubsubhubbub.googlecode.com/svn/trunk/ pubsubhubbub


# we will need to fetch the tubes library and the example that we will be using.
git clone git://github.com/marianoguerra/tubes.git

# now we will start the example

echo "run \"cd $EXAMPLE; python main.py\" on a shell to run the example"
echo "run \"cd $PWD; python2.5 google_appengine/dev_appserver.py pubsubhubbub/hub/\" on a shell to run the hub"