As we are about to move to Elasticsearch for indexing our logs, my Hack Week idea was to experiment with building an app-wide search function. It is just a prototype but it enables users to search across Contacts, Projects and Expenses and can easily be extended. Elasticsearch is accessed from Rails using the Tire gem. Instead of using Tire’s after_save callback to keep the index up to date, Elasticsearch has the concept of rivers which pulls new data. Every update triggers an AMQP message using Bunny which is then picked up by Elasticsearch RabbitMQ river.
It was an exciting idea and I really enjoyed the hack week and had the opportunity of experimenting with new pieces of infrastructure which we hope to use soon.
I’ve long been wanted to learn Erlang and I found a very good guide (isn’t as thorough but for an Erlang-noob it looks perfect).
The syntax is kind of strange but the most difficult part is thinking in a functional way. Hopefully I am going to write some toy projects in Erlang and get accustomed with it.
Another article I’ve been reading is a hands on review of Dynamo by implementing a key-value store in Erlang. It’s not a tutorial but the first article is simple enough to learn how to write Erlang modules.
If you have any suggestions for projects or other documentation please leave a comment.
Stanford started the Winter quarter class which now covers iPhone 3.0.
It’s a nice opportunity to play a bit with iPhone apps and learn Obj-C (which at first looked weird).
Get the videos from iTunes U.
I thought about how to process logs faster than using a cron job to poll for updates and I found mkfifo which you can use to get what’s written in real time. It makes a FIFO pipe which acts like a regular unix file except it has to be opened for reading (using fopen) before writing to it. I tested it on Snow Leopard, it is the same in linux as well.
After getting the updates in real time you can pushing them to a queue server like RabbitMQ for processing. Here’s how I done it using Ruby, Eventmachine and amqp gem:
After a clean install of Snow Leopard I decided not to use MacPorts but to go with homebrew. Its main advantages are no duplication (macports installed its own version of ruby, python and other libs) and the path layout: packages are installed into their own prefix (eg. /usr/local/Cellar/wget) and then symlinked into the Homebrew prefix (eg. /usr/local).
And because all of the formulas are in git, to add one you just fork the project on GitHub, push your formula and make a pull request. Because of that it’s one of the most forked project on GitHub.
That’s what I did and now you can install Tokyo Cabinet and Tokyo Tyrant from my fork (I hope it will be merged into master soon) from main repo:
At I/O there were some really interesting Office Hours sessions where you could go and talk to the engineering teams about questions and issues. I wanted to find out a bit more about Wave and so went along to their APIs one. At the end of a really interesting chat an understandably tired Douwe gave me a mysterious looking card and invited me to come up to Mountain View the following day to hack on the Wave APIs… amazing, thank you Douwe.
I liked the ARGy nature of the mysterious tr.im URL on a card with nothing else on it, which lead you to a webpage where you could sign up.
And a set of e-mails telling you where to go and how to get into the Waves sandbox. I had a few meetings in the morning in San Francisco I couldn’t miss and was worried it was getting too late to really build stuff when I got to the Hackathon so I spent most of the journey on the Caltrain up there looking through the docs, pulling together some simple ideas I could do in the 2 hours I’d have to code and also writing some very simple guestimate code based on the docs.
The ARGy nature of the day carried on with fantastic lo-fi handwritten signs telling you where to go which I wished I’d photographed. When I got into the hacking room there were about 50 of us including the Wave team and it was a real honour and pleasure to meet Lars and many of the others who’d made this exciting new thing.
Within about an hour or so (glad I already know AppEngine, all Robots are currently implemented in AppEngine) I had the Robot responding to the text properly and then in a little while longer I had the latest 10 stories from The Guardian Content API being pushed into the right Blip within the Wave. We’ll be covering my code over on the Open Platform blog and we’ll put a screencast of it in action there and here soon. I called it Grauniady, partly because of The Guardian’s pet name in the UK of The Grauniad, partly because a naming convention seems to have come out of the Wave team of robots ending in “y” and partly because Lar’s demo of the spell checking was just so amazing, it all seemed appropriate.
It was a really amazing day, some fantastic demos from all the developers which ranged from a collaborative drum machine and a piano embedded in a Wave, to a robot from Twilio which looked for phone numbers and enabled a call between people, and recorded that call and embedded it in the wave (awesome demo). There were a couple of Robots which behaved like bots and responded to text.
Apart from the usual laptop/projector hookup mayhem Grauniady performed admirably and worked fine. Thanks to a Tory MP trying to claim for a “duck island” and Anna Pickard’s use of the word “cockweasel” in The Apprentice live blog I had some good funny examples to show (always good to hear laughter at something other than sketchy code). Obviously I was following Simon Willison’s convention of API demo’s involving animal names.
Thanks to Kevin Marks for the perfectly timed lift back to Mountain View station (5 minutes later and I’d have been sitting on the platform for an hour). Love the double decker CalTrains, couldn’t stop grinning about them and the day on the way back. Totally memorable. Feel very lucky. Thank you.
Another script I stole from evilchelu is j.sh. It enables you to quickly jump to a folder that you often access by writing a small part of its name. I use it to quickly cd to my projects or other repositories with "j <a few letters>"
To install it save it somewhere (I use a bin/ folder in my home) and add "source <path to j.sh>" to your .bashrc or .bash_profile.
As I said on twitter, I now switched to Vim (MacVim specifically). I am using evilchelu’s vimrc which has FuzzyFinder (my favourite TextMate feature). It wasn’t that hard to get productive at acceptable level.
Just saw Innovation in Rails by Gregg Pollack and Jason Seifer (RailsEnvy guys) where they talked about the new things in the past year in the Ruby and Rails world. Confreaks posted high resolution 720p videos of the conference on their site.
Gregg showed how to use a reverse proxy cache to speed up your application. You can find links about the discussed topics on the RailsEnvy blog. Earlier Gregg Pollack also released a series of screencasts about scaling Rails debunking the myth that Rails can’t scale.
For my Rails projects currently I’m using Thoughtbot's stack (Shoulda, factory_girl) and now I switched to Clearance. I love it because it’s tested and it’s easy to extend. One such modification that I need is to allow users to login with username. I wrote a demo app and here’s a tutorial on how to do it yourself: