Processing logs in real-time
I thought about how to process logs faster than using a cron job to poll for updates and I found mkfifo which you can use to get what’s written in real time. It makes a FIFO pipe which acts like a regular unix file except it has to be opened for reading (using fopen) before writing to it. I tested it on Snow Leopard, it is the same in linux as well.
After getting the updates in real time you can pushing them to a queue server like RabbitMQ for processing. Here’s how I done it using Ruby, Eventmachine and amqp gem:
mihai$ mkfifo log mihai$ ruby1.9 publisher.rb log & mihai$ echo "Test" > log
-
lucien-saint1 liked this
-
ropiku posted this
