typhoeus

You can change the gem so that you don't need to use Typhoeus if you don't want to.

My preferred (by far) is typhoeus, as it supports parallel requests and wraps around libcurl.

You can also remove the part of the gem that uses Typhoeus if you want.

I've been a fan of your work since Typhoeus and that's part of why I was happy to bet on you making influx amazing.

I would be curious the speed with Typhoeus for the adapter...

In this specific case we're using Unicorn on MRI, so I wouldn't really expect Typhoeus to do a whole lot since it wouldn't really be parallel.

I use Typhoeus which uses Ethon depends on ffi.

I have tried lots of different options including EventMachine with em-http-request and reactor loop and concurrent-ruby (both a re very poorly documented) Typhoeus has a built-in concurrency mechanism with callbacks with specified number of concurrent http requests.

I've done extremely complex jobs in plain Ruby, the Kiba DSL or just using Ruby to augment/encapsulate shell commands (ruby script in crontab->curl wrapper[typhoeus] to fetch data & write to file->%x(some grep/sed/awk stuff)->do more work with output in ruby).

From the article: >After each batch of new data was received from the AP, this server determined which pages needed to be re-rendered and, using the Typhoeus libcurl-multi bindings for Ruby, pulled new data for each of these pages from the render pool.

I know I can't run typhoeus + thin on 1.9.2 on OSX as it reliably crashes every ten minutes and I have no clue on how to debug it, but it is not a problem with the interpreter, it's a problem with external libraries.

Rephrase as if it was written by a 5 year old: "Typhoeus is a special thing that you can use with the gem, but it's not necessary.

The "technical digression" with Github code on how to do a mass scrape is cool, though not surprising (er...how else would you accomplish such a big data task...though I'm glad I'm not the only one who uses Typhoeus, a great but underused gem).

Thanks for showing me to Typhoeus, that's sweet lib!

I love Typhoeus, though I can't spell it 9/10 times.