[Twisted-Python] Non-blocking http client?

[Erik Wickstrom]

Hi all,

I have an application that is doing some web spidering.  Right now I'm
using urllib to retrieve the URLs, but it is painfully slow.  I was
wondering if it's feasible to swap out urllib with a twisted client
that uses deferds so I can process urls in a more "parallel" fashion?

I've done a bunch of Googleing, but I haven't come across anything
that I can use as a drop in replacement.   If you can point me in the
right direction I'd really appreciate it!

Thanks for your help!
Erik