[Twisted-Python] Twisted HTTP client supporting failover for multiple A records?

[Itamar Turner-Trauring]

On Thu, 2010-07-15 at 10:46 +0100, Reza Lotun wrote:

> As for connecting to hosts that resolve to multiple A records - I
> presume as a means of load balancing via DNS round robin - I'm not
> quite sure this is natively supported in Twisted. I believe since all
> TCP connections are mediated via connectTCP hostnames are ultimately
> resolved via socket.gethostbyname. I think you really want the support
> provided by socket.gethostbyname_ex
> (http://docs.python.org/library/socket.html#socket.gethostbyname_ex).
> 
> It's a good question though. I'm sure a core dev will come along and
> give a proper answer soon ;-)

Gar. I should read better. Twisted uses a threadpool of gethostname by
default, but you can plug in your own resolver (e.g. you can use
twisted.names):

http://twistedmatrix.com/documents/10.1.0/api/twisted.internet.interfaces.IReactorPluggableResolver.html#installResolver

The question is whether the client code re-resolves on each re-connect,
and whether the current lookup interface is sufficient for this use
case.

Alas, I'm pretty sure the answer is no.

You could however always just do the DNS lookup yourself, passing
resulting correct IP to connectTCP, just make sure you don't block (e.g.
by using deferToThread to call gethostbyname_ex).