[Twisted-Python] Any Message Queue recommendations
Ross Jekel
ross at sourcelabs.com
Thu Jun 9 22:24:14 EDT 2005
Thanks Brett. Pretty much everything you describe is supposed to be in
xmlBlaster using the xmlrpc interface. I can give you the start of a
class that helps access xmlBlaster asynchronously if you want to check
it out. At this point since I'd just be reinventing xmlBlaster, I'll
spend a couple of days trying to get the queues working the way I want.
I basically need the messages to be delivered once and only one to the
subscriber. Even if the subscriber was disconnected for a while and
reconnects later I want the historical messages to be delivered once.
This isn't rocket science, so I'm surprised it isn't available.
Brett Viren wrote:
>Hi Ross,
>
>Ross Jekel <ross at sourcelabs.com> writes:
>
>
>
>>I'm writing several service processes that unfortunately need to
>>integrate with Java, PHP, and potentially C/C++ programs (non-legacy).
>>I've been searching for a message queuing layer that would support
>>Python/C++/PHP and Java clients (JMS is not necessarily a requirement).
>>
>>
>
>This sounds a lot like what I asked recently regarding a network
>distributed system for producing and consuming "alarms" (basically
>just your "messages").
>
>Consider this:
>
> - XML-RPC for communications. XML-RPC implementations exist in all
> your target languages. Twisted has addition support beyond the
> built-in Python XML-RPC.
>
> - Write a central proxy server speaking XML-RPC, (implemented in
> Twisted, of course!). Every message comes through this proxy.
>
> - Message producers do simple XML-RPC query/response (method call) to
> send a message to the central proxy server.
>
> - Then, depending on how you want your message consumers to behave
> you can either:
>
> a) have them poll the server. This can work in two ways. Either
> the proxy returns any messages collected since some time, or you can
> have the query block until a message returns. Or a mixture of both.
>
> b) have consumers register with the proxy and provide a callback
> XML-RPC url, the consumers then go into listen mode and the proxy
> pushes messages as they come in.
>
>Option (a) is good because your consumers can be behind a firewall.
>It's not so good as it introduces latency and makes the proxy more
>complex as it must handle the buffering of messages.
>
>Option (b) is simpler and introduces less latency between message
>production and consumption but does require your consumers to be able
>to listen on a port and receive incoming connections. If there is a
>firewall between the consumers and the proxy or if your local network
>overlords don't like programs listening on ports this won't do.
>
>You probably want to also allow for message levels and/or classes so
>consumers can pick and choose what they receive.
>
>And, if you go this route be sure to post your work so I can steal it
>for my project!
>
>Regards,
>-Brett.
>
>_______________________________________________
>Twisted-Python mailing list
>Twisted-Python at twistedmatrix.com
>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
More information about the Twisted-Python
mailing list