Hi Brad,<div><br><div class="gmail_quote">On Thu, Oct 20, 2011 at 11:50 AM, Brad Milne <span dir="ltr"><<a href="mailto:brad.milne@devx.runthered.com">brad.milne@devx.runthered.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>...</div><div>So my questions:</div><div>1) I'm right, right? Twisted doesn't have mechanisms or a library for hot swapping code modules or similar solutions for low impact upgrades and high availability? (I originally read about Twisted *plugins* hoping for this, but was disappointed to find how basic they actually were)</div>
</blockquote><div><br></div><div>No, not a complete, first-class mechanism (at least as far as I know).</div><div><br></div><div>However, there are a few pieces in Twisted which can be used to build such a solution. I've been meaning to piece them together for a while to do this, but time-constraints and what-not have prohibited that so far:</div>
<div><br></div><div>1. <a href="http://twistedmatrix.com/documents/current/api/twisted.python.rebuild.html">http://twistedmatrix.com/documents/current/api/twisted.python.rebuild.html</a></div><div>This a module that can be used to reload class definitions and instances, etc. You can conceive of a system whereby you load a class that defines a code loading module that can pick up modules in a directory and periodically refresh their definitions, for example. One way to do it with RabbitMQ is have a queue specifically for code-refreshes - when a code-reload service receives a message it does one or more twisted.python.rebuilds to reload the code definitions, or something. There is obviously still a lot you have to do yourself, but it's something. (I'm aware that reloading modules in Python is generally difficult because of all the stray references that can still exist, but you can probably structure the code in such a way that a reload, or series of reloads will do the job).</div>
<div><br></div><div>2. <a href="http://twistedmatrix.com/documents/current/api/twisted.runner.procmon.ProcessMonitor.html">http://twistedmatrix.com/documents/current/api/twisted.runner.procmon.ProcessMonitor.html</a></div>
<div>A service to monitor processes. You can have other services running in container processes, and use this functionality to monitor and control them.</div><div><br></div><div>3. <a href="http://twistedmatrix.com/documents/current/api/twisted.internet.protocol.ProcessProtocol.html">http://twistedmatrix.com/documents/current/api/twisted.internet.protocol.ProcessProtocol.html</a></div>
<div>You can define process protocols that implement specific job types. Usually this is used to spread process amongst cores, but you could also use it for sanboxing functionality I suppose. Another project that uses this for spreading working process amongst cores is Ampoule: <a href="https://launchpad.net/ampoule">https://launchpad.net/ampoule</a></div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>2) Is RabbitMQ with txAMQP-empowered Twisted instances on the ends of the queues a sensible, compatible, best-practice approach as agreed by the community?</div></blockquote><div><br></div><div>Sure. It's one approach. You can also use Redis and txRedis - <a href="https://github.com/rlotun/txRedis">https://github.com/rlotun/txRedis</a>. Redis has pub-sub functionality that's much simpler than AMQP (but not as powerful - for example you can't do something like ACK a task). But for basic pub-sub it's awesome.</div>
<div><br></div><div>We use both RabbitMQ and Redis, via txAMQP and txRedis.</div><div><br></div><div>Reza </div></div><div><br></div>-- <br>Reza Lotun<br>mobile: +44 (0)7521 310 763<br>email: <a href="mailto:rlotun@gmail.com" target="_blank">rlotun@gmail.com</a><br>
work: <a href="mailto:rlotun@twitter.com" target="_blank">rlotun@twitter.com</a><br>@rlotun<br><br>
</div>