[Twisted-Python] Twisted tips for designing highly concurrent twisted REST API
Tobias Oberstein
tobias.oberstein at crossbario.com
Sun Jul 14 03:39:29 MDT 2019
Am 11.07.19 um 23:34 schrieb Sean DiZazzo:
> Klein and Crossbar.io seem relevant as well
>
> https://crossbario.com/blog/Going-Asynchronous-from-Flask-to-Twisted-Klein/
yeah, klein is neat!
fwiw, this might also be of interest, as it allows to scale-up twisted
web (and hence also klein) on multi-core (on linux)
https://github.com/crossbario/crossbar-examples/tree/master/benchmark/web
combining SO_REUSEPORT with Klein results in a concurrent, async
(threadless) server parallelized via processes ..
>
> On Thu, Jul 11, 2019 at 1:46 AM Scott, Barry <barry.scott at forcepoint.com
> <mailto:barry.scott at forcepoint.com>> wrote:
>
> On Tuesday, 9 July 2019 22:04:11 BST Tom Most wrote:
>
> ...snip...
>
> > The reactor's own thread pool is really for DNS
> > resolution.
>
> Is that still true in the default case? We are use the twisted code
> that talks
> to DNS servers as the threaded resolver adds too much latency.
>
> > You risk deadlocks in a system that
> > ThreadPoolThreadPoolThreadPool
> >
> > 3. The specifics of what long_computation are also important. If
> it doesn't
> > release the GIL you won't get real parallelism (this is a Python
> thing, not
> > a Twisted thing). See this recent thread on the topic
> >
> <https://twistedmatrix.com/pipermail/twisted-python/2019-June/032371.html>.
>
> We pass out the computational work to other processes over
> unix-domain-sockets
> to avoid the GIL issues.
>
> >
> > Though the mechanisms differ athis thread on the topicny of the
> above would
> > cause the response time to increase as you add load.
> >
> > Good luck,
> > Tom
>
> Barry
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> <mailto:Twisted-Python at twistedmatrix.com>
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
--
Tobias Oberstein - phone +49 176 2375 2055 - tobias.oberstein at crossbario.com
Crossbar.io GmbH - Waldstrasse 18 - 91054 Erlangen
HRB 15870 - Amtsgericht Fuerth - Geschäftsfuehrer/CEO - Tobias Oberstein
https://crossbar.io
https://crossbario.com
More information about the Twisted-Python
mailing list