[Twisted-Python] Regarding Twisted Matrix

[naman jain]

Hey guys,

Thanks for your responses.
Nathan your pointer to the "deferToThread" really helped.

I finally did this:

ran my external script in a thread using:
threads.deferToThread(test.main....)

and started a function called status_check in another thread
threads.deferToThread(self.check_status)

then immidiately returned control to reactor.

def check_status:
                self.testprocess_status = self.parent_conn.recv()

now, this call on pipe blocks the thread not the reactor main loop.
in the reactor when the value is changed of this self.testprocess_status I
ping my client with the new status.

"Threads are twisted :)"

Thanks
Naman

On Tue, Oct 27, 2009 at 4:31 PM, Valeriy Pogrebitskiy
<vpogrebi at verizon.net>wrote:


> Naman,
>
>
>
> Another problem you might (possibly) be encountering – is the way your
> server and/or client protocol is implemented. If you just inherit from
> “plain” twisted.internet.protocol.Protocol – you are automatically using
> write () and dataReceived() methods that do not implement buffering (it is
> left up to developer). To enforce that line (string) oriented communication
> works correctly – you might want to refactor to use
> twisted.protocols.basic.LineReceiver instead.
>
>
>
> Kind regards,
>
>
>
> Valeriy Pogrebitskiy
>
> Email: vpogrebi at iname.com
>
>
>
>
>
> -----Original Message-----
> *From:* twisted-python-bounces at twistedmatrix.com [mailto:
> twisted-python-bounces at twistedmatrix.com] *On Behalf Of *naman jain
> *Sent:* Tuesday, October 27, 2009 6:36 AM
> *To:* Twisted general discussion
> *Subject:* Re: [Twisted-Python] Regarding Twisted Matrix
>
>
>
> I tried to implement this:
>
> parent_conn, child_conn = Pipe()
>
> f = defer.Deferred()
> f = threads.deferToThread(start_test.main_func, SCRIPT_PATH,
> TEMP_OUTPUT_PATH, self.output_name, child_conn)
>
> response = parent_conn.recv()
> print response //prints like: initialization done
> self.transport.write(response)
>
> response = parent_conn.recv()
> print response // configuration done
> self.transport.write(response)
>
> But get the same output ie all status messages concatenated and sent at
> once.!
>
> [Do we need to put some deffered/callback  mechanism on parent_conn.recv()
> ??
> as that is the call blocking the thread waiting for reading.]
>
> Any ideas ? How is such stuff done in twisted servers??
>
> Naman
>
>  On Tue, Oct 27, 2009 at 7:03 AM, Nathan <nathan.stocks at gmail.com> wrote:
>
> On Mon, Oct 26, 2009 at 11:44 PM, naman jain <namanvit at gmail.com> wrote:
> > Hi,
> >
> > I have a client server model in twisted, where the server spawns a thread
> (
> > basically a test script in python that runs for about 20 mins)
> > I want to track the progress of the thread, and send the progress to the
> > client back
> >
> > So, I write something like this in my server:
> >
> > parent_conn, child_conn = Pipe()
> > thread = Process(target = start_test.main_func, args=(SCRIPT_PATH,
> > TEMP_OUTPUT_PATH, self.output_name, child_conn))
> > thread.start()
> >
> > response = parent_conn.recv()
> > print response //prints like: initialization done
> > self.transport.write(response)
> >
> >
> > response = parent_conn.recv()
> > print response // configuration done
> > self.transport.write(response)
> >
> > .
> > .
> > .
> >
> > thread.join()
> >
> >
> > But the transport.write calls don't send at the same time. instead they
> wait
> > for the thread to finish (coz of thread.join) and then append all the
> > response and send it back; like "initialization doneconfiguration
> > done...done"
> > thereby defeating the purpose of creating a thread.
> >
> > How do I give the control to the reactor to write the data back, and
> still
> > keep the thread running?
> > or is there any other way these kinda of progress can be tracked ?
> > I am kinda stuck with this :(
>
> I'm no expert with threads, but I don't see any deferToThread in
> there, which is what people on this list tend to mention whenever
> threading comes up.  Perhaps you need to switch to the "twisted way"
> to create your thread?
>
> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.threads.html
>
> ~ Nathan
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20091028/cf2078f3/attachment-0001.html>