[Twisted-Python] t.trial.remote -- does anyone use it?

[Brian Warner]

> Trial currently does some black magic to support remote reporters,
> defined in t.trial.remote.  What use cases does t.trial.remote
> implement?  Does anyone actually use it?

Not me.

> Rumour has it that it was originally written to make buildbot easier
> to write.  However, I'm told that buildbot just parses stdio output.

Both rumors are correct. The buildbot currently just captures stdout/stderr
and parses the last few lines for the test-count summaries.

The original idea was to make a PB connection from the trial process to the
parent buildbot slave, and send fine-grained test results over that
connection. To accomplish this, we (slyphon and I) were planning to write a
Reporter class that would actually be distributed with the buildbot, then use
a trial option like --reporter-class=buildbot.trialreporter.RemoteReporter or
something. Another direction was to use PB running over stdin/stdout, using
the same scheme. The goal was to get a tuple of (testname, results,
exceptions, log.msg output, stdout, stderr, starttime, finishtime) for each
test, over on the buildslave, so that the buildbot status reporting could
give extremely fine-grained information about the tests which failed (perhaps
including full stack traces with local variables for the ERROR cases). The
most important thing to me is that we can get a list of test names and their
individual results, but having the trial.log text for each one separately
would make debugging test failures a lot easier (and the buildbot could
arrange to just mail the relevant parts of the logs to the person responsible
for that test, instead of sending the whole multi-megabyte logfile).

If I remember correctly, the extra Reporter functionality was there to give
buildbot enough feedback about tests starting/stopping to provide useful
status information (specifically to accurately update the ETA timing info: it
could eventually figure out that 45 tests out of 90 total means that it's N%
complete and therefore has an ETA of 2 minutes, etc). It's been a while since
I looked at the code, though.

I gave up on it at some point, mainly because the reactor cleanup routines
were getting in the way. (somewhat embarrassing because I was the one who put
many of those cleanup routines in there in the first place). Basically the PB
connection that was established by our RemoteReporter was torn down with a
vengeance by the clean-up-all-Selectables pass, flunking the first test and
condemning the rest to be run with no hope of ever delivering their status to
the buildslave. I decided to put off the whole clever machine-parseable
status thing until the trial's reactor cleanup had a way to let specific
selectables stick around.

So if that code is in the way, don't keep it around on account of the
buildbot. Having a pluggable reporter will be handy once reactor cleanup is
more convenient, but as long as that feature is retained in some form I think
we'll be able to figure out a way to get the status information to the
buildslave later.

cheers,
 -Brian

PS: another approach (implemented in trial at some point, I think, but I
don't believe I ever got buildbot to use it) was to jelly some status objects
and fling them out stdout in the hopes of being picked up by whoever launched
trial. This might have worked better, because one-way jelly over stdout
wouldn't have involved a Selectable (unlike two-way PB over stdio), so the
cleanup might not have stomped on it. However, I was uncomfortable with the
potential version skew of the code creating these jellied objects (coming
from twisted.trial) and the code that was hoping to interpret them (coming
from buildbot). The pluggable Reporter allowed the buildbot to supply the
code on both sides of the wire, which struck me as a better long-term
solution.