<html><body>On 08:00 pm, exarkun@divmod.com wrote:<br />>On Fri, 12 Jan 2007 10:54:10 -0800, Scott Lamb <slamb@slamb.org> wrote:<br /><br />>>(1) create a test that likely fails by running many, many times. It'd  bog <br />>>down the test suite.<br />>>(2) create a test that might fail sometime, maybe.<br />>>(3) add instrumentation to the main code solely for the purpose of  the <br />>>test (break twisted's kneecap before the race).<br />>>(4) just say "clearly[*] if event A happens between lines X and Y,  then it <br />>>is handled improperly."<br />>><br />>>I prefer #4.<br />><br />>I think this actually varies from case to case.  For a problem where all<br />>of the relevant critical points cannot be hooked or mocked or stubbed or<br />>what have you, then a combination of #2 and #4 seems correct to me.  You<br />>should be able to demonstrate that a real problem does exist by adding a<br />>test which will fail eventually when run repeatedly (eg, using trial -u)<br />>and which runs for orders of magnitude longer without failing once a fix<br />>has been applied.<br /><br />Race conditions are tricky, but in every case it is worth _trying_ to mock/stub/hook the code as it already is to demonstrate it deterministically.  It isn't always possible with a reasonable amount of effort, but very often it is actually quite easy.  As you do it more, it gets easier to figure out the trickier cases.<br /><br />In any case, writing a test (which runs quickly) to be used with trial -u is a pretty low bar.  There are some extremely rare cases (mostly involving threading infrastructure) where those tests take a long time to fail, but even there it is usually possible to make them *succeed* reliably.<br /><br />>Always feel free to file the ticket before figuring any of this out, by<br />>the way.  It is still possible that it will strike someone else's fancy<br />>and the rest of the work will be taken care of for you. :)<br /><br />And of course this is always true.<br /></body></html>