[Twisted-Python] linux kernel 2.6.11-rc broke twisted process pipes

[James Y Knight]

On Feb 28, 2005, at 7:37 PM, Andrea Arcangeli wrote:
> It's never the inverse since pollin was always forced to be set, both
> for readers and writers, so a write fd was always returned as readable
> unconditionally.

I'm glad we all agree as to what the new behavior should be, but you're 
incorrect about the extent of the old brokenness. Simple tests prove 
that POLLIN was not forced to be true always:

On a debian 2.6.9-1-k7 kernel:

====
import os,select
r,w=os.pipe()
p=select.poll()
p.register(w, select.POLLIN|select.POLLOUT)

print select.select([w],[w],[w],0)
print p.poll(0)

os.write(w, "asdf")
print select.select([w],[w],[w],0)
print p.poll(0)

os.close(r)
print select.select([w],[w],[w],0)
print p.poll(0)
====

outputs:
([], [4], [])
[(4, 4)] (aka POLLOUT)
([4], [], [])
[(4, 1)] (aka POLLIN)
([4], [4], [])
[(4, 9)] (aka POLLIN|POLLERR)

> It's POLLERR not POLLHUP. POLLHUP is set only when the "writer" side
> disconnected and you're listening to a reader fd. POLLERR is instead
> returned when the "reader" disconnected and you're listening to a
> writer fd.

Right. Twisted treats all of POLLERR/HUP/NVAL the same, anyways.

> 671         if (events & (POLLIN | POLLRDNORM)) {
> 672                 if ((rpipe->pipe_buffer.cnt > 0) ||
> 673                     (rpipe->pipe_state & PIPE_EOF))
> 674                         revents |= events & (POLLIN | POLLRDNORM);
> 675         }
> The way I read it, even openbsd will report the fd as readable
> regardless if the channel is disconnected, if the buffer has something
> into it.

Nope -- Notice that rpipe and wpipe are backwards for the read and 
write fds. "rpipe" on the write fd won't ever have any data in it. 
(Except of course that pipes in BSD are bidirectional!). In BSD, 
everything is symmetric, so it'd be very difficult for a stupid bug 
like reporting the same status for both ends to occur.

> Are you reall sure it can't be called with pollreactor?
>
> Sorry for asking again but I really want to be sure select won't screw
> things up with >1024 fds open ;)

Yes, it will get called, because linux returns (returned?) 
POLLERR|POLLIN when the other side is closed. pollreactor doesn't 
assume the connection was lost immediately when POLLERR/etc is set, if 
POLLIN is also set, because POLLIN|POLLHUP means there is more data 
available to be read from the kernel buffer that was sent before the 
connection closed. And, since normal transport implementations of 
doRead actually do try to read, if it was lying about POLLIN, that's 
okay: the read syscall will fail, and CONNECTION_LOST will be returned 
from doRead.

James