[Twisted-Python] Unjellying and circular references?

[Jasper Phillips]

On Fri, Dec 11, 2020 at 6:08 AM Jean-Paul Calderone <
exarkun at twistedmatrix.com> wrote:

> On Fri, Dec 11, 2020 at 8:19 AM Jasper Phillips <jasperisjaded at gmail.com>
> wrote:
>
>> I'm using perspective broker to transfer objects in a networked game,
>> which I'm having trouble unjellying -- the remote versions wind up with
>> dangling twisted.persisted.crefutil._Dereference instances, so don't match
>> the original objects.
>>
>
>> I'm seeing this for objects that have circular references to each other.
>> I've refactored things to mostly avoid circular references and sidestep
>> this, but have one remaining case where I find circular references mean
>> clearer code that I'm reluctant to refactor.
>>
>> Is there some trick I'm missing to avoid _Dereferences?
>>
>
> No, it's supposed to Just Work™ so you've found a bug in some part of the
> implementation.  If you can produce a minimal reproducing example then it
> may be worth a bug report.  But after that, I'd suggest investigating HTTP
> or AMP as a replacement for PB.
>

Here's a test case demonstrating the bug:

import sys
from twisted.persisted.crefutil import _Dereference
from twisted.spread             import pb, jelly

class RemoteCopyable( pb.Copyable, pb.RemoteCopy ):  pass
jelly.globalSecurity.allowInstancesOf( RemoteCopyable )
pb.setCopierForClassTree( sys.modules[__name__], pb.Copyable )

if __name__ == '__main__':
    # circular object ref
    cell           = RemoteCopyable()
    cell.link      = RemoteCopyable()
    cell.link.cell = cell

    # Mimic sending across network
    broker         = pb.Broker()
    serializedCell = broker.serialize( cell )
    remoteCell     = broker.unserialize( serializedCell )

    print( _Dereference is type(remoteCell.link.cell) )

This bug stems from twisted.spread.flavors.RemoteCopy.setCopyableState(),
which works fine for Python 2! But the UTF fix for Python 3 broke circular
references. It looks like this:

def setCopyableState(self, state):
    if _PY3:
        state = {x.decode('utf8') if isinstance(x, bytes)
                 else x:y for x,y in state.items()}
    self.__dict__ = state

Here is a simple tweak that fixes the problem:

def setCopyableState(self, state):
    if _PY3:
        for x, y in list(state.items()):
            if isinstance(x, bytes):
                del state[x]
                state[x.decode('utf8')] = y
    self.__dict__ = state


Basically the reference unwinding that takes place in
twisted.spread.jelly._Unjellier._unjelly_reference()'s
call to ref.resolveDependants() relies upon setCopyableStates()'s passed in
state being used directly, such that all matching references' __dict__
point to the same object.

Hope this helps clarify. Is there some more formal location than this list
that you'd like a bug report filed?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20201216/d8cc8dd5/attachment-0001.htm>