[Twisted-Python] adbapi, PostgreSQL and Unicode

[Remy Cool]

Michal Eldar wrote:
> Remy Cool wrote:
> 
>> Matt Goodall wrote:
>>
>>> Michal Eldar wrote:
>>>
>>>> I keep getting - Unhandled error in Deferred
>>>>     
>>>
>>> You should attach an errback to the deferred and handle, or at least
>>> log, the exception. That's what "Unhandled error in Deferred" means ;-).
>>>
>>>> I'm trying to perform an INSERT on my db (using adbapi with *pgdb*
>>>> module).
>>>> INSERT which comprises *Unicode* string
>>>> e.g. -
>>>> unicodeChar1 = u"Character=[\N{GREEK CAPITAL LETTER GAMMA}]"
>>>>
>>>> the db is defined to Unicode
>>>> *client_encoding* value is set to *Unicode* when checking its value in
>>>> *cp_openfun *called function
>>>> execute the same INSERT directly with *pgdb *works great
>>>>
>>>> Does anyone has experience with adbapi with Unicode db ?!?!

>>
>>
>> <snip>
>>
>> I also use pgdb with adbapi and unicode without problems. Insert's with
>> unicode strings should not be problem even if the database is set to
>> SQLASCII. The pgdb module inspects the string before sending it to the
>> postgres machinery. If it's a unicode string (isinstance('the string',
>> unicode) ) it's encoded to utf-8.
>>
>> I've submitted a patch to Darcy at pygresql.org for enhanced unicode
>> support for pgdb. This patch takes care of the client_encoding stuff and
>> makes it possible to fetch unicode strings from select queries.

> First of all - Thanks !!! It was a nice surprise to find responses to my
> quesion :)
> 
> I read about Unicode in the Twisted Archive and a link to the FAQ gave
> me the solution:
> I'm suppose to *encode* the unicode string before I'm passing it over...
> This was THE difference when working with Twisted.
> http://twistedmatrix.com/pipermail/twisted-python/2005-April/010198.html
> http://twistedmatrix.com/projects/core/documentation/howto/faq.html#auto25
> 
> Matt - I did try to attach an errback, but it failed to print the error,
> because it probably included some non-ascii chars... I don't know why
> Twisted don't show an appropriate error about this problem
> Remi - is it not possible to retrieve Unicode string from SELECT query
> with pgdb ? what is the behaviour nowadays?

The standard behaviour of pgdb is that unicode strings are automagically
encoded to utf-8 when inserting something in the database and you get
plain python strings for select queries. I've looked at the code of
other python database connectors and added (transparent) functionality
to pgdb so that it can it return unicode strings if you want it to.