[Catalyst] View::TT FILTERS and utf-8

[Svilen Ivanov]

2005/12/26, Pedro Melo:
[...]
> > Both flat files and MySQL with the tables specifically set to utf8.
>
> Yeah, that's probably the problem rtight there :)
>
> Although your MySQL tables and connection is set to UTF8, the
> DBD::mysql driver does not set the utf8 flag on utf8 content.
>
> It's long story why it doesn't. As far as I could figure out,
> DBD::mysql is waiting for a solution in the DBI layer itself...
>
> Some relevant links:
>   - http://marc.theaimsgroup.com/?t=106321975700026&r=1&w=2
>   - http://marc.theaimsgroup.com/?l=msql-mysql-
> modules&m=111970179409036&w=2 (patch for DBD::mysql)

I've also had UTF8 difficulties when mixing TT and MySQL, so I'd share
my experience.

In first place, the UTF8 TT templates should be saved with Byte Order
Mark (vim editor does it,  kate - doesn't), otherwise the result will
be double encoded and scrambled.

Second, I don't want to use patched DBD::mysql so I decided to convert
text values coming from MySQL to using Encode::decode_utf8. This would
be a suicide if I didn't use O/R layer Rose::DB::Object in my model.
After a little tweeking [1] the text is correctly converted to UTF8.
The main benefit is that conversion is done on char/varchar/text
columns only and it is done on-demand (delayed until first access to
the text column).

Also, this [2] draft from Christian Hansen is good starting point for
UTF8-challenged developers (too bad for me that I had discovered it
after I solved my problems by myself :))

[1] http://sourceforge.net/mailarchive/forum.php?thread_id=9267247&forum_id=46239
[2] http://home.ngmedia.net/chansen/catalyst/catalyst-i18n.html

Happy 2006, Svilen