Home FAQ Contact Sign in
perl.i18n only
Advanced search
July 2008
 123456 27
78910111213 28
14151617181920 29
21222324252627 30
28293031    31
 Jan   Feb   Mar   Apr 
 May   Jun   Jul   Aug 
 Sep   Oct   Nov   Dec 
2008 2007    
perl.i18n Profile…



  Printing non-ascii on the Win32 console         

Author: Bjoern Hoehrmann
Date: Jul 1, 2008 13:14


Writing an example script for Win32::MultiLanguage that simply lists
all supported code pages in the user's default language, I naturally got
all the umlauts in the german output messed up in the console. Wondering
how to fix that I thought there should be a simple way to say, e.g.,

binmode STDOUT => ':encoding(:terminal)' if -t STDOUT;

I only found the :locale sub-pragma in and and it
seems it does not work with binmode at all and would also fail on Win32
systems due to not setting the relevant environment variables. So I've
implemented a crude and simple solution like this:

use Encode;
use Encode::Alias;

sub terminal_encoding {
my $encoding;
Show full article (2.59Kb)
no comments
  Stripping out Unicode combining characters (diacritics)         

Author: Michael D Doran
Date: May 5, 2008 17:26

I'm trying to strip out combining diacritics from some form input using this code:

use CGI;
$query = CGI::new();
$search_term = $query->param('text');
$sans_diacritics = $search_term;
$sans_diacritics =~ s/\p{M}*//g;
#$sans_diacritics =~ s/o//g;
print qq(Content-type: text/plain; charset=utf-8
Show full article (2.18Kb)
  Locale::Maketext::Gettext 1.24 Released         

Author: Imacat
Date: Feb 25, 2008 11:40

Dear whoever is using Locale::Maketext::Gettext,

Locale::Maketext::Gettext 1.24 is released, after another year.
This release adds support for GNU gettext pgettext() as pmaketext(), to
translate messages in a particular context.

Thanks to Chris Travers> for the suggestion.
I was away from the GNU gettext development for too long.

GNU gettext pgettext() (and hence pmaketext()) is an important
improvement for GUI applications, which may have short menu items that
are ambiguous to translators without its menu context. If you do not
know what is "context", refer to:

11.2.5 Using contexts for solving ambiguities

Effectively, failure_handler_auto() differs from its
Locale::Maketext parent method from now on.

Please tell me if there is any problem. Your feedbacks are welcome.
Thank you.
Show full article (1.39Kb)
no comments
  RE: Problem processing UTF-8 strings from email         

Author: Martin Thurn
Date: Jan 14, 2008 05:24

I believe that format is RFC 2047 Mime Header Encoding, try the
Encode::MIME::Header module to decode it

- - Martin

-----Original Message-----
From: Neil Gunton [mailto:[email protected]]
Sent: Saturday, January 12, 2008 19:18
To: [email protected]
Subject: Problem processing UTF-8 strings from email

Hi all,

I am somewhat experienced with Perl in general, but absolutely no
experience dealing with UTF-8. I have a community journals website which

allows updates from users via email. I'm having trouble with emails that

contain Chinese characters encoded (I think) as UTF-8. The strings look
like this:

1 Comment
  Re: [perl-i18n] Perl locale information sources for server apps, and the CLDR         

Author: Guido Flohr
Date: Jul 2, 2007 23:58

John ORourke wrote:
> I suppose the *nix vendors keep their POSIX locale data up to date but I
> can't seem to find any info on how that process happens - who collects
> locale data, etc.

For the GNU libc locale definitions see

Imperia AG, Development
Leyboldstr. 10 - D-50354 Hürth -
no comments
  Re: Character Encoding (UTF-8) in PERL         

Author: oliver
Date: May 13, 2007 08:52

On Sunday 13 May 2007 17:14:43 you wrote:
> Krzysztof Krzy??aniak dijo [Thu, May 10, 2007 at 09:48:46AM +0200]:
>> after connection do query "set NAMES utf8"
>> or (from man DBD::mysql):
>> mysql_enable_utf8
>> This attribute determines whether DBD::mysql should assume strings
>> stored in the database are utf8. This feature defaults to off.
>> When set, a data retrieved from a textual column type (char, varchar,
>> etc) will have the UTF-8 flag turned on if necessary. This enables
>> character semantics on that string. You will also need to ensure that
>> your database / table / column is configured to use UTF8. See Chapter
>> 10 of the mysql manual for details.
>> Additionally, turning on this flag tells MySQL that incoming data should
>> be treated as UTF-8. This will only take effect if used as part of the ...
Show full article (3.12Kb)
  Problems with Perl Asian encodings?         

Author: Samuel L. Bayer
Date: May 10, 2007 12:23

All -

I'm having a problem that perhaps someone here can cast some light on.
For a very long time, a project I work on has been using GNU recode 3.6
to transcode a wide range of encodings into UTF-8, including some of the
more common Korean, Japanese and Chinese encodings (e.g., SJIS, gb2312,
EUC-KR). For efficiency reasons, we've been looking at moving to the
Perl Encode module, which we already use for windows-1252 because of
corruption issues with Arabic with GNU recode.

I compared a batch of about 120,000 documents for which I had both the
original and the output of GNU recode, and discovered a (relatively
small) number of differences, say about 4K documents. In almost all of
those cases, Perl recode appears to be inferior. The vast majority of
the differing documents are gb2132; however, many of the other Asian
encodings have sporadic problems. When I examine the documents for
differences, I typically find that Perl recode has introduced some stray
"unknown" characters at various points in the document, while the GNU
recode version is clean.
Show full article (2.09Kb)
1 Comment
  Character Encoding (UTF-8) in PERL         

Author: oliver
Date: May 10, 2007 00:04

Hello list,
we have used Debian (3.1) Sarge for almost 2 years on our serverand everything
was encoded in UTF-8.

When Debain Etch (4.0) came out we read that everything in Etch now defaults
to UTF-8 character encoding so we decided to upgrade from from Sarge to Etch.

Kernel 2.4.27
Apache 2.0.54
mysql 4.0.24
Perl 5.8.4

Kernel 2.6.18
Apache 2.2.3
mysql 5.0.32
Perl 5.8.8

Unfortunately after the upgrade the character encoding on our website was
messed up and it looks like the reason for that is PERL.
Show full article (1.97Kb)
1 Comment
  Locale::Maketext::Gettext 1.21 Released         

Author: Imacat
Date: Mar 27, 2007 23:37

Dear whoever is using Locale::Maketext::Gettext,

Locale::Maketext::Gettext 1.21 is released, after two years. There
are several improvements. The majors improvements are:

1. The lookup failure handling mechanism is completely rewritten to
follow the Locale::Maketext's way. $lh->fail_with() and $lh->{"fail"}
in Locale::Maketext is wroking in Locale::Maketext::Gettext now.

This is caused by the fact that I did not read the documentation
of Locale::Maketext throughly. ^^; But since the previous lookup
failure handling mechanism was completely different, it has successfully
avoided bug#33938 and bug#25877:

2. There is a fixed failure_handler_auto() method that override its
parent in Locale::Maketext, that fixes bug#33938 and bug#25877. You may
enjoy this fixed failure_handler_auto() if you use
Locale::Maketext::Gettext over Locale::Maketext. :p

3. Some encoding mess-up caused by my attempt to play magic in the
previous version was fixed.
Show full article (1.74Kb)
no comments