NineChime forum

Furry stuff, oekaki stuff, and other stuff.

You are not logged in.

#1 01-29-2007 10:23:19

srothroc
New member

Unicode/Japanese support?

I just found out that it's apparently not possible to post comments (or titles, I assume) in Japanese. Is there any way to add/enable unicode/Japanese support for wacintaki?

Offline

#2 01-31-2007 05:12:47

Waccoon
Administrator

Re: Unicode/Japanese support?

It depends on the language pack (Japanese isn't available), and the database (UTF-8 is default, which does not support Japanese characters).

If you change the default charset for the database to Unicode, it should work.  Browser support for Unicode is pretty spotty, though, without proper charset declarations.  I haven't really looked into it much, though.

Offline

#3 01-31-2007 13:30:30

srothroc
New member

Re: Unicode/Japanese support?

Changed the default to SJIS, made an .htaccess file declaring SJIS the default... still no go.

これはテストだけ。。

It looks like the script is actually chewing up the Japanese and not spitting it out properly.

That's odd -- I can copy/paste the stuff from the board and it'll be shown properly here. I wonder if the post function here runs everything through a subroutine that translates the ASCII codes (I think that's what they are). I had to actually take out the semicolons after the numbers to get them to display as-is, since they were showing up as Japanese even in the code box.

Code:

&#12371&#12428&#12399&#12486&#12473&#12488&#12384&#12369&#12290&#12290

Last edited by srothroc (01-31-2007 13:32:36)

Offline

#4 02-03-2007 23:56:52

Waccoon
Administrator

Re: Unicode/Japanese support?

I've done some reading on this, and apparently, almost everything is very unfriendly when it comes to international support.  Most likely, htmlspecialchars() is mangling the Japanese characters.  Coders have to write their own PHP wrapper functions so functions like htmlspecialchars() won't destroy everything.  That's what happens in PunBB, with it's own .  If you look at the source code of this page, you'll see that all the Japanese characters are changed to HTML entities (with respective UNICODE numbers).  That's fine for a few words or a quote, but can bloat the hell out of a board filled with these kinds of characters, and make cut/paste problematic.  The proper way of doing it is to set a proper charset so the web browser will know it has to parse the text as UNICODE.  Of course, many web browsers still don't even support UTF-8 correctly, let alone UNICODE, to say nothing about machine issues like endian order.

To be honest, I don't know that much about "effective" text encoding.  Kevin made a Chinese language pack that seems to work just fine.  I don't know why BIG5 encoding would work but Shift-JIS wouldn't.

Offline

#5 04-29-2009 18:57:39

Tim
Guest

Re: Unicode/Japanese support?

I wanted to get Japanese support for this so I tried to adjust the settings in the english.php file to use $charset = "UTF-8"; and $metatag_language = "ja-JP";.
Amazingly it seems to fix this problem and allows me to comment in Japanese. So the problem is simply not setting the encoding correctly on the board like you said. It doesn't seeem to be a problem with everything else.

Perhaps for default, you could set the character set to UTF-8 to support more than just english characters.

I would like to make a Japanese friendly version since my oekaki will have Japanese visitors, so if i do end up translating the text, I'll share a translation file.

Cheers.
Tim.

Waccoon wrote:

I've done some reading on this, and apparently, almost everything is very unfriendly when it comes to international support.  Most likely, htmlspecialchars() is mangling the Japanese characters.  Coders have to write their own PHP wrapper functions so functions like htmlspecialchars() won't destroy everything.  That's what happens in PunBB, with it's own .  If you look at the source code of this page, you'll see that all the Japanese characters are changed to HTML entities (with respective UNICODE numbers).  That's fine for a few words or a quote, but can bloat the hell out of a board filled with these kinds of characters, and make cut/paste problematic.  The proper way of doing it is to set a proper charset so the web browser will know it has to parse the text as UNICODE.  Of course, many web browsers still don't even support UTF-8 correctly, let alone UNICODE, to say nothing about machine issues like endian order.

To be honest, I don't know that much about "effective" text encoding.  Kevin made a Chinese language pack that seems to work just fine.  I don't know why BIG5 encoding would work but Shift-JIS wouldn't.

#6 04-30-2009 01:57:13

Waccoon
Administrator

Re: Unicode/Japanese support?

@Tim

Yeah, supporting UTF-8 in the past was a pain, but these days every browser supports it properly.  I think it's about time to convert everything to UTF-8 by default.  That way, the traditional Chinese language pack would be the only exception that would require custom encoding.

I think I'll delay the release of 1.4.2 a few more days and see if UTF-8 support by default works properly in older browsers.  If everything is fine, I'll do that.  I just had someone else ask me about UTF-8 support so usernames can use international characters.  I wasn't aware that didn't work with the current code, because I was able to test some special characters with usernames, and it appeared to work.

Offline

#7 05-02-2009 02:42:08

Xod
New member

Re: Unicode/Japanese support?

On my oekaki board, I copied english.php to a separate file called english_utf8.php, with the charset set to utf-8, so that members who wish to read the board functions in english can still see the foreign characters properly, instead of strange symbols.

Offline

Board footer

Yep, still running PunBB
© Copyright 2002–2008 PunBB