I'm making a javascript app which retrieves .json
files with jquery and injects data into the webpage it is embedded in.
The .json
files are encoded with UTF-8 and contains accented chars like é, ö and å.
The problem is that I don't control the charset on the pages that are going to use the app.
Some will be using UTF-8, but others will be using the iso-8859-1 charset. This will of course garble the special chars from the .json
files.
How do I convert special UTF-8 chars to their iso-8859-1 equivalent using javascript?
Best Answer
Actually, everything is typically stored as Unicode of some kind internally, but lets not go into that. I'm assuming you're getting the iconic "åäö" type strings because you're using an ISO-8859 as your character encoding. There's a trick you can do to convert those characters. The
escape
andunescape
functions used for encoding and decoding query strings are defined for ISO characters, whereas the newerencodeURIComponent
anddecodeURIComponent
which do the same thing, are defined for UTF8 characters.escape
encodes extended ISO-8859-1 characters (UTF code points U+0080-U+00ff) as%xx
(two-digit hex) whereas it encodes UTF codepoints U+0100 and above as%uxxxx
(%u
followed by four-digit hex.) For example,escape("å") == "%E5"
andescape("あ") == "%u3042"
.encodeURIComponent
percent-encodes extended characters as a UTF8 byte sequence. For example,encodeURIComponent("å") == "%C3%A5"
andencodeURIComponent("あ") == "%E3%81%82"
.So you can do:
For example, an incorrectly encoded character "å" becomes "Ã¥". The command does
escape("Ã¥") == "%C3%A5"
which is the two incorrect ISO characters encoded as single bytes. ThendecodeURIComponent("%C3%A5") == "å"
, where the two percent-encoded bytes are being interpreted as a UTF8 sequence.If you'd need to do the reverse for some reason, that works too:
Is there a way to differentiate between bad UTF8 strings and ISO strings? Turns out there is. The decodeURIComponent function used above will throw an error if given a malformed encoded sequence. We can use this to detect with a great probability whether our string is UTF8 or ISO.