Unfortunately, the file.encoding property has to be specified as the JVM starts up; by the time your main method is entered, the character encoding used by String.getBytes() and the default constructors of InputStreamReader and OutputStreamWriter has been permanently cached.
As Edward Grech points out, in a special case like this, the environment variable JAVA_TOOL_OPTIONS can be used to specify this property, but it's normally done like this:
java -Dfile.encoding=UTF-8 … com.x.Main
Charset.defaultCharset() will reflect changes to the file.encoding property, but most of the code in the core Java libraries that need to determine the default character encoding do not use this mechanism.
When you are encoding or decoding, you can query the file.encoding property or Charset.defaultCharset() to find the current default encoding, and use the appropriate method or constructor overload to specify it.
To expand on the answers others have given:
We've got lots of languages with lots of characters that computers should ideally display. Unicode assigns each character a unique number, or code point.
Computers deal with such numbers as bytes... skipping a bit of history here and ignoring memory addressing issues, 8-bit computers would treat an 8-bit byte as the largest numerical unit easily represented on the hardware, 16-bit computers would expand that to two bytes, and so forth.
Old character encodings such as ASCII are from the (pre-) 8-bit era, and try to cram the dominant language in computing at the time, i.e. English, into numbers ranging from 0 to 127 (7 bits). With 26 letters in the alphabet, both in capital and non-capital form, numbers and punctuation signs, that worked pretty well. ASCII got extended by an 8th bit for other, non-English languages, but the additional 128 numbers/code points made available by this expansion would be mapped to different characters depending on the language being displayed. The ISO-8859 standards are the most common forms of this mapping; ISO-8859-1 and ISO-8859-15 (also known as ISO-Latin-1, latin1, and yes there are two different versions of the 8859 ISO standard as well).
But that's not enough when you want to represent characters from more than one language, so cramming all available characters into a single byte just won't work.
There are essentially two different types of encodings: one expands the value range by adding more bits. Examples of these encodings would be UCS2 (2 bytes = 16 bits) and UCS4 (4 bytes = 32 bits). They suffer from inherently the same problem as the ASCII and ISO-8859 standards, as their value range is still limited, even if the limit is vastly higher.
The other type of encoding uses a variable number of bytes per character, and the most commonly known encodings for this are the UTF encodings. All UTF encodings work in roughly the same manner: you choose a unit size, which for UTF-8 is 8 bits, for UTF-16 is 16 bits, and for UTF-32 is 32 bits. The standard then defines a few of these bits as flags: if they're set, then the next unit in a sequence of units is to be considered part of the same character. If they're not set, this unit represents one character fully. Thus the most common (English) characters only occupy one byte in UTF-8 (two in UTF-16, 4 in UTF-32), but other language characters can occupy six bytes or more.
Multi-byte encodings (I should say multi-unit after the above explanation) have the advantage that they are relatively space-efficient, but the downside that operations such as finding substrings, comparisons, etc. all have to decode the characters to unicode code points before such operations can be performed (there are some shortcuts, though).
Both the UCS standards and the UTF standards encode the code points as defined in Unicode. In theory, those encodings could be used to encode any number (within the range the encoding supports) - but of course these encodings were made to encode Unicode code points. And that's your relationship between them.
Windows handles so-called "Unicode" strings as UTF-16 strings, while most UNIXes default to UTF-8 these days. Communications protocols such as HTTP tend to work best with UTF-8, as the unit size in UTF-8 is the same as in ASCII, and most such protocols were designed in the ASCII era. On the other hand, UTF-16 gives the best average space/processing performance when representing all living languages.
The Unicode standard defines fewer code points than can be represented in 32 bits. Thus for all practical purposes, UTF-32 and UCS4 became the same encoding, as you're unlikely to have to deal with multi-unit characters in UTF-32.
Hope that fills in some details.
Best Answer
Edit: Please make sure your Office installation is up to date before trying the steps mentioned below. To update office, in any Office Application (e.g. Word, Excel), Click File->Office Account->Update Options->Update Now. Also, on Windows 7 systems, the below update should be installed as recommended by Microsoft.: https://support.microsoft.com/en-us/help/2729094/an-update-for-the-segoe-ui-symbol-font-in-windows-7-and-in-windows-ser
If you install updates, please restart the system and check if issue still exists.
I have been also trying to solve this issue, below is what I have learnt.
Until Office 2013, Office applications used to replace smiley character combinations [e.g. ":)"] with equivalent character from font Wingdings. The equivalent Unicode character code for the Wingdings smiley character was "004A". Please see the same in the below screenshot of Insert Symbol dialog box:
Outlook used to include formatting instructions to display this particular character using font "Wingdings", which is a common font on Windows systems and the smiley character appeared correctly on Windows systems.
Now, when the same message was opened in by an email app in phones, the app would discard instructions to use "Windings" font - obviously so because this font is not available in phones. Instead, the phone will display the character using the default font and displayed the real UTF-8 character for character code 004A, which is "J". Please check the complete list of UTF-8 characters here.
Microsoft recognized this issue and fixed the same in Office 2016. Now the Smiley characters are displayed using font "Segoe UI Symbols". This is a new font from Microsoft which has many additional characters (glyphs) for UTF-8 character set. Now the characters set used is standard UTF-8 so that it is displayed correctly in Windows as well as other devices that follow UTF-8 standards.
The UTF-8 character code for a smiley character is "1F60A", see here. The same character code is used in font Segoe UI Symbol in Office 2016, please see screenshot below:
Ok, all's well till here.
But why does my smiley character combination ":)" gets replaced by square character in Office 2016? Well, if you upgraded to Office 2016 from Office 2013, the Office 2013 AutoCorrect entries are updated to replace ":)" with character "1F60A", but the formatting instruction to use Segoe UI Symbol font are not updated - this is a bug in my view. Due to lack of instruction to use Segoe UI Symbol font, Outlook uses the font in use and no other font has glyph for character code "1F60A". Therefore, a square is displayed which is default behavior to display missing glyphs in Windows. The same happens if you paste new UTF-8 smileys in applications such as Notepad..Notepad uses Courier New font by default and Courier New does not have a glyph for "1F60A". To see the smiley in Notepad, change the Notepad font to Segoe UI Symbol (Format->Font). You will be able to see the Smiley characters in Notepad.
If you see the AutoCorrect dialog box in Word 2016 (File->Options->Proofing), you will find that the auto replacement entry for ":)" shows 2 asterisks, which means that the replacement character may not be displayed correctly.
Please follow the below steps for to correct AutoCorrect entries, you will need to do this for each smiley you want to correct or add [you can add new entries for smileys that you like or use frequently]:
Check in new message if Smiley shortcut is working now. Hopefully this works for you across desktop and phones. 😉