Macos – Can Terminal.app be made to respect ANSI escape codes

ansi-escapeiterm2macosterminal

I notice that with the TERM environment variable set to either xterm or xterm-256color that Mac OS X’s Terminal.app utility respects most ANSI escape codes, as least when those escape codes pertain to changing text color.

For example:

echo -e "\033[0;31mERROR:\033[0m It worked"

Produces:

However, I’m more interested in the cursor location manipulation functionality afforded by ANSI escape codes. Unfortunately, that type of code doesn’t seem to work too well in Terminal.app, from what I’ve been able to gather. For example, what I want to do is something like this:

echo -e "\033[sHello world\033[uG'day"

ESC[s Saves the current cursor position, whereas ESC[u restores the last saved position. As a result of running the script above, I’d expect the five characters in “G'day” to overwrite the five characters of “Hello” after the cursor has been repositioned, producing the following:

G'day world

Indeed, this is exactly what I get with iTerm2.app, ConEmu for Windows (running either MinGW or MSYS Git’s copy of bash.exe), etc. What I see in Terminal.app, however, is this:

Hello worldG'day

Is there a reason for this, aside form Terminal.app simply lacking support for these codes? Is there a way to enable this functionality? Is there a chance I have something misconfigured? My TERM setting? Something else?

I’ve been searching everywhere, but haven’t found anything relevant to Terminal.app, specifically. I find it odd that it would support colored text via ANSI escape codes, but not cursor repositioning through the exact same technology. That seems like a fairly arbitrary subset of a fairly well-defined standard. That’s what makes me think I’ve got something misconfigured, rather than that Terminal.app is the one to blame… but, I suppose it’s possible that it simply can’t be done. (Probably one of the reasons iTerm2 exists in the first place?)

If anyone could shed some light on this situation, it’d be much appreciated!

UPDATE

So, I've done a bit more reading and experimenting, and discovered the following oddity:

After looking into n.m.'s answer below, I decided to write out the bytes that were being returned by tput to a file to see how they differed from the regular ANSI instruction.

$ echo "$(tput sc)Hello world$(tput rc)G'day" > out.bin
$ cat -e out.bin
^[7Hello world^[8G'day$

It appears that everything works as expected if I send it the sequences ESC 7 and ESC 8, but NOT if I send it ESC [s and ESC [u, which as I understand things is the more typical representation of the ANSI SCP and RCP codes (Save Cursor Position and Restore Cursor Position, respectively). Since putting ASCII decimal characters 7 or 8 next to an escaped octal byte representation is impossible (\0337 != ESC), environment variables can be used instead to avoid relying on tput:

$ esc=$'\033'
$ csi="${esc}["
$ echo "${csi}0;31mERROR:${csi}0m It worked."
ERROR: It worked.  # Color works, as before
$ echo "${csi}sHello world${csi}uG'day"
Hello worldG'day   # No dice
$ echo "${esc}7Hello world${esc}8G'day"
G'day world        # Success

I'm not sure why this is. If ESC 7 and ESC 8 are some kind of proprietary or custom code for ANSI SCP and RCP that has the potential to vary from terminal implementation to terminal implementation, it would explain to me why tput was created in the first place.

Unfortunately, I am unable to use tput for what I am currently working on, as I am not working exclusively in a bash environment. I am more curious as to how the raw bytes are interpreted from terminal to terminal, and more specifically, whether or not there's a way to get Terminal.app to respect the same ANSI escape codes that all of the other terminal emulators I've tried seem to have no problem with. Is that possible? At this point, I'm beginning to think it simply might not be, which is fine, but it would be nice to know for sure, and possibly also to learn the reason why.

Best Answer

Don't use ANSI codes. Use proper terminfo-based techniques. Xterm-based terminals are not specified to support all ANSI codes. Some do for compatibility, some don't.

Save cursor position sequence is given by the tput sc command and the restore cursor position is tput rc.

echo -e "$(tput sc)Hello world$(tput rc)G'day"

should work on any terminal that supports these sequences.

To see a readable representation of the supported sequences, use the infocmp command. The output might be quite long. If you are interested in sc and rc:

infocmp | grep --color ' [sr]c='

Disclaimer: tested on my Linux machine, don't have a Mac nearby.

UPDATE

Terminal.app is modeled after xterm and xterm is modelled after the VT100 terminal. VT100 did not implement CSI u and CSI s sequences, but used DEC private ESC 7 and ESC 8 sequences (source). Later VT models supported both CSI s/u and ESC 7/8, under different name and with slightly different functionality (source).

ECMA 48 doesn't seem to specify any save/restore cursor position sequences (source (PDF)), or I could not find them there. I don't know where CSU s/u come from. Their name in VT510 documentation suggests they are somehow connected with SCO. This source suggests they are in fact private sequences with no standard meaning. SUN terminals use SCI s to do a reset. It is probably an error to brand these two sequences ANSI.

Modern versions of xterm and other X11 terminal programs (konsole, rxvt...) do support both ESC 7/8 and CSI s/u, but the terminfo database only advertises ESC 7/8. Terminal.app apparently only supports ESC 7/8.

Font Effects

Code	Effect	Note
0	Reset / Normal	all attributes off
1	Bold or increased intensity
2	Faint (decreased intensity)	Not widely supported.
3	Italic	Not widely supported. Sometimes treated as inverse.
4	Underline
5	Slow Blink	less than 150 per minute
6	Rapid Blink	MS-DOS ANSI.SYS; 150+ per minute; not widely supported
7	[[reverse video]]	swap foreground and background colors
8	Conceal	Not widely supported.
9	Crossed-out	Characters legible, but marked for deletion. Not widely supported.
10	Primary(default) font
11–19	Alternate font	Select alternate font `n-10`
20	Fraktur	hardly ever supported
21	Bold off or Double Underline	Bold off not widely supported; double underline hardly ever supported.
22	Normal color or intensity	Neither bold nor faint
23	Not italic, not Fraktur
24	Underline off	Not singly or doubly underlined
25	Blink off
27	Inverse off
28	Reveal	conceal off
29	Not crossed out
30–37	Set foreground color	See color table below
38	Set foreground color	Next arguments are `5;<n>` or `2;<r>;<g>;<b>`, see below
39	Default foreground color	implementation defined (according to standard)
40–47	Set background color	See color table below
48	Set background color	Next arguments are `5;<n>` or `2;<r>;<g>;<b>`, see below
49	Default background color	implementation defined (according to standard)
51	Framed
52	Encircled
53	Overlined
54	Not framed or encircled
55	Not overlined
60	ideogram underline	hardly ever supported
61	ideogram double underline	hardly ever supported
62	ideogram overline	hardly ever supported
63	ideogram double overline	hardly ever supported
64	ideogram stress marking	hardly ever supported
65	ideogram attributes off	reset the effects of all of 60-64
90–97	Set bright foreground color	aixterm (not in standard)
100–107	Set bright background color	aixterm (not in standard)

2-bit Colours

You've got this already!

4-bit Colours

The standards implementing terminal colours began with limited (4-bit) options. The table below lists the RGB values of the background and foreground colours used for these by a variety of terminal emulators:

Using the above, you can make red text on a green background (but why?) using:

\033[31;42m

11 Colours (An Interlude)

In their book "Basic Color Terms: Their Universality and Evolution", Brent Berlin and Paul Kay used data collected from twenty different languages from a range of language families to identify eleven possible basic color categories: white, black, red, green, yellow, blue, brown, purple, pink, orange, and gray.

Berlin and Kay found that, in languages with fewer than the maximum eleven color categories, the colors followed a specific evolutionary pattern. This pattern is as follows:

All languages contain terms for black (cool colours) and white (bright colours).
If a language contains three terms, then it contains a term for red.
If a language contains four terms, then it contains a term for either green or yellow (but not both).
If a language contains five terms, then it contains terms for both green and yellow.
If a language contains six terms, then it contains a term for blue.
If a language contains seven terms, then it contains a term for brown.
If a language contains eight or more terms, then it contains terms for purple, pink, orange or gray.

This may be why story Beowulf only contains the colours black, white, and red. It may also be why the Bible does not contain the colour blue. Homer's Odyssey contains black almost 200 times and white about 100 times. Red appears 15 times, while yellow and green appear only 10 times. (More information here)

Differences between languages are also interesting: note the profusion of distinct colour words used by English vs. Chinese. However, digging deeper into these languages shows that each uses colour in distinct ways. (More information)

Generally speaking, the naming, use, and grouping of colours in human languages is fascinating. Now, back to the show.

8-bit (256) colours

Technology advanced, and tables of 256 pre-selected colours became available, as shown below.

Using these above, you can make pink text like so:

\033[38;5;206m     #That is, \033[38;5;<FG COLOR>m

And make an early-morning blue background using

\033[48;5;57m      #That is, \033[48;5;<BG COLOR>m

And, of course, you can combine these:

\033[38;5;206;48;5;57m

The 8-bit colours are arranged like so:

0x00-0x07:  standard colors (same as the 4-bit colours)
0x08-0x0F:  high intensity colors
0x10-0xE7:  6 × 6 × 6 cube (216 colors): 16 + 36 × r + 6 × g + b (0 ≤ r, g, b ≤ 5)
0xE8-0xFF:  grayscale from black to white in 24 steps

ALL THE COLOURS

Now we are living in the future, and the full RGB spectrum is available using:

\033[38;2;<r>;<g>;<b>m     #Select RGB foreground color
\033[48;2;<r>;<g>;<b>m     #Select RGB background color

So you can put pinkish text on a brownish background using

\033[38;2;255;82;197;48;2;155;106;0mHello

Support for "true color" terminals is listed here.

Much of the above is drawn from the Wikipedia page "ANSI escape code".

A Handy Script to Remind Yourself

Since I'm often in the position of trying to remember what colours are what, I have a handy script called: ~/bin/ansi_colours:

#!/usr/bin/python

print "\\033[XXm"

for i in range(30,37+1):
    print "\033[%dm%d\t\t\033[%dm%d" % (i,i,i+60,i+60);

print "\033[39m\\033[49m - Reset colour"
print "\\033[2K - Clear Line"
print "\\033[<L>;<C>H OR \\033[<L>;<C>f puts the cursor at line L and column C."
print "\\033[<N>A Move the cursor up N lines"
print "\\033[<N>B Move the cursor down N lines"
print "\\033[<N>C Move the cursor forward N columns"
print "\\033[<N>D Move the cursor backward N columns"
print "\\033[2J Clear the screen, move to (0,0)"
print "\\033[K Erase to end of line"
print "\\033[s Save cursor position"
print "\\033[u Restore cursor position"
print " "
print "\\033[4m  Underline on"
print "\\033[24m Underline off"
print "\\033[1m  Bold on"
print "\\033[21m Bold off"

This prints