profile
viewpoint

Ask questionsIntegrate "direct color" / non-indexed 24bit color atop ncurses

first, a comment on nomenclature: these terms are subject to some disagreement. For the purpose of this bug, "TrueColor" will refer to "24-bit color making use of an 8-bit palette, which might be dynamic", "DirectColor" will refer to "24-bit color making use of 24-bit RGB specification", "256-indexed color" will be considered synonymous with "TrueColor", and "non-indexed" will be considered synonymous with "DirectColor". Note that these do not necessarily correspond to definitions in X11R6's "Visual types" (§2.5.5), nor do they necessarily correspond to NCURSES nor any terminal's terminology, nor do they necessarily correspond to any ISO 8613-6 (ITU T.416) definitions.

the story thus far....

NCURSES supports, on terminals which support it (inspect the output of tput colors), a 256-index palette composed of arbitrary 24-bit colors (8 bits each of R, G, and B). These colors can then be used as foregrounds and backgrounds in up to 32K colorpairs. NB: some terminals seem to indicate 64K colorpairs according to both tput pairs and COLOR_PAIRS, but will error out (#36) when init_extended_pair() is used with a pair above 32767 (sometimes it works fine; I've not yet figured this out; documentation seems to suggest a hard 32767 maximum based on the size of signed ints). These are known as "extended colors". This is all spelled out in the NCURSES FAQ, which is absolutely mandatory background reading. Note that the use of "color pairs" seems a pure NCURSESism, arising from terminals which did not allow the foreground and background to be specified distinctly:

Because some terminal types did not provide a way to set foreground and background colors independently, the concept of color pairs was introduced.

Reading the NCURSES documentation, it seems clear that backwards compatability, standards conformance, and wide hardware support (in terms of hardware diversity, not necessarily number of users) are major design goals. This is all noble. Furthermore (quoting again the NCURSES FAQ, 2019-11-14):

Using the ncurses 6 ABI, you have 256 colors, or 32767 pairs (the limit for a signed 16-bit number). That limit is good enough for realistic applications, which could not have that many character cells on a screen simultaneously (unless of course, using 1-pixel fonts to pretend to draw graphics, e.g., AA-lib). Since ncurses is used to draw text, that is not a valid issue.

and

That number of colors exceeds by a couple of orders of magnitude the ability of anyone to discern the differences and rely on those distinctions to aid them in viewing text.

Since outcurses seeks to be a blend of graphics and TUI, this doesn't really apply to us; these colors are a valid issue. I have furthermore demonstrated to my satisfaction that, for many images, it is impossible to achieve the visual quality of e.g. viu nor the TCT ("true color terminal") output of mpv without employing more than 256 simultaneous 24-bit colors. See #45 and #18 for more details.

Conclusion: NCURSES can and probably ought indeed stay above the fray until necessary escape sequences and semantics have been standardized and widely implemented. Unfortunately, we cannot achieve our goals, which include visibly appealing background graphics and animation effects in TUI programs on modern terminals optimizing for audience size rather than diversity of audience hardware. I hope this is an accurate and fair summation.

The NCURSES FAQ goes on about people with similar goals:

A small minority of terminals support an additional interpretation of SGR 38 and 48 alluded to in ISO 8613-6 as direct colour...Bypassing the limits which can be derived from the terminal database (and veering off into the swamp of hard-coded behavior), the idle spectator might ask “why not add a special ad hoc interface to (somehow) just do it? The problem is that you cannot “just do it”—both ncurses and s-lang do optimization to address their legitimate users.

The "small minority" here once again seems to prioritize "terminal diversity" over "terminal instances", which is NCURSES's prerogative. The "legitimate users" of outcurses are primarily people making use of modern terminals on workstation and mobile UNIX machines, quite possibly remotely over SSH, desirous of -- let's admit it -- eye candy. Our goals are silly in the context of NCURSES, but not silly for us. Breaking necessary optimizations, however, is very undesirable.

With that said, direct color can certainly be run effectively over an SSH connection. The following screencap is taken from a terminal SSH'd into my office workstation, making the trip over my personal VPN. The terminal emulator (xfce4-terminal) is running at a size of 239x64, pretty big:

mpv-tct-demo

this works just fine without any noticeable lag (admittedly, my terminal emulator is eating rather more CPU than I'd like). But that's a screen of a single, non-occluded output plane. Our goal is to make such vivid graphics available in a PANELed context.

So our challenge becomes:

  • make some manner of 24-bit DirectColor available to our users through our API, implemented on the backend using escape codes, external to NCURSES, while
  • continuing to support the NCURSES functionality we know and love, particularly the Panels API, disrupting NCURSES as little as possible

For instance, what happens when we move_panel() an entirely NCURSES-drawn PANEL across the width of an underlying PANEL in which we have used escape codes directly? NCURSES couldn't know how to restore the uncovered text, since we didn't color it through NCURSES. We would be responsible for the redraw...but that doesn't honestly seem unworkable, so long as we do know how and what to repaint?

A bigger issue would involve getting our colors into the draw cycle. NCURSES draws onto its virtual screen, then (possibly following many virtual screen updates) updates the physical one. We rely on this for any kind of decent, flicker-free performance. Would we be able to inject our escape codes into the NCURSES drawing routine, or would we need apply them immediately after a doupdate()? Furthermore, would we need apply them wholly without NCURSES routines--i.e., would we need move the cursor with termios rather than the more pleasant NCURSES routines? Probably. This might admittedly be a major problem. Before launching any major effort, some experimentation will be necessary here.

I think there's a good chance we can get this done, but it's not guaranteed. I'd like to get Thomas E. Dickey to take a look at this writeup. It'll almost certainly piss him off a little, but his feedback would be invaluable. Of course, the 16 million color section of his FAQ concludes:

Without significant revision, there is nothing more to discuss.

I wrote this bug up after last night reading Lexi Summer Hale's essay "on terminal control". It would probably be useful to reach out to her as well, likely pissing her off from a different direction:

you philistines won't stop using ncurses and oh my god WHY ARE WE STILL USING NCURSES IT IS THE TWENTY FIRST FUCKING CENTURY

So I'll reach out to both of these fine minds, and report back with any information they might provide. What we can do will really shape the "wow!" factor of our Thanksgiving demo.

dankamongmen/outcurses

Answer questions jart

@velartrill Lexi, I got a chance to read your blog. Your rendering code works really well. Flicking was the fault of the stdio library. It was dispatching too many system calls, according to strace. Thankfully the C library provides a solution to the difficulties it introduces. open_memstream seems to be just what you need. Give this code a try: https://gist.github.com/jart/b9104dd959f2dd65ecdcb5ce69d332e6 I added a couple more things too, that I hope you'll enjoy, such as variable monospace widths.

useful!

Related questions

No questions were found.
Github User Rank List