i have this thing where i get older, but just never wiser

One of the long-term annoyances I’ve had with my Emacs configuration is that my preferred theme – solarized-dark from solarized-emacs – wasn’t loading when I started Emacs, despite my config containing code that, when I manually evaluated it, did load the theme. Also, when I manually activated solarized-dark, it ended up looking like hot garbage if I started up an emacsclient session on a terminal. Last night, while watching some mindless TV, I started poking at this again, and I eventually got to the bottom of the rabbit hole — not once, but twice!

Read on for how that worked out…

hi, it’s me, i’m the problem

The first thing I figured out was that at least a third of this problem was self-inflicted. For years – literally, over a decade – I’d had the following code as part of my config. It sets up the appearance of the default face, which is documented as Basic default face. Most of the other faces used inside Emacs are derived from this one, so setting this up has a huge impact on the appearance of the editor (and which is the basis of the comment about needing to set it early, which is probably a reference to some goofy behavior from long ago…)

;;; DEFAULT FACE
;;;; If you don't set this early on, sometimes things get wonky.
(if (eq system-type 'darwin)
    (set-face-attribute 'default t
                        :background "#000000"
                        :foreground "#ffffff"
                        :family "FiraCode Nerd Font Mono"
                        :height 180)
  (set-face-attribute 'default t
                      :background "#000000"
                      :foreground "#ffffff"
                      :family "Mono"
                      :height 161))

For the non-Emacs Lisp aficionados in the crowd, what this says is, on a Mac, use FiraCode Nerd Font Mono at 18pt, with a black (#000000) background and a white (#ffffff) foreground. Otherwise, on non-Macs, also use a white foreground on a black background, but use the font “Mono” at 16.1pt.

I finally figured out that the reason my attempts to load the Solarized theme during startup were failing was because I was explicitly setting these foreground and background colors on the default face. Some doc-diving and experimentation later, I ended up replacing the above chunk of code with this:

;;; DEFAULT FACE
;;;; If you don't set this early on, sometimes things get wonky.
(custom-set-faces
 '(default
   (
    (((type ns)) ;; mac-specific config
     (
      :family "FiraCode Nerd Font Mono"
      :height 180
      ))
    (t
     (
      :family "Mono"
      :height 161)))))

One of the key things I learned during this whole process – something that’s going to come more heavily into play in a minute – is that Emacs lets you define faces with conditional parts, based on various predicates.

In the previous code, depending on whether we were on a Mac or not, we were configuring the default face one of two different ways. Here, in this code, we’re defining one default face, one time, that will use font family FiraCode Nerd Font Mono at 18pt on a Mac, and font family Mono at 16.1pt on anything else. (Aside: For historical reasons, (type ns) means “on a Mac”, because Emacs thinks of MacOS as NextStep.; the final (t… part is the default that applies when none of the other predicates match.)

sometimes, i feel like everybody is a sexy baby

So, now, with the above changes in place, and (load-theme 'solarized-dark t) elsewhere in my config, when I start up a GUI Emacs, it loads the Solarized theme! YAY! But now, when I connect to that same Emacs process using the TTY client, I get an incredibly dark blue background, completely unusable for my purposes. BOO!

At this point, I started to wonder, “could I run one theme on GUI frames and a second, different one on TTY frames?” Some web searching ensued, and eventually I ran across this Emacs SE answer, which contained the magic that I needed.

Remember above, where I said you can define a face with conditional bits, so it looks different depending on various things? Turns out one of the things you can conditionalize on is “am I being displayed on a TTY or not?” — and since Emacs themes, like everything else in Emacs, are just data structures you can manipulate with Emacs Lisp, you can take a theme and programatically change all the faces in it so they’ve got the “only display on a TTY” bit added to them.

One of the other cool things you can do with Emacs themes is apply them on top of each other. So after you have your “only on a TTY” version of your second theme constructed, you load your original theme, and overlay the “TTY only” version on top of that, and BOOM one theme in the GUI, a totally different one in the TTY.

The code for that looks like this:

;;;; from https://emacs.stackexchange.com/a/68179
(defun rgmacs/copy-theme-tty (from-theme to-theme)
  "Copies all faces from `FROM-THEME' to `TO-THEME'.
Restricts to TTY frames only."
  (dolist (entry (get from-theme 'theme-settings))
    (when (eq (car entry) 'theme-face)
      (let ((face (nth 1 entry))
            (face-specs (nth 3 entry))
            (new-specs))
        (dolist (face-spec face-specs)
          (let ((display (car face-spec))
                (rest (cdr face-spec)))
            (cond
             ((listp display)
              (progn
                (setq display (cl-copy-seq display))
                (add-to-list 'display '(type tty))))
             ((eq display t)
              (setq display '((type tty)))))
            (add-to-list 'new-specs (append `(,display) rest))))
        (custom-theme-set-faces to-theme `(,face ,new-specs))))))

(load-theme 'solarized-light t t)
(deftheme solarized-tty)
(rgmacs/copy-theme-tty 'solarized-light 'solarized-tty)

;; now enable it (on top of currently active themes)
(load-theme 'solarized-dark t)
(enable-theme 'solarized-tty)

(ALL credit for this to rgemulla; this is their sole contribution to the Emacs Stack Exchange, which is kinda mind-blowing to me.)

i wake up screaming from dreaming

I was pretty happy with where I’d gotten things to — especially since I didn’t really set out to solve this problem, I just sort of noodled on it during the post-dinner TV veg time with my wife. But, as frequently happens, after a decent night’s sleep, I started to suspect I’d solved the wrong problem.

See, the Solarized theme – at least the Emacs one I’m using – is a 24bit color theme. And my shell/terminal/tmux/whatever config is set up such that I end up with TERM set to screen-256color. That’s why it was looking so bad when I attached a TTY client to a GUI emacs process.

So, can I get the terminal to support 24bit color? Turns out, there’s a Emacs SE answer for that too, which lead me to a gist, which I turned into this helper script. After generating the terminfo file, and setting TERM=xterm-24bit, emacsclient in my terminal looks just as good as my GUI emacs!

I do have one tiny remaining issue: once in a blue moon, I’ll run Emacs from an actual Linux console, not a terminal emulator. In that case, the 24bit terminfo file causes issues, and the 256color version still looks like crud. But…

everybody agrees

As I was writing up this post, I realized I can probably combine these two solutions — there’s a predicate for “minimum number of colors”, and I could use that to build a secondary theme to use on displays that only have 256 colors available. That way, I’d get the 24bit Solarized on both GUI and 24bit terminals, but something else, something usable, on the 8bit terminals.

Let’s try! an hour of two of futzing with Emacs data structures later…

Yep, this is totally possible. Here’s the updated theme munging code:

;;;; **modified** from https://emacs.stackexchange.com/a/68179
(defun rgmacs/copy-theme-24bit (from-theme to-theme)
  "Copies all faces from `FROM-THEME' to `TO-THEME'.
Adds condition to restrict to frames with 24bit color only."
  (dolist (entry (get from-theme 'theme-settings))
    (when (eq (car entry) 'theme-face)
      (let ((face (nth 1 entry))
            (face-specs (nth 3 entry))
            (new-specs))
        (dolist (face-spec face-specs)
          (let ((display (car face-spec))
                (rest (cdr face-spec)))
            (cond
             ;; if `display' is a list...
             ((listp display)
              (progn
                ;; ...assume it is an alist, make a copy of it, delete
                ;; any cells with the key `min-colors`, and assign
                ;; that to `display'...
                (setq display (assoc-delete-all 'min-colors (cl-copy-seq display)))
                ;; ...and then add a new `min-colors' cell that
                ;; demands 24bit color
                (push '(min-colors 16777216) display)))
             ;; ...otherwise if display is just `t`...
             ((eq display 't)
              ;; ...make it a single element alist with the
              ;; appropriate min-colors cell
              (setq display '((min-colors 16777216)))))

            ;; stick the updated display part together with the old
            ;; rest part and glob it onto the new theme spec
            (add-to-list 'new-specs (append `(,display) rest))))

        ;; finally build the new `to-theme'
        (custom-theme-set-faces to-theme `(,face ,new-specs))))))

;; load solarized-dark and convert into 24bit-derived version
(load-theme 'solarized-dark t t)
(deftheme solarized-dark-24bit)
(rgmacs/copy-theme-24bit 'solarized-dark 'solarized-dark-24bit)

;; load solarized-wombat as the "base" theme...
(load-theme 'solarized-wombat-dark t)
;; ...and layer the 24bit solarized-dark on top of it
(enable-theme 'solarized-dark-24bit)

In addition to the updates to the code to have it update any min-colors cells in the alists in the theme definition, I’ve swapped the order of operations. We load the “fallback” theme, solarized-dark-wombat as the “base”, and then apply the modified solarized-dark-24bit theme as the overlay.

I now have the best of both worlds: solarized-dark in the GUI and 24bit TTYs, and a usable solarized-wombat-dark fallback in older 8bit TTYs.

Days since I finally got my Emacs set up how I like it: 0