Here’s another tidy bit of Clojure that makes me happy…

Of course it’s easy to sort items in the natural order:

(sort [1 3 4 2]) ; => (1 2 3 4)

Or via a mapping to elements that sort in a natural order:

(sort-by count ["xx" "xxx" "x"]) ; => ("x" "xx" "xxx")

But how do we sort in a user-defined, fixed order?

For example, let’s say we have projects, and each project has a status which is one of:

(def statuses [:draft :in-progress :completed :archived])

(Note that those statuses are ordered in a logical, temporal progression.)

Here’s a quick helper function:

(defn fixed-order [xs]
 (let [pos (zipmap xs (range))]
 (fn [x y]
 (compare (pos x) (pos y)))))

To sort statuses, we simply do:

(def status-order (fixed-order statuses))

(sort status-order [:archived :in-progress :draft :completed])
;; =>
(:draft :in-progress :completed :archived)

Now let’s say we want to display all of our projects, sorted by their status:

(def projects
 [{:name "Paint the house", :status :completed}
 {:name "Update the blog", :status :in-progress}
 {:name "Fly to the moon" :status :draft}
 {:name "Master Clojure" :status :in-progress}])

(sort-by :status status-order projects)
;; =>
({:name "Fly to the moon", :status :draft}
 {:name "Update the blog", :status :in-progress}
 {:name "Master Clojure", :status :in-progress}
 {:name "Paint the house", :status :completed})

Maybe our app should group those sorted in the right order:

(->> projects
 (group-by :status)
 (into (sorted-map-by status-order)))
;; =>
{:draft [{:name "Fly to the moon", :status :draft}]
 :in-progress [{:name "Update the blog", :status :in-progress}
 {:name "Master Clojure", :status :in-progress}]
 :completed [{:name "Paint the house", :status :completed}]}

Ship it! 🚀

Following up from my previous article on using Emacs to maintain a plain-text ledger for use with plain text accounting software: I reconfigured my Doom Emacs to use the pre-packaged Ledger mode, and I can confirm that it is much nicer to use than the hledger mode I was using before. This could be because I hadn’t set up the hledger-mode configuration properly, but with ledger-mode I find the autocompletion works much better, and the M-q keystroke can be used to nicely line up the amounts in a transaction.

If I were to only have one computer, I’d use notmuch for email in Emacs. I might also import non-email stuff as notmuch messages so I can search everything in one place.

But, I now have 3 computers; 2 running macOS and one running (Fedora) Linux. Notmuch takes too much of my energy to keep synced between machines. So what about Mu4e? Mu4e is probably the “nicest” Emacs package for managing email, but it still requires a local synced copy of all my messages. This means configuring mbsync on all machines, etc.

It’s a lot, so I’ve decided to “simplify” things and use Gnus exclusively for email in Emacs. Gnus is weird and hard to get ones head around, but it’s built-in and it only requires a ~/.gnus.el file on each machine. Gnus works directly with my email service’s IMAP back end, so everything is the same everywhere, without having to think about it.

I don’t get the fancy search features of notmuch, and I don’t get an offline copy of my email store. Honestly, having local email is one of those “but what if…?” things that never need an answer.

With Gnus, I get fewer dependencies and not nearly as many “How do I keep this all in order?” issues.

Here at Irreal and in many other venues, you hear a lot about staying in Emacs. The hardcore among us treat having to leave Emacs as a fail. To outsiders, it can seem a little obsessive but it all goes back to having a uniform interface for dealing with text.

Here’s a delightful meme that expresses that impulse. As usual, the comments are really interesting. For a bit of context—as one of the commenters explains—with CUA the Ctrl+a Ctrl+x Ctrl+s selects everything, deletes it, and saves the file; definitely not what you get from Emacs.

It’s astounding how many of the commenters say that they’ve fallen victim to the same sort of thing. Others complain of deleting browser tabs with Ctrl+w. One said that he had moved even browsing into Emacs despite its being suboptimal. At least the key bindings do what you expect them to.

There’s nothing really profound in the post but it is amusing and, as I said, the comments are interesting. They show that those of us who would prefer to stay in Emacs are not alone.

For a while I have been concerned that I haven’t really been on top of my finances as well as I should be, and I decided to bite the bullet and do something about it. I’ve been looking to get much better visibility into my share portfolio, and make sure I track it properly so that if I sell any shares I can easily calculate captital gains and such at tax time. I am also keen to get a better idea of my spending patterns so that I can see where I may be wasting money.

On Linux, I now use Pop!_OS, which is dark by default. I find that ef-deuteranopia-dark looks better with my theme there. Meanwhile, on the Mac, ef-frost works for the light theme and ef-night for the dark theme.

Here’s what I do to make this as automatic as possible (F8 is mapped as the key to guggle from light to dark when on the Mac). These are Prot’s EF themes, including the function to switch.

    (cond 
     ((eq system-type 'gnu/linux)
      (ef-themes-select 'ef-deuteranopia-dark)
      )
     ((eq system-type 'darwin)
       (mapc #'disable-theme custom-enabled-themes)
       (ef-themes-select 'ef-frost)
       (setq ef-themes-to-toggle '(ef-frost ef-night))
       )
     )

Right after it’s too hard to learn, the second most frequent complaint about Emacs is how hard it is on your hands—especially your left pinky—because of its default keybindings. A lot of that has to do with the placement of the Ctrl and Alt keys on modern keyboards.

Sudoshred has a video that explains his system for avoiding Emacs pinky. The TL;DR is that on the Mac you should swap the left Ctrl and Alt keys. That way, you can press and hold Ctrl with your thumb and avoid stressing your fingers. Sudoshred says that since he began doing that his RSI problems have disappeared. The solution for other operating systems is similar.

He also suggests some hand stretching exercises and keeping your hands off the keyboard when you aren’t actually typing. Notice that his advice differs from the usual suggestion to swap Ctrl and Caps Lock, which is what I do.

Some of the comments claim that Emacs pinky is a made up problem harking back to older keyboards and that swapping Ctrl and Caps Lock is all you really need to do.

Some Irreal readers will beg to disagree. They say, for example, that they have to use Evil mode because of RSI problems. I take them at their word. On the other hand, I’ve been using Emacs virtually everyday, all day for almost 20 years and haven’t had any problems. And, by the way, I do everything wrong. These days, most of my tube time is on my couch with my laptop on my lap and I do keep my hands on the keyboard most of the time.

The takeaway is that I don’t know what the best method of avoiding Emacs pinky is. If you’re lucky like me it’s not a problem. If you’re like Sudoshred, a simple key swap will make the problem disappear. If you’re like many other people, you’ll have to give up the default keybinding and move to Evil mode or perhaps something like Xah Lee’s Fly Keys.

A commenter on my previous post about this stuff suggested using LibraryThing to deduplicate editions, so I thought I’d give it a go. I’m using Amy Hempel as the test case, because she’s only published a handful of books.

Or as OpenLibrary says: 27.

Let’s have a look at, say, the collection from 2006:

The documentation says that it’s supposed to return a list of “works”, not editions, but of course the data here doesn’t have much quality control. So here we have "The Collected Stories of Amy Hempel", "The collected stories of Amy Hempel", and finally "Collected Stories of Amy Hempel". For these, we have in total three different ISBNs (ISBN-10 and ISBN-13 are both listed in the output, apparently).

Let’s look up one of these in the LibraryThing API:

And actually… it looks like LibraryThing does ISBN-10 only? But it kinda looks like the LibraryThing de-duplication would work for that book. So that’s promising, even if it means doing a whole bunch of calls to LibraryThing.

Hm… Oh, OpenLibrary also lists this:

Which is a translated edition, but would also have gotten caught by the LibraryThing de-duplication. OK, I think I’m going to code up something and see what I get.

type type type

Viola!

That’s actually a pretty good list! OpenLibrary returned 27 publications, and after 17 LibraryThing API calls, we’re down to 12 works.

(There’s only five-ish of these that are “books by Amy Hempel” by any reasonable measure, but the rest are chapbooks, collections and collaborations, so it’s OK that they’re listed.)

Now, the number of LibraryThing API calls would make it pretty abusive to use this on a more prolific author, but as a proof of concept, it works.

(LibraryThing publishes data dumps of all this stuff, which would be more sensible to use, but that apparently costs $$$.)

So… uhm… could I use this to get “give all books published by the fifty authors I follow published since 2023”? I think that would be possible: For each author, ask OpenLibrary for the list, and then for each book published after 2023, do the LibraryThing deduplication to see whether it’s a book that also appears earlier on the OpenLibrary list? Yeah, I think that could work, but I guess the proof is in the programming.

Anyway! Since I’m blathering on about this here, amusingly enough the previous post landed on “Hacker News”. But it didn’t get enough points to get high enough on their home to totally ruin my statistics chart:

The last time Hacker News happened, there were so many visits that the “normal” days were just a single line of green pixels at the bottom, making the chart useless. I know, I know, I should use a logarithmic chart, but I just don’t like those.

(Or a discontinuous chart, even.)

There’s the expected interactions on Hacker News:

I like the idea of adding trigger warnings on non-hype LLM articles.

But also some useful stuff. For instance, there’s a real Emacs library to interact with the LLMs, so I didn’t really have to write my own shims. But it was trivial code (in my case; that project linked there looks quite ambitious), so whatevs.

And:

There’s an app that’s had to deal with the same issues.

So there you go.

I finally got around to re-recording and posting this talk on Clojure’s transducers that I gave last year to the Austin Clojure Meetup:

The talk walks through what transducers are, their benefits, where they make sense to use (and where they don’t), and how to implement them from scratch.

The title refers to an idea that really helped make transducers click for me: namely that there are two different conceptual models of transducers that I needed to apply (to different contexts) to really get it. (This reminded me a lot of the wave-particle duality of light in physics, which describes a single underlying reality in two different ways, with each way tending to prove more practical in analyzing particular scenarios).

The two models, then, are:

1. The encapsulated transformation model, where the transducer is an opaque representation of a (likely-compound) transformation of a collection, which merges with other transformations via the mechanical application of comp. And…

2. The constructive model, where we’re dealing in the underlying machinery of transducers (say, implementing a transducible process), and it’s helpful to conceptualize a transducer simply as a function from reducing-function to reducing-function (rf -> rf).

I hope that comes through clearly in the presentation.

If you don’t use transducers in your Clojure code today, I highly suggest you do—you will see benefits. And I’m convinced that the best way to get them to really click is to implement them from scratch, which the video will walk you through.

For your convenience, here are the slides for the presentation. The links are not clickable there (sorry), but are all included in the video description on YouTube.

If you have any questions, feedback, or need mentorship, feel free to reach out on Bluesky, Mastodon, or X and I’d be happy to help.

Here at the Irreal bunker we’re trying to recover from an, um, excess of enthusiasm for our Easter dinner with relatives. My torpor is preventing me from putting together a detailed post so here’s a short take on something I’ve been meaning to mention.
When I think about reading RSS feeds from within Emacs, the only things that come to mind are Gnus and Elfeed. You all know by now that I’m a huge Elfeed fan and can’t imagine using anything else.

A couple of weeks ago I can across a post from Ruslan Bekenev about a third option: Newsticker. The interesting thing about it is that it’s builtin. Those of you who like minimal configurations with as few packages as possible may want to check it out. If you think you might be interested, take a look at Bekenev’s post. It looks like a very nice application but as I said, I’m very happy with Elfeed and have no desire to try anything new.

• Upcoming events (iCal file, Org):
  • Emacs APAC: Emacs APAC meetup (virtual) https://emacs-apac.gitlab.io/announcements/ Sat Apr 26 0130 America/Vancouver - 0330 America/Chicago - 0430 America/Toronto - 0830 Etc/GMT - 1030 Europe/Berlin - 1400 Asia/Kolkata - 1630 Asia/Singapore
  • Emacs Berlin (hybrid, in English) https://emacs-berlin.org/ Wed Apr 30 0930 America/Vancouver - 1130 America/Chicago - 1230 America/Toronto - 1630 Etc/GMT - 1830 Europe/Berlin - 2200 Asia/Kolkata – Thu May 1 0030 Asia/Singapore
  • M-x Research: TBA https://m-x-research.github.io/ Fri May 2 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
• Emacs configuration:
  • Emacs For Beginners | Your first configuration (10:29)
  • Using use-package the right way (Reddit, Irreal, programming.dev)
  • Elpaca: Harnessing the power of an advanced Emacs package manager (14:14)
• Emacs Lisp:
  • pavodive/spiral: Learn Emacs Lisp Interactively - Codeberg.org (lemmy.ml)
  • Ep644 emacs lisp, text processing tutorial, validate HTML annotation pairs (01:04:10)
• Appearance:
  • Protesilaos Stavrou: Emacs: modus-themes version 4.7.0
  • New theme to reduce eye strain - `wood-theme`
  • Visual Emacs Studio - It's Emacs, but looks like VSCode (05:33)
• Navigation:
  • Emacs Tutorial 01 - Basic Movements (25:35)
  • James Dyer: Flex Matching with isearch
• Dired:
  • Diredc a.k.a. Dired orthodoxly (Reddit)
• Writing:
  • 日本人Emacsユーザはluatex-jaを使おう！(無音) (02:33)
• Org Mode:
  • The Zen of Task Management with Org (Reddit, Irreal)
  • How I use Org-mode to organize my life (19:11)
  • Org Speed Commands #emacs #orgmode #programming #coding (10:56)
  • Quickly Save Org Docs in Emacs #coding #programming (03:02)
  • org-mode babel asymptote 日本語化(無音)。luatex-jaを使おうの続きかな？！ (01:50)
  • Neat trees from org-ql to org-mode
  • ablatedsprocket: Doing Taxes in Emacs
  • Jeremy Friesen: Automating Adding Books to My Org-Mode Document
  • Hugo posting in Emacs
  • Charles Choi: Making an App, Looking for Beta Testers (Irreal) - org-protocol proxy for macOS
  • Org development:
    • org-archive-save-context-info: Allow parent ID to be saved in archived entry
    • org-archive: Add org-archive-finalize-hook
    • lisp/ox-html.el: Add file level option to manage headlines self link
    • org-element: Record blank lines at the beginning of document in org-data
• Denote:
  • Solving the Denote/Gollum links quandary (@mph@social.lol)
  • FYI: Denote version 4 released (Reddit) - breaking change: reorganized to core + extensions
• Completion:
  • Skeletons in the Closet
  • Bible Verse of The Day in Emacs — Now also on MELPA (Reddit) - update: insert verse, passage, or chapter at point
• Coding:
  • How to automatically make shell scripts executable
  • TIL magit-toggle-buffer-lock
  • Windows上のMagitでコミットメッセージを編集できるようになるまでが遅すぎるので調査してテキトーキャッシュで改善してみた
  • TIL: using triple dot (…) range operators in magit diff
  • Anand Tamariya: Emacs: Binary File Viewer (Reddit)
• Mail, news, and chat:
  • How to compose a mail message and Emacs and have your external mail program send it
  • Jeremy Friesen: Fixing a mu4e and Proton Bridge Foible
• Fun:
  • knowledge gives strength to the hands (Reddit) - see comments for explanation; using ASCII art to help plan projects, I think
  • 10PRINT inspired "Snowcrash" in Emacs (Reddit)
• AI:
  • ra-aid-el (interface for an open-source AI assistant)
  • Lars Ingebrigtsen: Perplexingly Book-Learned Emacs (HN)
  • Semantically linking LaTeX documents jumping to URLs, with GNU Emacs. (13:13)
  • Semantic linking of LaTeX documents by using GNU Emacs. (03:52)
• Community:
  • Abstract Heresies: Emacs and Lisp (Irreal)
  • Irreal: The Use Of Tools
• Other:
  • Understanding the Dynamic Module System of GNU Emacs
  • GTD + Emacs (03:59)
  • Lars Ingebrigtsen: A Book Tracking Package for Emacs
  • hsolg/emacs-departure-times-norway: displays departure times for public transit in Norway (@oantolin@mathstodon.xyz)
  • Yi Tang: Filter Ledger Transactions using Tags
  • Protesilaos Stavrou: Emacs: tmr version 1.1.0
• Emacs development:
  • emacs-devel:
    • Re: Trust API (was: bug#74879: 30.0.92; trusted-content-p and trusted-fi
    • Re: Versions of language grammar library - help needed with built-in major modes
    • good idea to use -O0 when building development branches
  • Flymake: bump package version to 1.4.0
  • Flymake: rework, simplify and address problems of recent change (bug#75841)
  • Rename 'treesit-auto-install' to 'treesit-auto-install-grammar'
  • * lisp/treesit.el (treesit-auto-install): New defcustom.
  • Bind new window layout commands under C-x w
  • (eieio-backward-compatibility): Set to `warn` (bug#77612)
  • ; * etc/NEWS: Move ts-modes after tree-sitter section.
  • Optionally add new empty file to Dired subdirectory
• New packages:
  • aws-athena-babel: Run AWS Athena queries from Org Babel (MELPA)
  • bank-buddy: Financial analysis and reporting (MELPA)
  • departure-times-norway: Description (MELPA)
  • doc-view-follow: Synchronize windows showing the same document (GNU ELPA)
  • inform-mode: Major mode for Inform 6 interactive fiction code (MELPA)
  • persist-text-scale: Persist and restore text scale (MELPA)
  • total-recall: Spaced repetition system (MELPA)
  • vc-jj: VC backend for the Jujutsu version control system (GNU ELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

You can e-mail me at sacha@sachachua.com.

Quick visualization tool for Binary data using C header definition.

1. Semantic parses the C header file and generates a type definition.
2. The type definition is passed to bindat-unpack along with the binary data.
3. The result is displayed using speedbar.

Conversions

Insert Binary Data

Emacs creates a multibyte buffer by default. While working with binary file (not in hexl-mode), this must be disabled.

Code: https://gitlab.com/atamariya/emacs/-/blob/dev/lisp/hexl.el

The gentleman behind Clojure Diary was kind enough to recently post a video based on a small comment I made on his YouTube channel: Clojure Diary - Elegant way of filtering maps based on key value pairs.

(Please check out his channel as he’s doing a great job regularly posting videos about his journey through Clojure!)

In my comment I used a little submap? function from my personal utilities library that I’ve gotten a lot of mileage out of the last year or two.

I thought I’d mention it here. It’s isn’t anything magical, but I think it’s part of my personal standard library from here on out:

(defn submap?
 "Are all of the key-value pairs in `m1` also in `m2`?"
 [m1 m2]
 (= m1 (select-keys m2 (keys m1))))

(submap {:a 1, :b 2} {:a 1, :b 2, :c 3}} ; => true
(submap {:a 1, :b 2} {:a 1, :b 4, :c 3}} ; => false

One of the things I use it for constantly is unit tests where I want to assert that a map contains multiple key-value pairs, but I don’t want to assume that a map contains only the specified key-value pairs (in the spirit of the open-world assumption)¹.

It also plays nicely with clojure.test, yielding useful test output which shows the full value tested against:

Fail in handler

expected: (utils/submap? {:status 200} (sut/handler {:method :get, :path "/"}))
 actual: (not
 (utils/submap?
 {:status 200}
 {:status 404,
 :body "Not Found",
 :headers {"Content-type" "text/html"}}))

Like many folks, I check my dotfiles into a git repo to share them across multiple machines. When I set up a newmachine I can simply clone that repo, run an install script that adds some symlinks, and get to work with my happypersonalized config.

My work machine needs to have some special configuration. I’m required to use certain security tools, there are somedev tools I only use at work, plus obviously I need to use my work GitHub account. The security team at my company has(wisely) asked me to keep the work-specific configuration out of my public GitHub, in order to prevent attackers fromperforming reconnaissance on our internal tooling and network configuration. Here’s how I managed that.

Git config

Our internal git setup uses a credential helper, a tool I’ve written about at Building a custom Emacs auth-source. In order to use it, we need this in our gitconfig:

...[core]sshCommand = /usr/local/bin/awesome-security-tool ssh...[credential]helper = /usr/local/bin/awesome-security-tool helper

I added this line near the to of my dotfiles/git.config (which issymlinked to ~/.gitconfig):

[include]    path = ~/.local/git.config

and then moved the work-specific config to ~/.local/git.config. Git is happy to silently ignore the include ifthere’s nothing at that path, so I can safely sync this config down to any machine even if it doesn’t need any localconfiguration.

Per-repo overrides

On the other hand, when I’m working on non-work repos on my work machine, I want to use my personal configurationinstead. I also want to make sure I’m not using the fancy credential helper,because the credentials it gets won’t work with my personal account.

This can be configured on a per-repo basis in the.git/config for each repo (or by using git config without the --global flag), but git has a neat feature that willconditionally include a config file based on the path to the repo. I check out my personal repos to a particulardirectory, so I at this config at the bottom of my gitconfig¹:

[includeIf "gitdir:~/dev/github.com/emoses/"]   path = ~/.gitconfig.personal

Where .gitconfig.personal looks like

[user]        email = webmaster@emoses.org[core]        # Override the ssh command from the work-specific config back to standard        sshCommand = /usr/bin/ssh

Emacs configs

My emacs also has some work-specific customizations, which I keep in a file called work.el that used to be checked into my dotfiles repo. My routine for loading all my customizations in my .emacs looks like this (if you’re reallycurious you can find the rest here) :

(defvar my:osx (eq system-type 'darwin))(my:load-config-file '("package-bootstrap.el"       (lambda () (if my:osx "osx.el" nil))               "evil.el"                ;; A bunch more files                "work.el" ;; Whoops, this was checked in, let's fix that                ))

I added a routine to load any configs from ~/.local/emacs if they’re present.

(defconst my:LOCAL_CONFIG_PATH (file-name-concat (getenv "HOME") ".local" "emacs"))(when (file-exists-p my:LOCAL_CONFIG_PATH)    (let ((local-el-files (directory-files my:LOCAL_CONFIG_PATH t "\.elc?$")))      (dolist (local-el local-el-files)        (load local-el)        (message "Loaded local config file: %s" local-el))))

So I moved my work.el out of my dotfiles repo to ~/.local/emacs/work.el and now I’m all set.

Seth Godin has an interesting post on the use and design of tools. His thesis is that some tools, such as a hammer, are immediately accessible to the casual user. Even experienced hammer users aren’t going to be significantly more efficient in their use.

The tools that you and I use—modern tools as Godin puts it—aren’t like that. They’re more complicated and very often difficult for the casual user to use effectively. Sadly, says Godin, these casual users don’t bother to learn how to use these tools well. They take the attitude that they’re too busy to waste time on learning, let alone mastering, their tools. Meanwhile those who do master these tools can be orders of magnitude more efficient with them.

So why does this post have the “Emacs” tag? It’s not a mystery. One of the most frequent complaints that I’ve been seeing against Emacs is that it’s too hard to learn and that, after all, real people have jobs to do and can’t afford to spend time learning things. Left unstated is how all those Emacs users, many of whom have produced truly astounding software, managed to learn Emacs.

Over and over again you see phrases like “VS Code just works out of the box.” That’s great for the casual user “too busy” to master difficult tools but failing to master your tools creates a debt that will come due, probably at a most inconvenient time.

As Godin puts it,

Don’t hold the hammer at the wrong end. And insist on software that’s worth the time it takes to learn.

Most important, once you find software that’s worth the time to learn, learn it.

None of this is to say that you must, or even should, use Emacs. Only that whatever editor you choose, take the time to learn it well.

Bastien Guerry has a longish post on how he uses Org mode for task management. If you’ve been around Org mode for any amount of time, Guerry will be familiar to you. He took over the maintainership of Org mode when Carsten Dominick decided he needed to get back to Astronomy, which is, after all, his profession. Since then, Guerry has stepped in when Org needed him and has continued to be a presence in the project.

His years of working on Org mode internals have given him a first rate understanding of how Org works and the best way to use it. Therefore his post on how he uses Org to manage his tasks is must reading for all Org users. Guerry will be the first to tell you that his system may not be what you need but his reasoning for why he made the choices he did is instructive for us all.

I don’t really use Org to manage my tasks to the extent that Guerry does. For me, my agenda is more of a display of my daily log although some tasks do find their way into my agenda. Regardless, I found his discussion of his organization of tags to be very helpful.

No matter how you use Org mode, you should take a look at Guerry’s post. He has a lot of good ideas that you may find useful.

This year is my third year submitting tax forms that were filled out programmatically with Emacs. There is way too much to cover in one blog post, so I will be glossing over things. As usual, the source code is available on Codeberg for perusal. In this post, I'm only covering the logic that went in to generating a 1040, but there are functions for generating a number of other forms as well. Fair warning, some of them are... brittle. The code is tailored specifically to what I needed on a given year, so some forms are missing logic related to other forms that I haven't used. It's getting better, though; I have gradually been making the code more robust over time. My workflow is broken down into four sections:

A week and a half ago, Bozhidar Batsov stirred up a storm by claiming that Emacs startup time doesn’t matter. There was much foaming at the mouth but I agreed with Batsov: Emacs startup time hardly every matters. When it does, it is in very special circumstances that, happily, most of us never experience.

At the end of my post on the matter, I said that if your Emacs startup time is lengthy, it probably indicates a problem with your configuration. Batsov apparently believes the same and in a new post offers some advice for addressing that problem.

He frames his discussion in terms of the use-package macro that most people use. The secret, he says, is to delay loading packages until they are needed. That makes sense because loading a single package won’t take long but loading, say, a hundred may add significant time to your init time. By deferring the loading, you speed up Emacs’ startup time at the cost of adding a tiny delay later when you first actually need a package.

Batsov’s suggestions are a bit complex but the TL;DR is to avoid the :preface, :init. and :config keywords. You should definitely read his post for the details. Ironically, he doesn’t follow his own advice because, as he says, Emacs startup time doesn’t matter.

TMR provides facilities for setting timers using a convenient notation. Lots of commands are available to operate on timers, while there also exists a tabulated view to display all timers in a nice grid.

• Package name (GNU ELPA): tmr
• Official manual: https://protesilaos.com/emacs/tmr
• Change log: https://protesilaos.com/emacs/tmr-changelog
• Git repositories:
  • GitHub: https://github.com/protesilaos/tmr
  • GitLab: https://gitlab.com/protesilaos/tmr
• Backronym: TMR May Ring; Timer Must Run.

Below are the release notes.

Version 1.1.0 on 2025-04-18

This version makes small refinements to an already stable package.

All of tmr-tabulated.el is part of tmr.el

The command tmr-tabulated-view, which produces a grid with timers+descriptions, used to be in a separate file. It now is part of the singular tmr.el to keep things simple. Users who were using (require 'tmr-tabulated) or similar will now get a warning. Simply load tmr instead.

Refined the behaviour of the tmr-tabulated-view command

When the command tmr-tabulated-view (alias tmr-list-timers) is called interactively, it uses the *tmr-tabulated-view* buffer just as it did before. Though it also evaluates the new user option tmr-list-timers-action-alist: it is a variable that controls where the buffer is displayed. The default value displays the buffer at the bottom of the Emacs frame and makes some other tweaks for usability.

Watch my video on the display-buffer-alist for further details on how to control the display of buffers: https://protesilaos.com/codelog/2024-02-08-emacs-window-rules-display-buffer-alist/.

The tmr-tabulated-view command is further revised to make it callable from a program. One scenario where we do this is to interrupt the termination of Emacs if there are running timers (more below).

TMR interrupts the termination of Emacs if it must

In the past, we did not have anything to prevent the termination of Emacs if timers were running: Emacs would simply shut down. Now we define the tmr-kill-emacs-query-function, which is added to the standard kill-emacs-query-functions: if there are running timers, it asks for confirmation before closing Emacs. To make it easier for users to decide how to proceed, it also pops up the list with all the timers (i.e. it uses tmr-tabulated-view from Lisp, as noted above).

The list view is easier to follow

The buffer produced by tmr-tabulated-view now uses more colours to make it easier to track the data it presents. These are all the faces it applies:

• tmr-tabulated-start-time: The time when the timer was started.

• tmr-tabulated-end-time: The time when the timer is set to end.

• tmr-tabulated-remaining-time: The remaining time.

• tmr-tabulated-acknowledgement: Whether the timer needs to be “acknowledged” after it ends (if it is marked as “acknowledged”, then it will not go away until the user confirms they have seen it).

• tmr-tabulated-description: The text describing what the timer is about.

The “Problem”

I have a lot of books and I want to keep track of what I “own.” I also have an Org-Mode 📖 document that has a set of books that I:

• have highlighted one or more quotes
• are on my shopping list
• are books I’ve read or am reading

However, I didn’t really have a concept of “and I own this book.” To address that issue, I set about the process of using a mobile application to track books I owned.

However, that meant I had two conceptual sources of data. Ownership and “a book of interest” if you will.

The Spark

Yesterday I saw A Book Tracking Package for Emacs by Lars Ingebrigtsen. In that post Lars shares his book tracking solution. It involves a bar code scanner and Emacs 📖 .

Curious, I set about getting that code working. And with a few adjustments, namely ignoring playing audio files during workflow steps, I was able to get it working.

The Pivot

I spent a bit of time hacking on it, but paused as I considered that what I wanted was a different.

Namely, given an International Standard Book Number (ISBN 📖) , I wanted to get the associated book information and either add the book to my Org-Mode document or update the entry in that document.

Also, I myself had a bar-code scanner, and wanted to use that as the thing that typed the ISBN into the buffer. I also considered that I wanted to intervene during the workflow; namely once I had retrieved the book information from the ISBN , I needed to decide if I added the book or updated an existing entry.

Establishing a Workflow

Given that I needed to make a decision, I reconsidered my workflow. Namely, I would grab a pile of ten or so books, and scan their ISBNs into a buffer. Then, with that pile, I would reconcile each book.

The idea being two-fold:

• It is a lot of fun to rapidly point and shoot a bar-code scanner.
• Looking at a list of ISBNs with no other context doesn’t tell me much.

Therefore, build a manageable list of ISBNs then flip to the other task. Namely reconciling the remote information with what I have.

Implementation

I drew some inspiration from Lars’s isbn.el; but ultimately set about my own implementation. Let’s walk through that.

Defining the Structure

I defined the jf/book structure. I would use this to create a common mapping for the data retrieved via ISBN process and my bibliography.

(require 'request)
(require 's)

(cl-defstruct jf/book
  "A basic representation of a book as it relates to my personal
bibliography.

Slots:
- label:     Used for comparing to labels from my bibliography; expected
             to conform to `jf/book-make-label' function.
- title:     The ubiquitous human readable identifier of a work.
- author:    The author(s) of the book; separated by \" and \".
- subtitle:  Usually the words after the colon of a title.
- tags:      A list of tags for the book; these are internal values.
- isbn:      The ISBN for the given book.  One of the unique
             identifiers.
- custom_id: The `org-mode' headline CUSTOM_ID property, used for
             helping find the headline."
  label
  label
  title
  author
  subtitle
  tags
  isbn
  custom_id)

Caching That Which I Already Have

Given that I had an existing set of books already stored locally, I wanted to cache that information for processing multiple ISBNs . I use my-cache-of-books to store that information.

(defvar my-cache-of-books
  (make-hash-table :test 'equal)
  "We use this as a cache of my bibliography entries that are books,
reprsented as an alist, and for each pair the `car' is a
`jf/book-label' (as formated by `jf/book-make-label') and the `cdr'
being an instance of `jf/book'.

See `my-cache-of-books/populate' for details on populating
this structure.

Normally I'd prefix this kind thing with \"jf/\", however I like how
this variable reads.")

Avoiding Magic Strings

As a matter of practice, I tend to avoid using “magic strings”, instead favoring defined values.

(defconst jf/bibliography/tag-own
  "own"
  "The tag used to indicate ownership of a work.")

(defconst jf/bibliography/tag-books
  "books"
  "The tag used to indicate that a work is book.")

Populating the Cache

As I was working through adding a book via ISBN , I didn’t want to rebuild a list of all of my books. The my-cache-of-books/populate function handles this logic.

  (defun my-cache-of-books/populate (&optional clear-cache)
  "Populates `my-cache-of-books' with my current books.

When CLEAR-CACHE is non-nil, clobber the cache and rebuild."
  (when clear-cache (clrhash my-cache-of-books))
  (when (hash-table-empty-p my-cache-of-books)
    (save-excursion
      (with-current-buffer
          (find-file-noselect jf/filename/bibliography)
        (org-map-entries
         (lambda ()
           ;; For some reason the org-map-entries is not filtering
           ;; on only items tagged as books.  Hence the
           ;; conditional.
           (when-let* ((tags
                        (org-element-property
                         :tags (org-element-at-point)))
                       (_
                        (member jf/bibliography/tag-books tags)))
             (let* ((title
                     (org-element-property
                      :title (org-element-at-point)))
                    (author
                     (org-entry-get
                      (org-element-at-point) "AUTHOR"))
                    (subtitle
                     (org-entry-get
                      (org-element-at-point) "SUBTITLE"))
                    (label (jf/book-make-label
                            title subtitle author)))
               (puthash label
                        (make-jf/book
                         :label label
                         :tags tags
                         :title title
                         :subtitle subtitle
                         :author author
                         :custom_id (org-entry-get
                                     (org-element-at-point)
                                     "CUSTOM_ID")
                         :isbn (org-entry-get
                                (org-element-at-point)
                                "ISBN"))
                        my-cache-of-books))))
         (concat "+level=2+" jf/bibliography/tag-books) 'file))))
  my-cache-of-books)

Fetch a Book from an ISBN

In jf/book-from-isbn I convert the ISBN into a jf/book data structure.

(defun jf/book-from-isbn (isbn)
  "Fetch the associated ISBN from Google API and return a `jf/book'.

  TODO: Instead of returning the book, consider taking a function that
  operates on the book.  There would need to be an inversion of behavior."
  (let ((book nil))
    (request "https://www.googleapis.com/books/v1/volumes"
      :params (list (cons "q" (concat "isbn:" isbn)))
      :parser (lambda ()
                (let ((json-object-type 'plist))
                  (json-read)))
      :sync t
      :success (cl-function
                (lambda (&key data &allow-other-keys)
                  (let* ((item
                          (aref (plist-get data :items) 0))
                         (volumeInfo
                          (plist-get item :volumeInfo)))
                    (setq book
                          (let ((title
                                 (plist-get volumeInfo :title))
                                (subtitle
                                 (plist-get volumeInfo :subtitle))
                                (author
                                 (s-join " and "
                                         (plist-get
                                          volumeInfo :authors))))
                            (make-jf/book
                             :isbn isbn
                             :label (jf/book-make-label
                                     title subtitle author)
                             :tags (list
                                    jf/bibliography/tag-own
                                    jf/bibliography/tag-books)
                             :subtitle subtitle
                             :author author
                             :title title)))))))
    book))

Here I Enter the ISBN

In jf/add-to-bibliography function is the entry point into this workflow. As the docstring states, I use the function to: “Append or amend to my bibliography the book associated with the ISBN.”

It is also the function where I intervene, namely determining if the given book is already in my bibliography or not.

(defun jf/add-to-bibliography (isbn &optional clear-cache force)
  "Append or amend to my bibliography the book associated with the ISBN.

When CLEAR-CACHE is non-nil, clobber and rebuild `my-cache-of-books'.

When FORCE ignore already existing ISBN; a bit of a refresh."
  (interactive (list (read-string "ISBN: ")))
  (let ((books
         (my-cache-of-books/populate clear-cache)))
    (when (or force (not (my-cache-of-books/contains-isbn-p isbn)))
      (let* ((book-from-isbn
              (jf/book-from-isbn isbn))
             (completed-value
              (completing-read
               (format "Match %s: " (jf/book-label book-from-isbn))
               books nil nil
               ;; Maybe we'll get a direct hit?
               (jf/book-label book-from-isbn))))
        (if-let ((book-from-bibliography
                  (gethash completed-value books)))
            (jf/bibliography/update-book-via-merge
             book-from-bibliography
             book-from-isbn)
          (jf/bibliography/insert-book
           book-from-isbn))))))

Insert a Book

When the book I fetched via an ISBN is not in my Org-Mode file, the jf/bibliography/insert-book function inserts a record. The inline comments describe some false starts and the why of the solution.

Restating the comments, I use the Org-Mode capture ecosystem to perform the heavy lifting of adding a book entry to the document. Namely by using variable binding and then calling org-capture; an approach that felt as though I leveled-up in my understanding of what was possible.

(defun jf/bibliography/insert-book (book)
  "Insert BOOK into bibliography.

Where BOOK is a `jf/book' struct."
  ;; NOTE: My first incarnation was to use `org-capture-string' which
  ;; meant adding to `org-capture-templates' a conceptually \"private\"
  ;; capture template.  That incarnation used \"?\" as the template
  ;; body.  However, when I'd test the behavior, I was getting an empty
  ;; entry.
  ;;
  ;; Instead I deconstructed the concise `org-capture-string' and
  ;; shifted towards binding `kill-ring' and using the \"%c\" capture
  ;; variable.  I like this approach as it eschews adding a useless
  ;; capture template while leveraging the power of the `org-capture'
  ;; ecosystem.
  (let* ((kill-ring
          (list
           (concat (jf/book-title book)
                   " :" (s-join ":" (jf/book-tags book)) ":\n"
                   ":PROPERTIES:\n"
                   ":CUSTOM_ID: "
                   (jf/denote-sluggify-title (jf/book-label book)) "\n"
                   (when (s-present? (jf/book-subtitle book))
                     (concat ":SUBTITLE: "
                             (jf/book-subtitle book) "\n"))
                   (when (s-present? (jf/book-author book))
                     (concat ":AUTHOR: " (jf/book-author book) "\n"))
                   ":ISBN: " (jf/book-isbn book) "\n"
                   ":END:\n")))
         (org-capture-entry
          '("B" "Book from ISBN Lookup"
            entry (file+headline jf/filename/bibliography "Works")
            "%c"
            :immediate-finish t)))
    (org-capture)
    (puthash (jf/book-label book) book my-cache-of-books)
    (message "Appended %s to bibliography" (jf/book-label book))))

Update a Book

When the book I fetched via an ISBN is in my Org-Mode file, the jf/bibliography/update-via-merge function merges and updates the document.

In essence, this is about finding the CUSTOM_ID and adding the own tag and ISBN to the book.

(defun jf/bibliography/update-via-merge (from-bibliography from-isbn)
  "Update a book FROM-BIBLIOGRAPHY with FROM-ISBN information.

Where both FROM-BIBLIOGRAPHY and FROM-ISBN are `jf/book' structs."
  (setf (jf/book-isbn from-bibliography) (jf/book-isbn from-isbn))
  (setf (jf/book-tags from-bibliography)
        (sort (seq-union
               (jf/book-tags from-bibliography)
               (jf/book-tags from-isbn)
               #'string=)))
  (save-restriction
    (widen)
    (save-excursion
      (with-current-buffer
          (find-file-noselect jf/filename/bibliography)
        (org-map-entries
         (lambda ()
           (let ((hl (org-element-at-point)))
             (when (string=
                    (org-entry-get hl "CUSTOM_ID")
                    (jf/book-custom_id from-bibliography))
               (progn
                 (org-set-property "ISBN"
                                   (jf/book-isbn from-bibliography))
                 (org-set-tags
                  (jf/book-tags from-bibliography))
                 (save-buffer)))))
         (concat "+" jf/bibliography/tag-books 'file))
        (message "Updated %s with ISBN %s"
                 (jf/book-label from-bibliography)
                 (jf/book-isbn from-bibliography))))))

Next Steps

In writing this, I’m thinking of a few next steps. There’s a TODO item in my docstring. Namely to refactor jf/book-from-isbn. Instead of returning a jf/book, I would pass a function which would receive a jf/book. I’d call that function on success.

Why consider that refactor? It touches back on that mobile application. I have already scanned 80 or so books. And I can export that application’s data and feed that into my process. I wouldn’t need to call the Google function but instead develop the method for creating a jf/book structure from that exported data.

And you may ask, where’s the code? It is a bit in flux. In part because I realized how trivial these next steps were. And I have this bar code scanner that demands I start “shooting books.”

Conclusion

With this new functionality, I’ve been able to add more functionality into my “bibliography.” It now serves as:

• A repository of quotes associated with their source
• A shopping list
• A reading tracker
• An inventory of my currently owned books

All in a rich, yet structured, format.

Further, the resulting workflow and functions helps me “chunk my work”, making it easier to review without simply mashing “accept.”

Postscript

I used the Emacs Read-eval-print loop (REPL 📖) to iterate on the solution. And all of the code was in my Bibliography file; usable via Org-Mode ’s tangle process.

Why do I mention this?

I found myself first writing prose, and thinking through Lars’ solution and how I might modify or extend that implementation in relation to the problem as I understood it.

Joshua Blais has an interesting post that looks at the idea of using Emacs for everything. That’s a familiar meme for us Emacsers, of course, but you don’t often see a careful, considered examination of the idea. Blais is an Emacs user so his post is, naturally, supportive of the idea but he does try to give both sides of the issue.

Blais’ main argument is built around the flow concept. Interrupting a knowledge worker’s flow can have devastating effects on their productivity. These interruptions include context switches and one study suggests that context switching and multitasking takes up around 60% of our working time. That seems high to me but whatever the value, it’s certainly significant.

One of the virtues of Emacs is that virtually all your text based tasks can be done in Emacs. That’s significant because the editor provides a uniform interface to all those tasks. Indeed, you can even slip it into other apps with packages like Emacs Everywhere that allow you to bring up an Emacs window to edit some app’s text area. For example, I use it with iMessage so that I can compose my texts in Emacs.

The advantage to doing most things in Emacs is that you avoid context switches and can better stay in the flow. A secondary benefit, as least for me, is that it provides a single set of muscle memories to deal with. As I’ve said before, I do better with a single set of keybindings so folding everything possible into Emacs is a win for me.

Blais also considers and dismisses the argument that Emacs takes so long to learn that you never regain the time “lost” to learning it. Even a few seconds thought is enough to recognize that idea for the silliness it is but Blais makes the point that if you aren’t using Emacs you’re still spending time learning other utilities that Emacs would otherwise handle. I know this first hand. Before I came to Emacs I was a long time command line user and spent years learning to use it effectively. Now I rarely use the command line; everything I need to do I can do from within Emacs.

Blais’ post is well worth a few minutes of your time whether or not you’re an Emacs user. Take a look.

As I was whining yesterday, it’s perplexingly difficult to find a (semi-)programmatic way of determining whether an author has written a new book. Or even manually in some cases. The best advice is, like, “Follow them on Goodreads” or something?

(To recap those two previous posts for those who hate clicking links — there’s plenty of databases of published books, but as far as I can tell, there are no databases of “published works”. That is, if the book was published today, would you consider it to be “a new work by this author” and not a new edition, a reprint, a translation, or whatever.)

But I wondered whether I could finally find something useful for an LLM to do? Sure, LLMs are plagiarism machines that use all of the world’s energy, but what can it do for me? So I signed up for the Perplexity AI, and an hour later:

Yes, I’m a prompt engineer now.

I’ve gotta give it to Perplexity — signing up for the API, adding some money and getting it to spew some data at me was totally painless. I created a little library called perplexity.el (and put it on Microsoft Github), but it’s totally trivial: It just does a url-retrieve request with some headers set, and you get JSON back.

So in the screenshot above, it’s listing all the books by David Sedaris it knows about, and also helpfully what kind of book it is.

That’s OK, and it’s of course trivial to just use that to list books that are newer than the last book I have from him. So I guess I could use this to automatically get all new books from the authors I’m interested in.

I also did a variation where I asked “list all the books by this author, but exclude books on this list:”, and then list all the books I already have by this author.

And as you can see, that, er, kinda changed what it’s outputting? Now it’s including a lots of plays that it didn’t before?

Typically enough, LLMs give different answers every time I ask, so this isn’t, you know, data to be relied on in any way. Sure, it’d be better if it was actual data instead of hallucinations and LLM navel lint, but it’s better than nothing.

The response times from Perplexity are all over the place — sometimes it responds within a couple seconds, and sometimes it takes a minute, but whatchagonnado.

Let’s test another author:

Amy Hempel. She’s awesome. I hit the m command (to list “missing” books), and:

All those three are books that exist! And that I don’t have.

I tried asking it again, and then it only listed two of these books — the non-repeatability of the results is annoying, but again, LLMs are toys, and if you get anything useful out of them, that’s nice, but your expectations can’t be low enough.

For giggles, let’s try this a “list missing books” on Megan Whalen a few times:

1st.

2nd.

3rd.

Oh well. I tried adjusting the temperature, but no go. I guess I could run the query several times and aggregate the results, but it’s already pretty slow…

After using this for a handful of authors, I’m now deep in debt. OK, perhaps not from the API usage, but because I’ve bought, like, fifteen books while just testing things out here…

I’ve added convenient commands to the search buffer to go to bookshop.org so that I can shop, as well as Goodreads to check whether the books actually exist. (They mostly do, but there are of course some hallucinations.)

And now I can also do that thing I wanted where there are certain authors that I track, and then query Perplexity for books they’ve released the last few years. Let’s see, I’ve marked a few authors, and:

Let’s repeat that:

And again:

*sigh*

Heh, if I remove Anthony Horowitz, I get:

Sure, sure… If I have just Stross and MacLeod, I get:

OK OK OK.

Is there a way to make Perplexity try, like, harder? (This is with Sonar Pro.) The prompts are in the package.

[Edit an hour later: OK, I guess I get it… Perplexity is based on web searches, and it just doesn’t like to do a lot of searches? So I’m going to have to loop through each tracked author and then aggregate the results.]

But… it does actually kinda look like even I managed to find something actually useful to use an LLM for, even if it’s literally janky? Actually?

Whodathunk.

[Edit a couple days later:]

I couldn’t help myself, and checked what results OpenAI and Gemini gave me. Of course they can’t tell me about “what’s new” because they don’t get “knowledge updates” very often, but I can ask them for missing books, for instance. Let’s do David Sedaris as an example again, and the query is basically “list all books by David Sedaris, but exclude books from this list:”, and then the list of the books I already have.

Here’s Perplexity (sonar-pro).

Here’s Gemini (gemini-2.0.flash).

Here’s OpenAI (gpt-4o).

I’ve put all the query stuff in query-assistant.el on Microsoft Github. I didn’t really expect them to be better — and they aren’t — so no big surprises there.

I recently wrote that Emacs startup time doesn’t matter and I got quite a lot of heat for it. I totally stand by everything I said there, but I acknowledge that different people have different use-cases and perspectives when it comes to this.

That’s why I’ve decided to share with you the #1 tip to speed up your Emacs - defer the load time of your packages (in other words - load them as late as possible, ideally when you actually need them for the first time). There are many ways to achieve this, but probably the easiest and most popular these days is to use use-package to organize your package configuration.

Unfortunately, using use-package the right way is not very obvious and there’s plenty of incorrect information about it all over the Internet. Here’s classic example of a problematic use-package usage:¹

(use-package projectile
  :init
  (setq projectile-project-search-path '("~/projects/" "~/work/" "~/playground"))
  :config
  ;; I typically use this keymap prefix on macOS
  (define-key projectile-mode-map (kbd "s-p") 'projectile-command-map)
  ;; On Linux, however, I usually go with another one
  (define-key projectile-mode-map (kbd "C-c C-p") 'projectile-command-map)
  (global-set-key (kbd "C-c p") 'projectile-command-map)
  (projectile-mode +1))

While this is technically speaking correct, the use of :init and :config means that the package will be loaded immediately.

You might be wondering at this point when to use things like :preface, :config and :init and you would be right to. As usual, the best answer is in the Emacs manual, and I’ll try to expand on it below.

Where possible, it is better to avoid :preface, :config and :init. Instead, prefer autoloading keywords such as :bind, :hook, and :mode, as they will take care of setting up autoloads for you without any need for boilerplate code. While the usage of preface in the wild is fairly rare, you’ll see a ton of usage of :init and :config for whatever reasons.²

For example, consider the following declaration:

(use-package foo
  :init
  (add-hook 'some-hook 'foo-mode))

This has two problems. First, it will unconditionally load the package foo on startup, which will make things slower. You can fix this by adding :defer t:

(use-package foo
  :defer t
  :init
  (add-hook 'some-hook 'foo-mode))

This is better, as foo is now only loaded when it is actually needed (that is, when the hook some-hook is run).

The second problem is that there is a lot of boilerplate that you have to write. In this case, it might not be so bad, but avoiding that was what use-package was made to allow. The better option in this case is therefore to use :hook, which also implies :defer t. The above is thereby reduced down to:

(use-package foo
  :hook some-hook)

Now use-package will set up autoloading for you, and your Emacs startup time will not suffer one bit. Nice, ah?

So, let’s return now to our original example and think how we can improve it. Our first instinct is probably to do something like:

(use-package projectile
  :defer t
  :init
  (setq projectile-project-search-path '("~/projects/" "~/work/" "~/playground"))
  :config
  ;; I typically use this keymap prefix on macOS
  (define-key projectile-mode-map (kbd "s-p") 'projectile-command-map)
  ;; On Linux, however, I usually go with another one
  (define-key projectile-mode-map (kbd "C-c C-p") 'projectile-command-map)
  (global-set-key (kbd "C-c p") 'projectile-command-map)
  (projectile-mode +1))

This is going to be useless, though, as projectile-mode will run at the end of the :config block forcing the package to be loaded. We can make things a bit better if we instruct the mode to be loaded only after Emacs’s initialization has finished:

(use-package projectile
  :defer t
  :init
  (setq projectile-project-search-path '("~/projects/" "~/work/" "~/playground"))
  :config
  ;; I typically use this keymap prefix on macOS
  (define-key projectile-mode-map (kbd "s-p") 'projectile-command-map)
  ;; On Linux, however, I usually go with another one
  (define-key projectile-mode-map (kbd "C-c C-p") 'projectile-command-map)
  (global-set-key (kbd "C-c p") 'projectile-command-map)
  :hook (after-init . projectile-mode)

Note that we’re using the name after-init instead of after-init-hook, as the hook is actually named. That’s done to spare you some typing, but I understand it might also be a bit confusing. You can enforce the usage of the full hook names like this:

(setopt use-package-hook-name-suffix nil)

So, what can we improve next? Ideally we should get rid of :defer, :init and :config:

(use-package projectile
  :custom (projectile-project-search-path '("~/projects/" "~/work/" "~/playground"))
  :bind-keymap (("C-c C-p" . projectile-command-map)
                ("C-c p" . projectile-command-map)
                ("s-p" . projectile-command-map))
  :hook (after-init . projectile-mode)

Much better!

Of course, you can’t always achieve this clean setup, but if you try you’ll get there 90% of the time!

So, to recap:

• Avoid the use of :init, :config and :preface whenever possible
• Most of the time you don’t need to use :defer
• Usually you should aim to activate minor modes only after Emacs’s main initialization has finished (otherwse :defer is pointless)

I’ll add here that less is more, even in Emacs. It’s usually a good idea to review the list of packages in your .init.el every few months and trim it from time to time. I used to be the type of guy who loads 100+ packages in their config, but these days I limit myself only to packages really improve my workflows.

If use-package still feels like black magic to you I can suggest the following:

• Macroexpand various use-package blocks in your config to see what’s the generated Emacs Lisp code. Here’s an example you can try:

(macroexpand-1
'(use-package projectile
  :custom (projectile-project-search-path '("~/projects/" "~/work/" "~/playground"))
  :bind-keymap (("C-c C-p" . projectile-command-map)
                ("C-c p" . projectile-command-map)
                ("s-p" . projectile-command-map))
  :hook (after-init . projectile-mode)))

;; macroexpansion
(progn
  (use-package-ensure-elpa 'projectile '(t) 'nil)
  (defvar use-package--warning78
    #'(lambda (keyword err)
        (let
            ((msg
              (format "%s/%s: %s" 'projectile keyword (error-message-string err))))
          (display-warning 'use-package msg :error))))
  (condition-case-unless-debug err
      (progn
        (let ((custom--inhibit-theme-enable nil))
          (unless (memq 'use-package custom-known-themes)
            (deftheme use-package) (enable-theme 'use-package)
            (setq custom-enabled-themes
                  (remq 'use-package custom-enabled-themes)))
          (custom-theme-set-variables 'use-package
                                      '(projectile-project-search-path
                                        '("~/projects/" "~/work/" "~/playground")
                                        nil nil
                                        "Customized with use-package projectile")))
        (unless (fboundp 'projectile-mode)
          (autoload #'projectile-mode "projectile" nil t))
        (add-hook 'after-init-hook #'projectile-mode)
        (bind-key "C-c C-p"
                  #'(lambda nil (interactive)
                      (use-package-autoload-keymap 'projectile-command-map
                                                   'projectile nil)))
        (bind-key "C-c p"
                  #'(lambda nil (interactive)
                      (use-package-autoload-keymap 'projectile-command-map
                                                   'projectile nil)))
        (bind-key "s-p"
                  #'(lambda nil (interactive)
                      (use-package-autoload-keymap 'projectile-command-map
                                                   'projectile nil))))
    (error (funcall use-package--warning78 :catch err))))

I know this looks a bit intimidating at first, but if you spent a bit of time reading the code you’ll see there’s nothing scary about it.³

• use-package also comes with profiler, you can set use-package-compute-statistics to t, restart Emacs and call use-package-report to see which packages are taking too much time to set up and what stage they’re at.⁴

That’s all I have for you today. Feel free to share other use-package tips in the comments!

So I’ve used restclient.el in the past to embed HTTP requests in readmes and other kinds of documentation to test login flows and APIs as I’ve been writing the corresponding servers.

But chances are you can’t use that package since you don’t use Emacs.

I found a command-line replacement – no thanks to my web searches, but thanks to alternativeto.net listings of httpie of all things.

My problem with cURL, Httpie, netcat, etc. for this is that you need to specify the host name as a command line argument. That makes sense to pipe HTTP responses into the next request from the terminal, but it doesn’t work well to tell a program to “please execute this self-contained HTTP request”.

Hurl to the rescue. Ignoring the unfortunate name, Hurl is, of course, written in Rust. It wraps libcurl and offers standard input and file-based processing of HTTP requests like so:

$ echo "GET https://example.com" | hurl 

Their Hurl file format does far more than I need for complex workflows. But it also does the simple things, i.e. single requests with payloads.

# Get home:
GET https://example.org
HTTP 200
[Captures]
csrf_token: xpath "string(//meta[@name='_csrf_token']/@content)"


# Do login!
POST https://example.org/login?user=toto&password=1234
X-CSRF-TOKEN: {{csrf_token}}
HTTP 302

This example demonstrates how you can parse the response and capture parts of it – be it JSON, plain text, or in this case HTML to grep the CSRF token. Check out the samples page for inspiration.

Soooo …. if you happen to use md-babel (which I haven’t formally announced, yet!), and execute this httpbin.org request, say from the Markdown document of this very blog post:

POST https://httpbin.org/post
Content-Type: application/json
{ "test":"foo","bar":"baz" }

… then you get this result inserted for you right below:

{
  "args": {}, 
  "data": "{ \"test\":\"foo\",\"bar\":\"baz\" }", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "28", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "User-Agent": "hurl/6.1.1", 
    "X-Amzn-Trace-Id": "Root=1-680093bf-142169f93e6208ec096d1024"
  }, 
  "json": {
    "bar": "baz", 
    "test": "foo"
  }, 
  "origin": "79.221.187.120", 
  "url": "https://httpbin.org/post"
}

That’s pretty cool and I’m going to be using that in the future to test my APIs in a way that non-Emacs-users can also enjoy, using hurl, Markdown, and md-babel.

Hire me for freelance macOS/iOS work and consulting.

Buy my apps.

Receive new posts via email.

I’m having issues with isearch!

I’ve been using fido-mode for a while now (I’m one of those weirdos who transitioned from the Vertico/Marginalia/Orderless stack back to an Emacs built-in) and have become accustomed to the flexible matching that fido offers. For example, I often shorten commands to find them, such as:

It is very common for me to search in a buffer for strings that contain a dash (especially now I am doing a lot more elisp programming), and I keep instinctively fido-shortening the search (so searching without the dash). Each time, isearch says no (or fails), so I have to awkwardly reach out for the “-” and go back and perform the search again.

What if there was a more flexible way to perform an in-buffer search? Well, of course, there is, and that is the isearch-regexp variants. I have seen some people just binding this to their main isearch because, why not? Most of the time, the search will be identical, and there is always the opportunity for a more flexible search using “.*”.

Let’s run an example. I would like to search for the first occurrence of use-package in my config. Let’s see how many keys we can get down to for an efficient search and how easy ergonomically they are to get to (excluding the possibility of some annoyingly similar matching lines in the config)

Firstly, let’s try isearch - you would have to type up to at least “use-pa” (probably).

Now for isearch-regex - you would type “us.*p” (probably) or of course just the same as isearch, so “use-pa”. This is better but a bit awkward to access the “*” and you would have to be quite aware of what you were searching for and the rough split of words.

If this were translated into a form of fido search, you would type at least “usep” (probably), a similar number of characters, but no awkward punctuation (so faster to type) and the find is more flexible and for me now, more familiar.

As far as I’m aware, by default fido uses some form of flex matching (but not quite flex), which has been good enough for me for a while now. So, how do I get this form of matching using isearch and it being in-buffer?

After a little investigation, there are packages out there in Emacs-land for some fuzzy searching, and the most relevant seemed to be flex-isearch. I’m not sure I quite got it working for me, but looking through the code, I thought I could distil some concepts into simple defuns. I learned that you can actually slot any search function at the backend of isearch, so let’s transform a normal search into something on steroids that would give us a flexible search, fido style!

(defvar flex-isearch-group-size 3
  "Number of initial characters to group together for more accurate flex searching.")

(defun flex-isearch-regexp-compile (string)
  "Convert a search string to a more intelligent flex-matching regexp.
The first `flex-isearch-group-size` characters are grouped together for more accurate matching."
  (let* ((parts (split-string string " " t))
         (compile-part
          (lambda (part)
            (let ((grouped (substring part 0 (min flex-isearch-group-size (length part))))
                  (rest (substring part (min flex-isearch-group-size (length part)))))
              (concat
               (regexp-quote grouped)
               (mapconcat
                (lambda (char)
                  (let ((c (char-to-string char)))
                    (cond
                     ((and (>= char ?A) (<= char ?Z))
                      (concat "[^" c "\n]*" c))
                     ((and (>= char ?a) (<= char ?z))
                      (concat "[^" c (upcase c) "\n]*[" c (upcase c) "]"))
                     (t
                      (concat "[^" (regexp-quote c) "\n]*" (regexp-quote c))))))
                rest
                "")
               "[^-_[:alnum:]\n]*")))))
    (concat
     "\\b"
     (mapconcat compile-part parts "[^-_[:alnum:]\n]+"))))

(defun flex-isearch-search-fun ()
  "Return the appropriate search function for flex searching."
  (if isearch-forward 'flex-isearch-forward 'flex-isearch-backward))

(defun flex-isearch-forward (string &optional bound noerror count)
  "Flex search forward for STRING."
  (let ((regexp (flex-isearch-regexp-compile string)))
    (re-search-forward regexp bound t)))

(defun flex-isearch-backward (string &optional bound noerror count)
  "Flex search backward for STRING."
  (let ((regexp (flex-isearch-regexp-compile string)))
    (re-search-backward regexp bound t)))

(setq isearch-search-fun-function 'flex-isearch-search-fun)

With my first attempt at this, the search at times seemed to settle on more candidates than I would have liked, but I thought, “I generally always know at least the first 2 or 3 characters that I’m searching for, at which point things get a little fuzzy” (pun intended). So, can I group the first few letters as part of the search, thus narrowing down the candidate list?

The answer is yes, yes I can.

The code above is a first attempt and with all these things, I shall play around with it and see if it works for me. At the moment, this seems to give me a more fido-iish feel when searching in-buffer with isearch, and no awkward search punctuation in sight!. So, using the examples above I can now find “use-package” in my config with a simple “usep”.

I shall report back in a later post to see if it has settled into my workflow, or whether there is an annoyance that I just cannot put up with so it will have to be discarded, along with my other myriad of Emacs experiments!

I just published the latest stable release of the Modus themes. The change log entry is reproduced further below. For any questions, you are welcome to contact me. I will now work to apply these same changes to emacs.git, so please wait a little longer for the updates to trickle down to you.

• Package name (GNU ELPA): modus-themes (also built into Emacs 28+)
• Official manual: https://protesilaos.com/emacs/modus-themes
• Change log: https://protesilaos.com/emacs/modus-themes-changelog
• Colour palette: https://protesilaos.com/emacs/modus-themes-colors
• Sample pictures: https://protesilaos.com/emacs/modus-themes-pictures
• Git repositories:
  • GitHub: https://github.com/protesilaos/modus-themes
  • GitLab: https://gitlab.com/protesilaos/modus-themes
• Backronym: My Old Display Unexpectedly Sharpened … themes.

4.7.0 on 2025-04-17

This release introduces many subtle stylistic tweaks to the “tinted”, “deuteranopia”, and “tritanopia” theme variants.

The modus-themes-list-colors command uses a tabulated list

This command and its modus-themes-list-colors-current variant help users see the colour values and semantic palette mappings defined by the given theme. In the past, their buffer was designed in the same spirit as that of the command list-faces-display, whereas now it is like the buffer of the command list-packages. Concretely, users may now sort by column. Do M-x describe-mode while in that buffer to learn about the available commands and their respective key bindings.

The “tinted” themes have slightly different colours

The overall feel of the modus-operandi-tinted and modus-vivendi-tinted themes is the same as before. Though in a side-by-side comparison between the old and new versions reveals lots of subtle differences. The general idea is to make the themes a bit more consistent by tweaking the foreground values to be more harmonious in combination with their background.

The “deuteranopia” themes are more consistently blue and yellow

These are the modus-operandi-deuteranopia and modus-vivendi-deuteranopia, which are optimised for users with red-green colour deficiency. In the past, these themes used blue and yellow hues wherever a concept of “success” versus “failure” had to be established. This approach is more generalised now, to include programming syntax highlighting and many other contexts. In short, the themes are more blue+yellow, while retaining their original feel.

The “tritanopia” themes are more consistently red and cyan

As above, the modus-operandi-tritanopia and modus-vivendi-tritanopia themes, which are optimised for users with blue-yellow colour deficiency, use a red+cyan palette in more places. Overall, they feel like they did before, only they are more consistent.

Miscellaneous

• Extended support for the icomplete faces that are coming in Emacs version 31.

• Added support for treemacs faces, courtesy of Rahul Juliato in pull request 121: https://github.com/protesilaos/modus-themes/pull/121. Rahul has assigned copyright to the Free Software Foundation.

• Added support for the tldr package.

• Extended support for adoc-mode. Thanks to Leilei332 for the contribution in pull request 137: https://github.com/protesilaos/modus-themes/pull/137. The change is within the ~15-line limit, meaning that the author does not need to assign copyright to the Free Software Foundation.

• Added support for my spacious-padding package, specifically the faces it can use when the spacious-padding-subtle-mode-line user option is enabled.

• Added support for the howm package.

• Extended support to the new faces of the transient package. More specifically, all those faces use the same colour for key bindings because the idea of colour coding keys (e.g. light yellow means something different than dark blue) does not work in practice when considering accessibility. Such semantics should not be limited to differences in colour: they should also have distinct indicators, such as ASCII or Unicode characters.

• Revised the avy package’s faces to only use one coloured background. The multiple coloured backgrounds have been a perennial problem for our accessibility requirements and have made the themes needlessly more complex just to support an edge case. With this simplified style, avy continues to work fine: it simply is less flamboyant. Other interfaces with avy-like model of interaction, such as optional extensions to the vertico and corfu packages, have these same changes, in the interest of consistency.

• Update the meow sample configuration in the manual. This package is not directly supported at the theme level because (i) I do not use it and (ii) it is very hard for an outsider to it to trigger the display of all of its faces in the right context. Without seeing how all of them look together, I cannot come up with a reliable design. The manual offers a “good enough” approximation.

• Broadened the support of the vterm faces to include the “bright” colours, while updating those that were already covered. Thanks to Edgar Vincent for informing me that some of the vterm faces were changed a while ago. This was done in issue 317 on the GitLab mirror: https://gitlab.com/protesilaos/modus-themes/-/issues/317.

• Revised the org-column-title face to inherit the fixed-pitch face if the user option modus-themes-mixed-fonts is non-nil. This user option makes it possible to have a buffer with proportionately spaced fonts (such as by enabling variable-pitch-mode), while keeping spacing-sensitive elements, like tables and code blocks, in a monospaced font.

  Thanks to pedro-nonfree for bringing this matter to my attention in issue 129: https://github.com/protesilaos/modus-themes/issues/129.

• Simplified the helper function modus-themes--retrieve-palette-value to make it more efficiently. Thanks to Basil L. Contovounesios for the contribution in merge request 60 on the GitLab mirror: https://gitlab.com/protesilaos/modus-themes/-/merge_requests/60.

• Reworded the minibuffer prompt of the modus-themes-list-colors command.

• Made Ivy and IDO subdirectories and “virtual” buffers easier to tell apart from matching text highlights.

• Included coverage for the auto-dim-other-buffers-hide-face of the package auto-dim-other-buffers

• Covered the built-in abbrev-table-name face.

I have been testing using Ledger-Cli to track my expenses, so far I have found the tagging system useful. In my ledger journal, each transaction is associated with a project, for example, the below transaction is assigned to project “2024 Monitor Stand”

2024-12-08 Screwfix
    ; project: 2024 Monitor Stand
    Expenses:HomeImprovement:Tools            £ 4.99 
    Expenses:HomeImprovement:PPE             £ 19.98 
    Expenses:HomeImprovement:PPE             £ 14.99 
    ; :refund:
    Assets:Amex

This constraint I came up with helps avoid meaningless spending on new shiny tools. Operationally, imposing this limitation on my book provides flexible ways of querying the data.

For example, bring up the transactions that do not have projects assigned to:

 
ledger reg exp and "expr" "not has_meta('project')" \
       --format "| %(date) | %P | %(amount) | %(note) |\n"

There is only one post that I forgot to add the project tag, so pretty good.

A bit of explanation of the ledger-cli query syntax

• exp: check only accounts contain ‘exp’, in the ledger’s convention, it is all expending accounts, i.e. Expense::*
• expr: invoke filters using expressions
• has_meta(‘project’): check if the transactions have the metadata key ‘project’
• and, not: logical operators
• –format: specify the output formatting

Another use case is counting the number of transactions per project. I use the number of purchased items as a proxy to gauge the project size.

 
ledger reg exp and "expr" "has_meta('project')" \
       --format "%(meta('project'))\n"  \
       | sort |  uniq -c | sort -bgr

The data shows the “2024 Loft Lights” project is by far the largest . That was a simple project by itself, however, since that was my first electrical project, I had to purchase a lot of stuff, 1.5mm cables, clamps, grommets, connectors, switches, sockets etc.

Finally, I have the “refund” tag so I can flag up the items to remind of myself to check if I received the refund fully.

 
ledger reg "expr" "has_tag('refund')" \
        --format "| %(date) | %P | %(amount) | %(note) |\n"

So far I enjoyed the plain text accounting using ledger-cli. The format and syntax are simple, and yet I can do complicated queries.

I use Proton Mail as my email provider. A while ago, I set out to read and send email from Emacs 📖 . I explored using both notmuch and mu; I’ve settled on mu in part because it was the one that I could easily setup deleting a local email and seeing it deleted upstream.

Things appeared to be working, but I kept noticing and ignoring the following message: [mu4e] Update process returned with non-zero exit code.

Today, during my lunch, I decided to tackle fixing the problem.

First the Hammer

I nuked my ~/Maildir directory and cache to hopefully “fix” the problem. In this aggressive delete, I accidentally deleted the associated cert.

I didn’t have notes on how to build that cert, so these are those notes:

• In Keychain Access app on MacOS select “System Roots” then select all roots and export a cert.pem; this is what I found in V.H. Belvadi — Setting up e-mail with Emacs and mu4e on macOS (Venkatram Harish Belvadi) This was inadequate.
• I then needed to run openssl s_client -starttls imap -connect 127.0.0.1:1143 -showcerts and copy the region starting with -----BEGIN CERTIFICATE----- and ending with -----END CERTIFICATE----- pre-pending that to the cert.pem. (as per Getting a self-signed certificate error with ProtonBridge and mbsync - Stack Overflow).

Then the Scalpel

With a “clean slate”, I started anew, but continued to see [mu4e] Update process returned with non-zero exit code in my *Messages* buffer. There wasn’t any more useful information.

I shelled out and ran the underlying mbsync <account> command and saw the following:

Error: far side refuses to store message 4775 from near side.
IMAP command 'CLOSE' returned an error: operation not allowed
IMAP error: malformed FETCH response from 127.0.0.1 (127.0.0.1:1143): unable to parse INTERNALDATE
Channels: 1    Boxes: 5    Far: +1 *0 #0 -0    Near: +0 *0 #0 -0

I searched the web for "IMAP command 'CLOSE' returned an error: operation not allowed" and stumbled upon IMAP CLOSE command not allowed · Issue #426 · ProtonMail/gluon. This looked to be my experienced problem.

However, the solution was not immediately obvious. So I searched for mbsync skip expunge for a mail folder and found email - Is it possible to ignore “Inbox” folder when using IMAP (mbsync)? - Unix & Linux Stack Exchange.

I changed my ~/.mbsync file replacing Patterns * with Patterns * !"All Mail" and ran mbsync <account>. No more errors.

In summary, Proton’s “All Mail” folder is a virtual folder, and thus disallows directly deleting from it. By changing the Patterns configuration value, I told mbsync to ignore that virtual folder.

Conclusion

My ~./mbsyncrc file is available on Github. And I used Setting up Protonmail in Emacs and V.H. Belvadi — Setting up e-mail with Emacs and mu4e on macOS (Venkatram Harish Belvadi) to help get me started.

I updated the Github issue that helped me work through the resolution, to provide a breadcrumb for other folks encountering the problem.

Charles Choi, someone whom Irreal often writes about, has a new app and is looking for beta testers. His app, scrim, is a way of making the org-protocol available to macOS users. The problem is that org-protocol depends on the ability to call emacs-client from within the invoking application but security rules on recent macOS versions make this difficult or impossible. Take a look at the scrim link for a fuller explanation.

Becoming a beta tester is really easy. All you have to is install TestFlight and choose the Scrim app. My experience with TestFlight has been outstanding. It automates virtually everything including keeping you up to date with the latest beta version.

The only app I ever want to export data from is Safari and I have some custom AppleScript code to handle that. Still, if I were starting today I would probably try to use org-protocol to solve the problem for any application supporting the share menu.

If you want to import data from other applications into Emacs or think that being able to do so would make your workflow smoother, take a look at Choi’s post and consider becoming a beta tester. If you are already a Captee (Choi’s previous application for this) user you should especially consider this.

I can tell you from my own experience with being a beta tester for Journelly, that it’s a rewarding and beneficial activity. You not only get an early look at an application but can also influence its final form. And, of course, you help the developer deliver the best possible app.

In 2013, I got tired of rooting through the book cases every time I bought a new book (to see whether I already had it, because who can remember those things).

So I bought an ISBN bar scanner, and whipped up some basic code in Emacs to keep track of what I’d bought. I’ve used it for over a decade now, and it works fine, but it’s super duper basic:

It’s basically just a buffer that lists all the authors, and then if you click them, you get:

A list of all the books from that author, as well as the registered publication date.

So that’s fine, but:

The data quality from the various ISBN lookup providers is pretty bad. I mean, it’s not only the inconsistency, but it’s also sometimes altogether wrong, because ISBN reuse is a thing, unfortunately.

In addition, I’ve been buying ebooks too, and these are totally outside this system. Whenever I buy a papery physical book, I blitz it with the bar code scanner, and it’s registered — it takes me literally two seconds or less. With ebooks, I’ve got no system at all, and I’m buying via three different online stores, so my book shopping headache has reestablished itself.

So I hear what you’re saying: Just bite the bullet and use Librarything for everything. And I say: No! So there! stomps foot

Voila!

I spent one day tinkering with this thing, and now I can edit the data to satisfy my CDO (it’s like OCD, only the letters are in the proper alphabetical order), and I can add ebooks (manually-ish, sigh). When the data is easily available like this, the obvious errors (like with the author name above) can be fixed very quickly.

I added more support to the isbn.el library to do Goodreads queries, too. Goodreads doesn’t have an API any more, so I have to do some web scraping, which sucks, but this increased the number of book covers by 180%, since the services that do have an API aren’t as comprehensive. So whatchagonnado.

Covers are nice.

I also added a new command to just list all the books in one big buffer:

Because I realised that having a buffer with authors, and then books by specific authors is all well and fine, but it’s really just much faster to C-s for a book name than to first find the author, and then find the book. I mean, I’ve not got a million books; it’s just er:

That many.

Hey, now I can get data out of this thing by just clicking on the columns… Let’s see… I’ve got 432 unread books! Sure! I’ll get to them all one day.

(Actually, looking over that list, it looks like I’ve forgotten to mark a large number of books as read… uhm… I’m guesstimating about 100? I should revise the data one of these days.)

Anyway, the code is on Microsoft Github. I’m not sure anybody would find this usable as is, but perhaps there’s some useful bits. At least the isbn.el library in there is somewhat helpful.

Denote aims to be a simple-to-use, focused-in-scope, and effective note-taking and file-naming tool for Emacs.

Denote is based on the idea that files should follow a predictable and descriptive file-naming scheme. The file name must offer a clear indication of what the contents are about, without reference to any other metadata. Denote basically streamlines the creation of such files or file names while providing facilities to link between them (where those files are editable).

Denote’s file-naming scheme is not limited to “notes”. It can be used for all types of file, including those that are not editable in Emacs, such as videos. Naming files in a constistent way makes their filtering and retrieval considerably easier. Denote provides relevant facilities to rename files, regardless of file type.

• Package name (GNU ELPA): denote
• Official manual: https://protesilaos.com/emacs/denote
• Change log: https://protesilaos.com/emacs/denote-changelog
• Git repositories:
  • GitHub: https://github.com/protesilaos/denote
  • GitLab: https://gitlab.com/protesilaos/denote
• Video demo: https://protesilaos.com/codelog/2022-06-18-denote-demo/
• Backronyms: Denote Everything Neatly; Omit The Excesses. Don’t Ever Note Only The Epiphenomenal.

Below are the release notes.

Version 4.0.0 on 2025-04-15

This is a massive release. There is one breaking change, which should be easy to adapt to: this pertains to the reorganisation of the project to separate the “core” of Denote from its “extensions”. The core is the denote package. Each extension now has its own package (details below).

Other than that, this version includes lots of new features for searching and linking as well as quality-of-life refinements. We have generalised the infrastructure for performing queries in the denote-directory and made the buffers with the search results more useful.

Take your time to read through this publication. I am writing it for you. Also remember that the most up-to-date resource for anything related to Denote is its manual. You are always welcome to contact me: https://protesilaos.com/contact. Or join the development on the Git repository.

As usual, special thanks to Jean-Philippe Gagné Guay for making high quality contributions to Denote since the beginning of the project ~3 years ago. Those will not always be headline features, but are important improvements to the underlying code base.

I mention contributions from Jean-Philippe and others in its context. Though I do not cover implementation details, otherwise this document will be the size of a book. This does not mean that they are no important though. Please consult the Git commit log for all the technicalities.

All the “extras” are in separate packages, including the Org dynamic blocks

In previous versions of Denote, we included some optional extensions as part of the denote package. These included the files denote-org-extras.el (Org dynamic blocks, among others), denote-journal-extras.el (streamlined for journaling), denote-silo-extras.el (working with multiple Denote silos).

The files denote-md-extras.el (Markdown extras) and denote-sequence.el (sequence notes, including Luhmann-style alphanumeric sequences) were also part of the project during the last development cycle, though they never made it into a tagged release.

All these are now available as standalone packages on the official GNU ELPA archive:

• denote-org: In the Emacs configuration file, replace all instances of denote-org-extras with denote-org.

• denote-journal: Replace denote-journal-extras with denote-journal.

• denote-silo: Replace denote-silo-extras with denote-silo.

• denote-markdown : Replace denote-md-extras with denote-markdown.

• denote-sequence: No changes to any of the defined symbols. Simply get the new package.

I will document each of these packages further below. The plan, going forward, is to maintain all the packages and coordinate their new versions.

More things in “core”

While the extras are moved out to their own code repositories, all other features are merged into denote.el. Those include everything that was in denote-sort.el and denote-rename-buffer.el.

• The “sort” mechanism is mostly for package developers. We use it extensively in our Org dynamic blocks, which are now part of the denote-org package.

• The denote-dired command (alias denote-sort-dired) is the only user-facing “sort” command we have always provided. It produces a fully fledged Dired buffer showing the results of the given search for file names. The matching files are sorted according to the user’s expressed preference. The details are described in the manual.

• The denote-rename-buffer-mode and all of its user options are unchanged. This mode automatically renames the buffer of a given Denote file so that it is easier to read it. Again, the manual covers the technicalities.

Users do not need to make changes, unless they are explicitly loading denote-sort-dired and denote-rename-buffer. In that case, they may just remove those calls: only denote needs to be loaded.

The denote-query-mode

Many of the features I will describe below produce search results via the built-in Xref mechanism. Xref performs a search with a Grep or Grep-like program, subject to the user option xref-search-program. The buffer those search results are displayed in runs the denote-query-mode. It supersedes denote-backlinks-mode.

The denote-query-mode supports the following:

• Results are shown in the context, with the exact match in highlight.
• Matches are grouped by file. Each file is a “heading”.
• Headings can be folded with TAB, just how it is done in Org buffers.
• The results can be used for further queries. Type C-h m (describe-mode) to learn about all the relevant commands.

We have had support for Xref since the original version of Denote. It now is more generalised to cover backlinks, query links, and denote-grep (more below).

Use query links for file contents or file names

Denote has always provided the option to link directly to a file with a given name by referencing its identifier. This can be done with the command denote-link, among a few others like it (always consult the manual of Denote).

In addition to these “direct links”, we also support “query links”. Those do not point to a file but instead trigger a search. The results are placed in a buffer that uses the appropriate major mode.

There are two types of query links:

• Query file contents: Use the command denote-query-contents-link to insert a query link at point for “file contents”. It perform a search inside files in the denote-directory and put the results in a denote-query-mode buffer.

• Query file names: Use the denote-query-filenames-link to insert a query link for “file names”. It performs the query against file names (not contents!) and puts the results in a dired buffer.

The display of the buffer with the query link results is controlled by the user option denote-query-links-display-buffer-action.

Query links are styled a little bit differently than direct links. Compare the denote-faces-link with denote-faces-query-link. Both should look okay with most themes.

Denote query links are supported as part of the denote: hyperlink type. They are available in all file types we define (per the user option denote-file-type) and should, in principle, work in any custom file type (advanced users can check the variable denote-file-types).

Backlinks now always show their context

In the past, the command denote-backlinks would produce a bespoke buffer showing a list of file names that included links to the current file (any file with the Denote file-naming scheme can have backlinks, by the way, including PDFs, videos, etc.). This buffer did not provide any additional functionality. We used to support the option to show results in their context via denote-backlinks-show-context. Those would be rendered in a standard Xref buffer.

The contextual results are now the default and sole option. This is because we have expanded the functionality of those buffers to use the denote-query-mode, as explained above. Plus, it makes our code base simpler.

Users will notice how backlikns look just like a query link for file contents. This is because backlinks are the original query links since day one of Denote.

Direct links to a file with matching contents

The command denote-link-to-file-with-contents allows users to produce a direct link to a file whose contents (not file name!) includes the given query.

Similarly, the command denote-link-to-all-files-with-contents generates a typographic list (bullet list) to all files whose contents match the given query.

The manual covers all linking commands in depth.

The essence of denote-search is part of denote

The denote-search package by Lucas Quintana uses the infrastructure of Denote to perform searches in file contents. We now provide its feature set as part of core denote.

We decided to do this since query links already introduced all of the requisite generalisations to denote-query-mode.

Users can rely on the commands denote-grep, denote-grep-marked-dired-files, and denote-grep-files-referenced-in-region.

The placement of these buffers is subject to the user option denote-grep-display-buffer-action.

This functionality was introduced in two pull requests by Lucas Quintana, 571 and 573, with further changes by me:

• https://github.com/protesilaos/denote/pull/571.
• https://github.com/protesilaos/denote/pull/573.

Lucas has assigned copyright to the Free Software Foundation.

I think this was a much-needed addition to the core of Denote. It complements denote-dired and query links.

Formatting of links with denote-link-description-format

The old user option denote-link-description-function is deprecated and superseded by the new denote-link-description-format. The new user option still accepts a custom function as its value, so the old behaviour should be retained.

What the new denote-link-description-format supports is an easier way to customise the description of a link by using format specifiers for common options. For example, users who only want to see the title of the linked file can do this:

(setq denote-link-description-format "%t")

The documentation of this user option covers all the format specifiers and further details.

Miscellaneous changes for all users

• The command denote-add-front-matter is superseded by denote-rename-file and related. Those renaming commands will add missing front matter or rewrite the modified lines of existing front matter. This is due to refinements made by Jean-Philippe Gagné Guay to the file renaming mechanism. We discussed this deprecation in issue 498: https://github.com/protesilaos/denote/issues/498. Also thanks to Samuel Flint for reporting an earlier problem with file name signatures: https://github.com/protesilaos/denote/issues/492.

• The user option denote-open-link-function specifies the function used by Denote to open the file of a direct link.

• The user option denote-org-store-link-to-heading can now be set to form generic context links without a PROPERTIES drawer and corresponding CUSTOM_ID. Set the value of this variable to 'context. Read its documentation for further details.

• Also about denote-org-store-link-to-heading, we have changed its default value to nil, which is what we were doing for most of Denote’s history. This means that, by default, org-store-link and anything building on top of it will create a link only to the current Denote file, like denote:IDENTIFIER, but not to the current heading within that file. To create links to the file+heading, set the value of this variable to 'id.

• The command denote-dired-link-marked-notes is an alias for denote-link-dired-marked-notes.

• The user option denote-sort-dired-extra-prompts control what denote-dired (alias denote-sort-dired) prompts for. It accepts either a nil value or a list of symbols among sort-by-component, reverse-sort, and exclude-regexp. The order those symbols appear in the list is significant, with the leftmost coming first.

• There is a new denote-sort-identifier-comparison-function variable which determines how identifier-based sorting should be done by default. It complements the existing denote-sort-title-comparison-function, denote-sort-keywords-comparison-function, denote-sort-signature-comparison-function. Thanks to Maikol Solís for the contribution in pull request 517: https://github.com/protesilaos/denote/pull/517. The change is small, meaning that Maikol does not need to assign copyright to the Free Software Foundation (though I believe the paperwork is done, anyway).

• Lots of refinements to the doc strings of individual variables and/or functions as well as the manual.

• Lots of other contributions to discussions and questions on the Git repository. Granted, these are not “changes” per se but are part of the development effort nonetheless.

• Made denote-get-path-by-id use denote-get-file-extension-sans-encryption instead of denote-get-file-extension. This fixes a bug where the extension is duplicated if it has an encryption component. Thanks to eum3l for the patch in pull request 562: https://github.com/protesilaos/denote/pull/562. The change is small, meaning that the author does not need to assign copyright to the Free Software Foundation.

• Same as above for denote--rename-file, which was done in pull request 557: https://github.com/protesilaos/denote/pull/557.

For developers or advanced users

The following have been added or modified.

• NEW Function denote-file-has-denoted-filename-p: Return non-nil if FILE respects the file-naming scheme of Denote. This tests the rules of Denote’s file-naming scheme. Sluggification is ignored. It is done by removing all file name components and validating what remains. Thanks to Jean-Philippe Gagné Guay for the pull request 515: https://github.com/protesilaos/denote/pull/515.

• NEW Functions denote-infer-keywords-from-files: Return list of keywords in denote-directory-files. With optional FILES-MATCHING-REGEXP, only extract keywords from the matching files. Otherwise, do it for all files. Keep any duplicates. Users who do not want duplicates should refer to the functions denote-keywords.

• MODIFIED Function denote-keywords: Returns an appropriate list of keyword candidates, while accounting for the value of the user option denote-infer-keywords. It now also accepts the optional FILES-MATCHING-REGEXP parameter.

• MODIFIED Function denote-directory-files: Returns a list of absolute file paths in variable denote-directory. It now accepts the optional EXCLUDE-REGEXP parameter.

• MODIFIED Function denote-format-file-name: Formats a file name. The way it treats its ID parameter has changed. Please read its doc string. Thanks to Jean-Philippe Gagné Guay for the pull request 496: https://github.com/protesilaos/denote/pull/496.

• ALIAS Function denote-retrieve-filename-keywords-as-list: This is a name that is easier to discover than denote-extract-keywords-from-path, because of the many other functions with the denote-retrieve-* prefix.

• MODIFIED Function denote-retrieve-filename-identifier: Extracts the identifier from FILE name, if present, else returns nil. To create a new one from a date, refer to the denote-get-identifier function. Thanks to Jean-Philippe Gagné Guay for the pull request 476: https://github.com/protesilaos/denote/pull/476.

• MODIFIED Function denote-get-identifier: Converts DATE into a Denote identifier using denote-id-format. If DATE is nil, it returns an empty string as the identifier. Also by Jean-Philippe in pull request 476 mentioned right above.

• MODIFIED Function denote-date-prompt: Prompts for a date, expecting YYYY-MM-DD or that plus HH:MM (or even HH:MM:SS). Can also use Org’s more advanced date selection utility if the user option denote-date-prompt-use-org-read-date is non-nil. It now has the optional parameters INITIAL-DATE and PROMPT-TEXT. Thanks to Jean-Philippe Gagné Guay for the pull request 576: https://github.com/protesilaos/denote/pull/576.

• NEW Function denote-retrieve-groups-xref-query: Accesses the location of xrefs for QUERY and group them per file. Limit the search to text files.

• NEW Function denote-retrieve-files-xref-query: Returns sorted, deduplicated file names with matches for QUERY in their contents. Limits the search to text files.

• NEW Function denote-retrieve-xref-alist: Returns xref alist of files with the location of matches for QUERY. With optional FILES-MATCHING-REGEXP, it limits the list of files accordingly (per denote-directory-files). At all times, it limits the search to text files.

• NEW Function denote-prepend-front-matter: Prepend front matter to FILE. The TITLE, KEYWORDS, DATE, ID, SIGNATURE, and FILE-TYPE are passed from the renaming command and are used to construct a new front matter block if appropriate.

• MODIFIED Function denote-rewrite-front-matter: Rewrites front matter of note after denote-rename-file (or related). The FILE, TITLE, KEYWORDS, SIGNATURE, DATE, IDENTIFIER, and FILE-TYPE arguments are given by the renaming command and are used to construct new front matter values if appropriate. If denote-rename-confirmations contains rewrite-front-matter, prompt to confirm the rewriting of the front matter. Otherwise produce a y-or-n-p prompt to that effect. Thanks to Jean-Philippe Gagné Guay for the pull request 558: https://github.com/protesilaos/denote/pull/558.

Denote “extensions” that are not in the denote package anymore

denote-journal integrates nicely with M-x calendar

The calendar can now highlight days that have journal entry. It may also be used as a date picker to view or write a journal entry for that day.

• Thanks to Alan Schmitt for reporting an issue with the calendar integration during development: https://github.com/protesilaos/denote-journal/issues/8.

• Thanks to Vineet C. Kulkarni for tweaking the identification of the journal keyword to be more robust: https://github.com/protesilaos/denote-journal/pull/4.

• Thanks to Honza Pokorny for fixing two small issues with the path expansion:

  • https://github.com/protesilaos/denote-journal/pull/5
  • https://github.com/protesilaos/denote-journal/pull/7

Other than that, the package is providing the same functionality as the discontinued denote-journal-extras.el.

• Manual: https://protesilaos.com/emacs/denote-journal.
• GitHub: https://github.com/protesilaos/denote-journal.

denote-org is almost the same as the discontinued denote-org-extras.el

The only addition to dynamic blocks the optional :not-regexp parameter. This is a regular expression that can further filter the results of a search, such that the matching items are removed from the output.

The official manual of denote-org covers the technicalities.

• Manual: https://protesilaos.com/emacs/denote-org.
• GitHub: https://github.com/protesilaos/denote-org.

Also thanks to Elias Storms for fixing a small issue with the “missing links” Org dynamic block, in pull request 486: https://github.com/protesilaos/denote/pull/486

denote-silo is the same as the discontinued denote-silo-extras.el

I have only made small tweaks to it, but nothing that changes the user experience.

• Manual: https://protesilaos.com/emacs/denote-silo
• GitHub: https://github.com/protesilaos/denote-silo

denote-markdown for some Markdown-specific extras

This package provides some convenience functions to better integrate Markdown with Denote. This is mostly about converting links from one type to another so that they can work in different applications (because Markdown does not have a standardised way to define custom link types). It also defines an “Obsidian” file type which does not have any front matter but only uses a level 1 heading for the title of the note.

The code of denote-markdown used to be bundled up with the denote package before version 4.0.0 of the latter and was available in the file denote-md-extras.el. Users of the old code will need to adapt their setup to use the denote-markdown package. This can be done by replacing all instances of denote-md-extras with denote-markdown across their configuration.

• Manual: https://protesilaos.com/emacs/denote-markdown
• GitHub: https://github.com/protesilaos/denote-markdown

Write sequence notes (or “folgezettel”) with denote-sequence

Users who want their notes to have an inherent structure can use denote-sequence. The idea is to have thoughts that naturally form sequences and are named accordingly. The sequence scheme is either numeric or alphanumeric. The manual of the package explains all the details.

• Manual: https://protesilaos.com/emacs/denote-sequence
• GitHub: https://github.com/protesilaos/denote-sequence

I had a lot of fun developing this comprehensive package during the winter holidays.

Thanks to Claudio Migliorelli, Kierin Bell, Mirko Hernandez for helping me fix some issues during development:

• https://github.com/protesilaos/denote/pull/518.
• https://github.com/protesilaos/denote/pull/528.
• https://github.com/protesilaos/denote/pull/540.
• https://github.com/protesilaos/denote/pull/541.
• https://github.com/protesilaos/denote-sequence/issues/2.

The consult-denote also gets a small update

This has always been a standalone package. I made the function consult-denote-file-prompt read the special-purpose variable denote-file-prompt-use-files-matching-regexp. This is related to commit e0f1d47 in denote.git, about issue 536 as reported by Alan Schmitt: https://github.com/protesilaos/denote/issues/536. The variable denote-file-prompt-use-files-matching-regexp is meant to be let bound and is for advanced users or developers.

Feature freeze at least until the end of April 2025

I will not develop new features or accept pull request for a couple of weeks. The idea is to focus on fixing any bug reports. We can then publish point releases quickly.

New features can be included after we are confident that the packages we have are okay.

Git commits

This is just an overview of the Git commits, though remember that there is more that goes into a project, such as the reporting of inconsistencies, discussion of new ideas, et cetera. Thanks to everybody involved! Plus, some commits are large while others are tiny.

~/Git/Projects/denote $ git shortlog 3.1.0..4.0.0  --summary --numbered
   470	Protesilaos Stavrou
    90	Jean-Philippe Gagné Guay
     6	Kierin Bell
     4	Alan Schmitt
     3	eum3l
     2	Claudio Migliorelli
     2	Lucas Quintana
     2	grtcdr
     1	Elias Storms
     1	Laurent Gatto
     1	Maikol Solís
     1	Octavian
     1	TomoeMami

The following are not accurate because they only reflect the changes after the reorganisation I made. But we have to start from somewhere.

~/Git/Projects/denote-journal $ git shortlog  --summary --numbered
    54	Protesilaos Stavrou
     2	Honza Pokorny
     1	Vineet C. Kulkarni

~/Git/Projects/denote-sequence $ git shortlog  --summary --numbered
    22	Protesilaos Stavrou

~/Git/Projects/denote-silo $ git shortlog  --summary --numbered
    17	Protesilaos Stavrou

~/Git/Projects/denote-org $ git shortlog  --summary --numbered
    15	Protesilaos Stavrou

~/Git/Projects/denote-markdown $ git shortlog  --summary --numbered
    11	Protesilaos Stavrou

Nearly two years back I wrote a utility called Captee that uses the native macOS Share Menu ( ) to share a link or text clipping in either Markdown or Org format. The idea was that any macOS app that supported the Share Menu could take advantage of this. Among the features that Captee supported was the ability to share content via Org Protocol, a custom URL scheme devised to import content into Emacs.

However, for Org Protocol to work, it required that there be a way to transport the org-protocol:// URL request to Emacs on macOS. Org Protocol’s original implementation presumed that emacsclient would do the lifting here, where the sharing app would be able to call emacsclient with the org-protocol// request as its argument. This was however a regrettable design decision made over a decade ago as recent versions of macOS now have a default sandbox policy that prevents an app from calling an executable like emacsclient outside of its sandbox.

The Mitsuharu Yamamoto fork of Emacs got around this restriction by supporting the Org Protocol custom URL scheme natively, where it would directly accept the URL request, bypassing emacsclient. Unfortunately for users of GNU FSF-published Emacs, Org Protocol was viable only if emacsclient was enabled, typically by relaxing macOS security permissions.

To address the above for GNU FSF-published Emacs, I set out to build a proxy utility that would accept the org-protocol:// URL request and transport it to Emacs in a way that conforms to macOS security policies. In effect, I made a macOS client app for Emacs server.

This app is named Scrim and is now in BETA test. I’m looking for testers, especially for those of you who already have Captee. If you’re interested and willing to provide feedback, please add yourself to the Beta test group on TestFlight, an Apple tool for distributing code-signed apps. At current I’m most interested in the following feedback:

• Sanity check - Does the app work as advertised?
• Were the instructions to setup Emacs server easy to follow? If not, then what was missing?
• Are you a user of Captee?

If you do sign yourself up for this, please provide answers to the above via the mail feedback in the Scrim app.

Thanks!

YES, I WANT TO BETA TEST SCRIM

Joe Marsahll has been blogging up a storm lately. One of his latest posts is a sort of personal history of Emacs and its relationship to Lisp. When Marshall started as an undergraduate, he was using the line oriented editor TECO. If you think your experience with an editor is difficult, read his description of what it was like to use TECO.

Soon he was directed to Vi and a whole new world was opened to him. He no longer had to work in the blind but could actually see what he was doing. Later, when he moved to MIT he was introduced to Lisp and various early incarnations of what we now think of as “Emacs”.

Emacs, of course, began life as a set of macros for TECO so, in a sense, Marshall revisited TECO when he started working at the MIT AI lab. but the Lisp Machines that were developed in the AI lab had their own Emacs clone integrated into the system. In his career he has used a wide variety of Emacs-like editors but like all of us he’s using GNU Emacs these days.

One of the things Marshall talks about is using Sly with Emacs to recreate his experiences on the Lisp Machine. Sly, he says, pretty much recreates the interface that you had using ZWEI—an early version of Emacs—on a Lisp Machine. I’m still using Slime for my Lisp coding but, as I have written many times, I have a long standing urge to work on a Lisp Machine so I may give Sly a try if only to get closer to the Lisp Machine experience.

• Upcoming events (iCal file, Org):
  • M-x Research: TBA https://m-x-research.github.io/ Wed Apr 16 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
  • Emacs APAC: Emacs APAC meetup (virtual) https://emacs-apac.gitlab.io/announcements/ Sat Apr 26 0130 America/Vancouver - 0330 America/Chicago - 0430 America/Toronto - 0830 Etc/GMT - 1030 Europe/Berlin - 1400 Asia/Kolkata - 1630 Asia/Singapore
• Upcoming events:
  • Emacs APAC: Announcing Emacs Asia-Pacific (APAC) virtual meetup, Saturday, April 26, 2025
• Emacs configuration:
  • Emacs Startup Time Doesn’t Matter (r/emacs, lobste.rs, r/planetemacs, Irreal)
  • Does emacs startup time ACTUALLY matter? (09:35)
  • Mastering Customization in Emacs #coding #programming #customization (12:39)
  • Emacs configuration for Editors (51:51)
  • Sharing my Emacs setup – stable, minimal, and daily-driven
• Emacs Lisp:
  • Protesilaos Stavrou: Emacs: My new ‘Emacs Lisp Elements’ book (Reddit, HN, lobste.rs, Irreal)
  • Learning elisp part 9 - regex conclusion (24:21)
  • Learning elisp 10 -built in data structures (16:07)
  • Learning elisp 11 - Emoji Replace part 1 (15:33)
  • Learning elisp 12 - emoji replace part 2 - hooks (13:47)
  • Learning elisp 13 - emoji replace part 3 - text properties (09:13)
  • Learning Elisp 14 - a minor mode (08:33)
  • Learning Elisp 15 - cleaning up our emoji mode (12:00)
  • Learning Elisp 16 - using a web API (21:17)
  • Ep643 emacs lisp, write a major mode for fsharp (01:04:36)
  • Xah Talk Show Ep641 emacs lisp coding, goto-line-number-next-buffer (24:25)
• Appearance:
  • Jeremy Friesen: Adding Actionable Indicators to my Emacs Mode Line
  • Emacs smart-mode-line Package #emacs #coding #programming (02:00)
  • Emacs vibe-talkin: making my modeline personal (09:28)
• Navigation:
  • Mastering Rectangles in Emacs #column #emacs (08:05)
• Dired:
  • Protesilaos Stavrou: Emacs: dired-preview version 0.5.0
  • The Emacs dired-du Package #emacs #coding #programming (02:12)
• Writing:
  • Set up Emacs for Classical Greek and Hebrew #greek #hebrew #emacs (07:17)
  • Store snippets of information fast with Emacs Remember #coding #programming (03:27)
• Org Mode:
  • Talk with Joshua Blais (Emacs enthusiast) | Emacs Org mode basics part 1 (1 hour video)
  • Nothing Is Simple - Org File Housekeeping (@ablatedsprocket@hachyderm.io)
  • How I added calculation of total effort time per day in org agenda
  • org-modern-indent v0.5
  • (update) org-zettel-ref-mode 0.5.7: Added reading status and rating management in the `org-zettel-ref-list` panel
  • Deltachat inbox bot - instant messenger bot that interacts with an org-mode file (orgmode list)
  • Maryanne Wachter: ERDs in Org Mode
  • ox-beamer-lecture - Export beamer lectures from Org Mode (Reddit)
  • Jeremy Friesen: Extending Org-Mode Export to Include Acronyms Section
  • [BLOG] #16 [[bbb:OrgMeetup]​] on Wed, Feb 12, 19:00 UTC+3 - notes
  • TAONAW - Emacs and Org Mode: First week with Journelly (Irreal) - Journelly as a mobile inbox for Org Mode
• Completion:
  • Introduction to Emacs Tempo Code Templates (@tiang@mastodon.social)
  • Jeremy Friesen: Extending consult-notes Package to Add Draft Blog Post Candidates
  • My god! It's full of tags! (@mms@mastodon.bsd.cafe) - completion using sqlite data
  • Emacs Mono Complete Package #emacs #coding #programming (01:05)
  • Arialdo Martini: Emacs: consult-line-symbol-at-point (Reddit)
• Coding:
  • New release of fj.el, a forejo/codeberg client
  • Setup: Clojure, Leiningen, Cider (14:02)
  • Emacs | Launch local server for HTML, CSS, JavaScript development (06:06)
  • Emacs and the art of Computer Programming - with Baishampayan Ghose (BG san) (01:44:57)
  • Anand Tamariya: CEDET: Across the Language Barrier (Reddit)
• Shells:
  • Use emacs as fzf with a shell script
• Mail, news, and chat:
  • Ruslan Bekenev: Newsticker - Emacs built-in RSS reader (Reddit)
• Multimedia:
  • Enhanced Image Features in Emacs 29 #coding #programming (07:05)
• EXWM:
  • emacs-exwm/exwm: Emacs X Window Manager (HN)
  • Emacs, an editor so versatile it can become a Window Manager - "Hakki" | Linux Fall 2024 / Emacs, czyli edytor tak wszechstronny, że może zostać Window Managerem - „Hakki” | Jesień Linuksowa 2024 - Tube.Pol.social (Reddit)
• AI:
  • Providing tools to AI
  • gornskew/skewed-emacs: Skewed Emacs: A Handy Setup for Gnu Emacs, Gendl, and AI (Reddit)
  • The Emacs Claude Code Package #coding #ai #programming #project (06:09)
  • New package: semext, Emacs alternatives for navigation and editing, via llms (YouTube 03:44)
  • Comparing Emacs region/speech or LLM queries to embeddings, LLM agents in work with Emacs Lisp (12:05)
  • RCD Notes RAG (Retrieval Augmented Generation) for GNU Emacs, demonstration #01 (02:38)
  • inf-gptel: Interactive Gptel shell for Emacs (Reddit)
  • Announce Aider.el v0.8.0, Integrating methods in classic programming books for code change and code reading
  • James Dyer: Ollama-Buddy 0.9.20: Curated AI Prompting with Awesome ChatGPT Prompts
• Community:
  • Fortnightly Tips, Tricks, and Questions — 2025-04-08 / week 14
  • Kris Carta: How I Emacs (#2)
  • Jeremy Friesen: Re: Moving from Spacemacs Package to Hand-Crafted
• Other:
  • Marcin Borkowski: Copying the current location on steroids
  • Emacs makes a big ediff #emacs #coding #programming (01:59)
  • Two Productivity Packages: bufferfile and stripspace #emacs #coding #programming (02:57)
  • Backup Each Save #emacs #backup #coding #programming (02:19)
  • Dragstuff - moving lines up and down in Emacs #lines (00:34)
  • What's new in Emacs 29 #emacs #coding #programming (11:18)
  • Accounting in Emacs with Ledger Mode - System Crafters Live! (01:48:45)
  • Navigating the Emacs Calendar #diary (05:10)
  • ablatedsprocket: Migrating Appointments and Holidays to Diary
  • Another workaround for another wsl2 bug (unwanted repeated keystrokes in emacs in wsl2).
  • What Emacs command or feature have you discovered by accident? I just discovered zap to char!
  • Introducing a new package: emacs-brew-man, a simple Homebrew manager (Reddit)
• Emacs development:
  • Support file:// URIs and readonly DB in 'sqlite-open'
  • ; * etc/NEWS: Move tree-sitter related items.
  • Add variable for control docstring length in elisp eldoc functions
  • New user option to hide minor mode lighters (bug#77361)
  • Disable clearing echo-area when 'inhibit-message' is non-nil
  • savehist.el: Handle concurrent access to `savehist-file`
  • flymake: fall back to margins on text terminals
  • ; * etc/NEWS: Move new variables only for Lisp programmers downwards.
  • Rename treesit-toggle-sexp-type to treesit-cycle-sexp-type.
  • ; * etc/NEWS: Expand description of vc-buffer-overriding-fileset.
  • New buffer-local vc-buffer-overriding-fileset and vc-buffer-revision
  • Rename treesit-toggle-sexp-mode to treesit-toggle-sexp-type.
  • Emacs responds to runtime display configuration changes on Android.
  • New command 'treesit-toggle-sexp-mode' (bug#76676).
  • Rename some of the new window layout commands
  • VC-Dir: Offer to register files before checking in
  • New function treesit-parsers-at for treesit-language-at (bug#77256).
  • (Fload, Feval_buffer): Emit a warning when lexbind is unset (bug#74145)
• New packages:
  • bufferfile: Rename/Delete/Copy Files and Associated Buffers (MELPA)
  • conda-project: Work with conda-project environments (MELPA)
  • ddp: Dynamic Data Processor with cmd tools (MELPA)
  • github-topics: Lookup PRs matching a query (MELPA)
  • org-mathsheet: Generate dynamic math worksheets (GNU ELPA)
  • votd: Bible Verse Of The Day (MELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

You can e-mail me at sacha@sachachua.com.

If you follow the various Emacs forums, you’ll see a lot a whining about how hard Elisp is to learn and how things would be so much better if only the extension language were something rational like Python, Ruby, or Lua. I rail frequently against this nonsense but Prot (Protesilaos Stavrou) stands as an unassailable refutation of the silliness.

He’s a guy with a liberal arts education who has no training in programming and yet has nevertheless produced some really great Emacs software. If you look at his code, you can see that he’s obviously mastered the intricacies of Elisp.

Now, Prot has shared some of his knowledge in the form of a short Elisp tutorial. Although Emacs is exhaustively documented, there’s a dearth of Elisp documentation for beginners. There is, of course, the Elisp manual, which is comprehensive, but not really beginner friendly. It’s a reference rather than a tutorial.

There’s the builtin Elisp tutorial and Mbork’s Elisp book but not much more. I am, therefore, happy to see Prot’s contribution. I haven’t had a chance to do more than scan it but it seems like a very worthwhile addition to the Emacs corpus.

One thing for sure, it should put all the whiners to shame. If you’re claiming to be a software engineer but find Elisp too difficult to learn, you should be chagrined that a guy with no formal training in software is nevertheless able to produce first rate Elisp code that the Emacs community embraces as useful and excellent.

The irony is that Lisps in general are actually easier to learn and use than other languages. It’s just that they’re unfamiliar to n00bs so they seem unapproachable. If you just suspend your expectations from other languages, you’ll find that Elisp is actually an easy language to learn.

Update [2025-04-13 Sun 16:24]: Added link to Prot’s book.

A few months ago I wrote a post detailing my Emacs journey so far.

Having read quite a lot of community posts, I think my path is one oft-taken:

1. Start out with someone else's opinionated config.
2. Slowly adapt that config to fit my own needs whims.
   • In my case how do I get this to be more like Vim?
3. Play around with all the things Emacs can do, stuffing the config with packages and snippets copied uncritically from various sources.
4. Read actual documentation (in my case Mastering Emacs) and learn to do things "the Emacs way".
5. Rip out almost everything and start from scratch, focusing narrowly on my 'actual' needs.

At the time of my previous post, I was firmly at step 3.

My config was growing and growing, and I had layers of custom bindings and commands that were starting to feel productive, but were also keeping me from learning how to do things the Emacs way.

Emacs is radically extensible and can easily be made to emulate other tools that I'm familiar with like VS Code or Vim (via styling, plugins, or even just direct 'emulation'), but I felt an increasing dissonance the farther I pushed it in those directions.

Furthermore, as I lamented in a 'sidebar' in a previous post, I hadn't yet learned how to consult the built-in documentation in Emacs - a real limitation given that Emacs does more than any tool I've ever used in providing a veritable wealth of comprehensive documentation and guidance.

Working in Emacs

In January, I returned to work after a nearly 4-month long parental leave.

Returning to work meant:

• Far more time spent in Org mode trying to keep my work life structured,
• Far less tolerance for half-baked or half-working configuration,
• Far less time (i.e. none) to play around with new packages and hobby-coding.

Org mode has been nothing short of a godsend for me at work, and as I wrote in my previous posts on "Why/How I Emacs", Org mode was the first "killer app" for me with adopting Emacs. I was skeptical at first, and the learning curve was steep, but I'm so happy I stuck with it. After a few 7am coffee sessions at work tweaking different parts of my Org mode config, I now have a simple little setup that I love and still has lots of room to mature.¹

When Emacs 30.1 released in mid-February, I decided to dive in and finally purchase Mastering Emacs. I'm so happy I did! The book eases you smoothly into "the Emacs way", with a great balance of technical and mindset explanations, to bring you along without ever feeling overwhelming.

Rip & tear

From January to today (April), my Emacs config - which I take as representative of how I use the tool - has transformed.

Over the past few months, a number of rip-and-tear sessions have dramatically reduced my init.el file, from:

• ~1000 LoC²
  • Split over multiple files (not a bad thing per se, but in my case indicative of too much complexity, too soon),
  • Mostly unannotated,
  • With:
    • ~150 LoC for custom bindings (using general.el and SPC as leader),
    • Most of the remainder a hodgepodge of useful & not-so-useful config lifted uncritically from other people's examples.

To:

• ~400 LoC
  • In one file,
  • Mostly annotated,
  • Either written by myself or copied from package docs (while consulting the built-in references),
  • With:
    • ~150 LoC for general Emacs tweaks,
    • ~150 for packages, all of which I use regularly,
    • ~100 for custom functions (mostly for writing this blog on Jekyll or for making shopping lists).

Put in terms of a mindset shift, I've gone from seeing Emacs mostly as an amazing playground where anything that can be done in plaintext is possible,³ to seeing Emacs as an unparalleled organization and writing (and sometimes coding) tool that I can customize exactly to my needs.

Next steps

There's still lots to do and learn!

Every week I learn a new handful of Emacs bindings, but I'm still very much a noob - next week I hope to finally start grokking copying & pasting and figuring out how to use the "kill ring" and registers. Not a work session goes by where I don't catch myself struggling to copy-and-paste, especially if it's from another app into Emacs.

I now think I know how to make a decent coding setup for Ruby & JS/TS with Eglot and Treesit, but as I'm still mired in new-baby-parenting (with baby in my lap at this very moment), I simply haven't had the free time to start coding again.

I'm still making my way through Elisp guides, albeit at a glacial pace. I'll get to writing my own commands and scripts, eventually!

I have some obvious areas for improvement in my config. Eventually, I reckon I'll bring back Elpaca or something like it for smarter package management than I get out of the box. I'm using Org mode all the time at work, but haven't been using Agenda so much - as a consultant, I'm very dependent on Outlook to juggle my multiple work accounts & calendars, and don't see this changing in the near future.

Lastly, I'm still leaning entirely on Evil mode for in-buffer navigation & movements. This is one thing I can't see changing, as the Vim motions are baked into my fingers after over a decade of daily reinforcement. I'd love it if I could remove or disable everything in Evil except the Vim motions, but after looking into this a little, it doesn't seem easy to achieve.

That's all for now! Seeing the development from the first post to this one, I'm curious how things will be when I get around to making the next post in this little series ☺️

What follows is my response to an email I received in 2023. The quoted regions are from the sender, and the non-quoted regions are my response.

My hope in publishing this post is to provide a bit of insight into my process of adopting Emacs 📖 .

Email and Response

I’d like to use Emacs for more things. I tend to lose focus if I have to switch context too much.

Switching context is the pathway to a thousand paper cuts of mental fatigue.

Currently I use Spacemacs 📖 for Org-Mode 📖 , but I’d love to get into Emacs for all project planning and coding workflows (maybe even doing research and emailing as well).

It took me awhile to use Org-Mode in Emacs ; I was using Markdown. But with my latest job, I’m all in on Org-Mode . It is how I track and eventually report my time. I haven’t yet made the switch to email via Emacs , but it is on my radar.

I’m still not using “all of it” but I’m building my way towards using more of it.

I’ve seen some articles out there about Org-Roam 📖 and using capture buffers to efficiently capture thoughts, but once I start going down the rabbit hole of custom Org-Roam capture templates and customizing any package outside of the Spacemacs layer guardrails I get lost!

What are some of the capture templates you were creating? I’m curious to understand how you’re looking to use Org-Roam . (Sidenote: For myself I have about 5 capture templates; and I think one of them I rarely use.)

Note: I had originally used Org-Roam . It is fantastic. However, I started extending it and found it somewhat more difficult and cumbersome. I grew concerned that I wouldn’t be smart enough in the future to further fix or enhance that. So I switched to Denote 📖 , which has amazing documentation and the functionality that I wanted for Org-Roam (namely backlinks).

I wrote about this in Migration Plan for Org-Roam Notes to Denote.

I also tend to get lost when I need to set up a development environment for any of the programming languages that I use. I’ve tried Spacemacs for Java at work, but all the packages installed by the Java layer cause my Spacemacs instance to slow to a crawl, and half of the Language Server Protocol (LSP 📖) commands don’t even work.

When I first started out, I picked Spacemacs , but I have little experience beyond a week or so with Spacemacs . I switched towards a basic install (e.g. fresh install no packages) and then went hunting.

I don’t program in Java; I’m a Rubyist, so I don’t know the language server ecosystem for Java. I know for Ruby 📖 there are two primary ones: Solargraph and Ruby Language Server. Both of those servers deliver different functionality. I’ve settled on Solargraph as it has more of what I’m after (namely jump to definition).

I write this wondering what functions you’re looking for from your language server.

Also, I have explored using LSP Mode, and it added a more visual experience; but settled on Eglot. In part because Eglot has a smaller implementation foot print, leveraging more of Emacs internals (and thus likely to stay “patched” and “maintained”).

This follows on something I identified when switching from Ivy to Consult as well as from Company to Corfu 📖 . See Switching from Company to Corfu for Emacs Completion.

I’d love to know how to surgically pick packages that will help me only, and cut out anything unnecessary.

This was the approach I took when moving from Spacemacs and Doom. I started from a blank slate, went through the Emacs tutorial, and then just started writing and coding. I would stumble and say “previous setups had done this. I’m going to go research how to do this.”

I’d find a few packages and look at their README’s, open issues, lines of code, etc and then adopt one. It was what first lead me to Ivy/Counsel/Swiper over Helm and would later lead me to Consult/Vertico/Marginalia.

I did the above with the mindset of “I am learning and building my tools; so other things will move slower.” In other words, if you have an urgent task to complete, maybe don’t do this. But if you have some slack in time, do this.

These days I recommend picking a good package manager (elpaca or straight.el) and installing one. I personally use straight.el.

I would also immediately adopt:

helpful

to improve discoverability

consult

a nice extensible wrapper around a lot of Emacs functions.

marginalia

to get inline guidance on your M-x commands; and many other things

vertico

I like the veritcal completion experience.

This is a principle of building out a solid foundation of interacting with your editor. From there move to the specific interactions.

I’d also love to know how to really fine-tune LSP and Debug Adapter Protocol (DAP 📖) to completely replace any need for an Integrated Development Environment (IDE 📖) in my day-to-day life.

I haven’t fine-tuned LSP and haven’t used DAP , so I’m curious what this means to you. (Sidenote: Since my original response, I have used DAP and found it to be very useful.)

Regarding the cruft of Spacemacs , the startup time bothers me, and being dependent on all the packages that layer maintainers stuff into their layers. I want to explicitly craft my experience and not rely on what others think are valuable packages to install. I want to know everything going into my instance and how to use it.

I appreciate that there are viable Emacs starter kits. (Sidenote: Spacemacs , Doom, and Prelude are three that I can list off the top of my head.)

Agreed. My pathway lead me to best understand what was happening. It has meant a lot more Yak-shaving, but I have an editor that moves with me.

My problem with this though, is I don’t know how to discover what I need. How do I know what packages to look for? How do I know when to write my own elisp config? No clue…

I used a multi-pronged approach. Starting from a basic setup, I would just start writing in Emacs and wonder “Hey, is there a function to do this?”

To know if there was something internally, meant I needed to figure out introspection of interactive functions. Then how to have meaningful descriptions of those functions. I would spend a bit of time scanning those functions. I might find something and read the docstring. Often this would lead to a neighborhood of functions. I could use that neighborhood to lookup the package. Then search Melpa for related or replacing packages.

Another strategy I adopted was to go to Melpa and perform some sorting. Finding the recently changed packages; I’d scan those quickly to see what is getting effort. I’d also sort on most downloaded, to see what other folks are using. And I’d look to last changed. All of which to orient to the breadth of options.

And if I stumbled upon a bunch of “thispackage-*” I’d dig in; after all folks were extending the baseline package.

I did all of this starting from a small baseline and building outward; from what was generally applicable to the specific. In a way, what is common to prog-mode and text-mode? Those are more useful than what is only text-mode and only prog-mode.

Also, should I break my reliance on full Vim 📖 keybindings and go for Emacs only? I don’t know. If I have to constantly re-bind keys for evil-mode with every package that I install I’m thinking that I’ll need to switch to Emacs

keybindings.

I don’t use Vim keybindings; even before using Emacs I was using its bindings. I’m wondering if something like the General package might be useful. I don’t use it but it helps manage these things.

I use Org for many things, two of which are managing appointments and tracking my paid time off. I just learned about Emacs' Diary package from one of Protesilaos Stavrou's videos, and decided to see if moving my appointments and holidays to Diary made sense.

This month’s Emacs Asia-Pacific (APAC) virtual meetup is scheduled for Saturday, April 26, 2025 with BigBlueButton and #emacs on Libera Chat IRC. The timing will be 1400 to 1500 IST.

The meetup might get extended by 30 minutes if there is any talk, this page will be updated accordingly.

If you would like to give a demo or talk (maximum 20 minutes) on GNU Emacs or any variant, please contact bhavin192 on Libera Chat with your talk details:

For Org mode users who want access to their files beyond the desktop, organice provides a solution. It's a feature-rich implementation of Org mode designed for modern web browsers on both mobile and desktop, offering a way to view and edit your .org files without needing Emacs. organice synchronizes your files with popular back-ends like Dropbox, GitLab, and WebDAV, ensuring you can access your notes and tasks wherever you are. At 200ok, we host a public instance at https://organice.200ok.ch which anyone can use – importantly, it's a front-end only application, meaning we don't store your data or use analytics.

We've just released organice version 1.2. This update focuses on improving the underlying technology to ensure organice runs reliably and that you receive future updates smoothly.

However, first, a quick but important heads-up:

Action May Be Needed (One Time Only!)

Some users previously encountered an issue where organice, particularly when installed as a PWA (Progressive Web App, like an icon on your phone's home screen), wouldn't automatically update to the latest version. This was due to a stubborn caching problem introduced in an earlier upgrade.

The good news: organice v1.2 fixes this!

To ensure you get this fix and all future updates correctly, you might need to perform a simple, one-time action to clear the old cache:

1. If you use organice as a PWA / Home Screen App (especially on iOS):* The most reliable way is to /delete the existing organice icon from your home screen and then re-install it by visiting the organice website in your browser and using the "Add to Home Screen" option.
2. If you use organice in a regular browser tab:* A /hard refresh should do the trick. (This is typically done with Ctrl+Shift+R on Windows/Linux or Cmd+Shift+R on macOS).

Once you've done this, you should be running v1.2, and future updates should arrive seamlessly without needing these extra steps.

What Changed "Under the Hood"?

This release involved a significant internal improvement. We migrated organice's build system from Create React App (CRA), which has reached its end-of-life, to a modern and efficient tool called Parcel.

This migration provides several benefits:

• Fixes the Update Bug: As mentioned, this was the primary driver – ensuring reliable delivery of new versions.
• Modern Foundation: Keeps organice maintainable and aligned with current web development practices.
• Efficiency Gains: Simplifies parts of our development and build process.

While this was mainly an internal change, the result is a more robust foundation for organice moving forward.

Where to Find More Details

For those interested in the technical specifics of the migration and other refinements included in this release, you can find the full details in the GitHub Release Notes for v1.2 and the associated [Architecture Decision Record ADR-004.

Thanks for using organice! We're excited to have resolved the update issue and look forward to bringing you more features and improvements on this more stable foundation.

If you liked this post, please consider supporting our Free and Open Source software work – you can sponsor us on Github and Patreon or star our FLOSS repositories.

I just published the book “Emacs Lisp Elements”. It is available for free and in freedom under the same terms as all documentation that ships with GNU Emacs:

• Official page: https://protesilaos.com/emacs/emacs-lisp-elements
• Git repository: https://github.com/protesilaos/emacs-lisp-elements

I provide a big picture view of the Emacs Lisp programming language by combining prose with code. The goal is to give readers an idea of how Elisp works by showing some of the main concepts or patterns discernible in everyday code.

Some chapters are beginner-friendly, while others dive into deeper waters. Though I think everything is still approachable, as I try to explain basic concepts and take things one step at a time.

The book is not meant to be a replacement for the built-in Emacs Lisp Reference Manual. It simply gives you enough information to reason about Elisp. Once you start extending Emacs, the rest will follow naturally.

I hope you enjoy it and continue to have fun with Emacs.

Introduction

I’m provisionally using the consult-notes package to provide find and search functionality for the notes I write using Denote 📖 . I write provisionally as I want to explore its utility relative to other things (such as the consult-denote package.

But that is not why I’m writing this post, instead I’m writing to draw attention to an approach I took at re-using implementation details in Emacs 📖 in a way that is different from other languages. Namely scoped variable binding.

I find this important as it is a conceptually different approach from every other programming languages I’ve used (e.g., Ruby, Python, Javascript, PHP, Basic, etc.)

Let’s dig in.

Consult Notes Denote Implementation

The consult-notes-denote–source is defined as follows. Importantly, the :items property is a chunky lambda. (Sidenote: By default, I’m hiding the following code, as it is a lot to read through.)

(defconst consult-notes-denote--source
  (list :name     (propertize "Denote notes" 'face 'consult-notes-sep)
        :narrow   ?d
        :category consult-notes-category
        :annotate consult-notes-denote-annotate-function
        :items    (lambda ()
                    (let* ((max-width 0)
                           (cands (mapcar (lambda (f)
                                            (let* ((id (denote-retrieve-filename-identifier f))
                                                   (title-1 (or (denote-retrieve-title-value f (denote-filetype-heuristics f)) (denote-retrieve-filename-title f)))
                                                   (title (if consult-notes-denote-display-id
                                                              (concat id " " title-1)
                                                            title-1))
                                                   (dir (file-relative-name (file-name-directory f) denote-directory))
                                                   (keywords (denote-extract-keywords-from-path f)))
                                              (let ((current-width (string-width title)))
                                                (when (> current-width max-width)
                                                  (setq max-width (+ consult-notes-denote-title-margin current-width))))
                                              (propertize title 'denote-path f 'denote-keywords keywords)))
                                          (funcall consult-notes-denote-files-function))))
                      (mapcar (lambda (c)
                                (let* ((keywords (get-text-property 0 'denote-keywords c))
                                       (path (get-text-property 0 'denote-path c))
                                       (dirs (directory-file-name (file-relative-name (file-name-directory path) denote-directory))))
                                  (concat c
                                          ;; align keywords
                                          (propertize " " 'display `(space :align-to (+ left ,(+ 2 max-width))))
                                          (propertize (funcall consult-notes-denote-display-keywords-function keywords) 'face 'consult-notes-name)
                                          (when consult-notes-denote-dir
                                            (propertize (funcall consult-notes-denote-display-dir-function dirs) 'face 'consult-notes-name)))))
                              cands)))
        ;; Custom preview
        :state  #'consult-notes-denote--state
        ;; Create new note on match fail
        :new     #'consult-notes-denote--new-note))

I wanted to re-use the implementation details of the :items property, but specify my own consult-notes-denote-files-function. Namely one that would create a candidate list of all denote notes tagged with blogPosts but did not have an associated published URL; which in my case is a note lacking a ROAM_REFS property.

Custom Candidate Function for Draft Blog Posts

In a shell, I can query for draft blog posts as follows:

fd "_blogPosts.*\.org" /path/to/denote/dir | xargs rg "^#\+ROAM_REFS:" \
--ignore-case --files-without-match --sortr modified

Let’s break that down:

• Use fd 📖 to find denote name conformant Org-Mode 📖 files with the tag blogPosts.
• Pipe those files to Ripgrep 📖 selecting from that file list files that do not have a line starting with #+ROAM_REFS:.

Re-Using the Existing

My first implementation copied the logic of consult-notes-denote--source. However, that was a lot of duplicate logic. I set about what it would take to re-use much of that logic.

The following, with code available on Github, is what I settled:

(use-package consult-notes
  :config
  (require 'consult-notes-denote)
  ;; Add a draft blog post section to my consult notes.
  (add-to-list 'consult-notes-all-sources
    `(:name ,(propertize "Draft Blog Posts" 'face 'consult-notes-sep)
       :narrow ?D
       :cateogry consult-notes-category
       :items jf/consult-notes/draft-blog-posts/items
       :state  consult-notes-denote--state))
  (defun jf/consult-notes/draft-blog-posts/items ()
    "Return a propertized list of draft blog posts.

The `consult-notes-denote--source' :items value has most all of the
logic I want (encoded as a `lambda').  However, I want to provide my own
files used in generating the candidates; this is done by way of a
contextual override of `consult-notes-denote-files-function'.

By convention, a draft blog post is one that is tagged with blogPosts
but does not yet have a ROAM_REFS property (meaning I have not
associated the org-mode document with the corresponding published URL of
the document)."
    (let ((consult-notes-denote-files-function
            (lambda ()
              (split-string-and-unquote
                (shell-command-to-string
                  ;; First narrow to files with tags
                  (concat
                    "fd \"_" jf/denote/keywords/blogPosts ".*\\."
                    (symbol-name denote-file-type) "\" "
                    (denote-directory) " | "
                    "xargs rg \"^#\\+ROAM_REFS:\" -i --files-without-match --sortr modified"))
                "\n"))))
      (funcall (plist-get consult-notes-denote--source :items)))))

First, instead of my :items having that chunky lambda, I specified a function (e.g. jf/consult-notes/draft-blog-posts/items). For the context of that function’s evaluation it overwrote the definition of consult-notes-denote-files-function. Then within that context called the :items property function stored in consult-notes-denote--source.

The docstring of jf/consult-notes/draft-blog-posts/items contains much of the relevant information.

Today’s implementation might look “chunky” but much of it is defining the overriding function. Below is a theoretical refactoring that might further clarify what is happening:

(defun jf/consult-notes/draft-blog-posts/items ()
  (let ((consult-notes-denote-files-function
         #'jf/consult-notes/draft-blog-posts/files-function))
    (funcall (plist-get consult-notes-denote--source :items))))

Alternatives?

I consider this somewhat magical coming from most other languages. I might be able to accomplish this in Ruby 📖 , but this would likely not be feasible in any other languages in which I’ve worked.

Other languages might require that I pass the consult-notes-denote-files-function as a parameter (either directly or by way of a property on a scoping context).

However the variable scoping meant that I:

• could re-use code
• didn’t need to submit a pull request

Conclusion

I wanted to have a function to find my draft blog posts. I also wanted to minimize duplicate effort. I could’ve used the underlying completing-read to do this lookup, but I also wanted to explore tying into a package I have been using.

And in tying into that usage, I am encouraging myself to use the package and make sure it is something I want to keep; or jettison and favor something else.

Emacs at its core is a Lisp interpreter written in C. If you are an Emacs developer, this means you often need to jump between C and Lisp programming languages. CEDET is capable of maintaining project-wide tag database and hence allows the user to seamlessly jump across the definitions in either C or Lisp. 

Demo: Jump to definition across Lisp and C (including pre-processor) using CEDET.

Note: Emacs provides necessary tools to make development easier.  find-function will also do the job. However, this article is about CEDET.

Another useful application is when you are editing an HTML which usually entails editing HTML, CSS and JS.

Setup instructions are here: https://lifeofpenguin.blogspot.com/2021/04/gnu-emacs-as-lightweight-ide.html

Introduction

In Denote Emacs Configuration I wrote about adding abbr and abbr-plural link types. In short, when I exported these link types the process would render an abbreviation and its title, based on my glossary entries. At least that’s what they did for HTML 📖 and plain text.

But until today, I had deferred on handling the LaTeX export. There was a rudimentary handling of it, but I figured I could do more. So I did.

Abbreviations on My Blog

First, I want to share how I handle this for my blog, as its different from other export backends.

I write my blog posts in Org-Mode 📖 and use Ox-Hugo 📖 to export into a Markdown format.

When exporting for my blog, I pass all of the acronyms through a custom my glossary Hugo 📖 short-code. (Sidenote: See Github for my glossary short-code implementation; warning this is a complicated short-code that I don’t want to touch.) During the processing, the first time I encounter a glossary entry, I render the title/name and its abbreviation. After that I use the abbr HTML tag.

Extending LaTeX Export

For LaTeX 📖 exports and thus generating PDFs 📖 I would like similar behavior as my blog.

I had found How to generate list of abbreviations in LaTeX? (from the LaTeX Stack Exchange); which demonstrates using one of two LaTeX packages:

acro

declare the acronyms in the header and then reference throughout.

acronym

explicitly render the acronymes in their own section.

Due to my initial approach and understanding, I went with the acronym package.

Let’s look at the code changes I made. (Sidenote: Here’s the Github commit that shows the totality of changes.) There are four steps:

• Register an Encountered Abbreviation
• Conditionally Render an Acronym Section
• Conditionally Use the Acronym Package
• Add Export Handling Functions

In summary, these described changes append a section named “List of Acronyms” that assist in untangling the acronyms I’ve used within the document.

Let’s dive in.

Register an Encountered Abbreviation

I was already handling abbreviations in LaTeX ; though they were somewhat broken in that I was rendering the associated name/title of the item.

Based on my read through of the Stack exchange suggestions, I wanted to add to a list of abbreviations I’d encountered. (Sidenote: I suppose I could’ve generated a full list of abbreviations for each LaTeX
export, but I have 400+ abbreviations and that seemed rather gross and excessive.
)

I added the following comments and code to my jf/denote/link-ol-abbr-with-property function.

;; When we encounter an abbreviation, add that to the list.  We'll
;; later use that list to build a localized abbreviation element.
(let ((abbr-links
        (plist-get info :abbr-links)))
  (unless (alist-get keyword-value abbr-links nil nil #'string=)
    (progn
      (add-to-list 'abbr-links (cons keyword-value title))
      (plist-put info :abbr-links abbr-links))))

With this, each time I encounter an abbreviation, I make not of it for later processing. The note includes the abbreviation key and its title (e.g. “PDF” and “Portable Document Format”).

Note: Based on my debugging, there is a caching layer that minimizes calls to this function. That is, we will only call the link generating function once for a given link (where the link identity is the key and description).

Conditionally Render an Acronym Section

The jf/ox/filter-body/latex function checks if we’ve encountered abbreviations. If so we append to the body of the document each encountered abbreviation (sorted alphabetically) along with its descriptive “title.”

(defun jf/ox/filter-body/latex (body backend info)
  "Conditionally add a list of acronyms to the exported LaTeX document.

To have a meaningful render, this requires using the acronym LaTeX
package.  The `jf/ox/filter-final-output/latex' handles injecting that
LaTeX package."
  (if-let ((abbr-links (plist-get info :abbr-links)))
    ;; We encountered some links, let's add a section.
    (progn
      (concat
        body
        "\n\\section{List of Acronyms}\n"
        "\\begin{acronym}\n"
        (mapconcat
          (lambda (cell)
            "Create an acro for link."
            (format "\\acro{%s}{%s}" (car cell) (cdr cell)))
          ;; Sort the keys alphabetically.  Otherwise they are rendered
          ;; in the reverse order in which they are encountered.
          (sort abbr-links :key #'car)
          "\n")
        "\n\\end{acronym}\n"))
    body))

In this case the body parameter is the inner-content of the eventual LaTeX document. It does not contain the header information nor the closing \end{document}.

Conditionally Use the Acronym Package

The purpose of this step is to ensure that we have are using the acronym LaTeX package. I could’ve updated all of my class declarations, but opted for this route.

The jf/ox/filter-final-output/latex function injects the acronym package into the final LaTeX document.

(defun jf/ox/filter-final-output/latex (body backend info)
  "Conditionally add an acronym package to exported LaTeX document."
  (if-let ((abbr-links (plist-get info :abbr-links)))
    (replace-regexp-in-string
      "^\\\\documentclass\\(.*\\)"
      (lambda (md)
        "Acronym package to matching line."
        (concat "\\\\documentclass" (match-string 1 md)
          "\n\\\\usepackage[printonlyused,withpage]{acronym}"))
      body)
    body))

In this context, the body parameter is the totality of content we will write to the interstitial tex file. That is it contains the LaTeX header information as well as the begin and end blocks.

I’m not enamored with using a regular expression, and might tinker with this a bit more.

Add Export Handling Functions

None of the above matters, until I append these filters to my LaTeX export.

The jf/org-export-change-options function and adding to the org-export-filter-options-functions ensures that we call the functions that conditionally render an acronym section and conditionally use the acronym package.

(defun jf/org-export-change-options (plist backend)
  "Conditinally add filter functions to our org-export."
  (cond
   ((equal backend 'latex)
    (if-let ((filter-body
              (plist-get plist :filter-body)))
        (progn
          (add-to-list 'filter-body jf/ox/filter-body/latex)
          (plist-put plist :filter-body filter-body))
      (plist-put plist :filter-body '(jf/ox/filter-body/latex)))
    (if-let ((filter-final-output
              (plist-get plist :filter-final-output)))
        (progn
          (add-to-list 'filter-final-output jf/ox/filter-final-output/latex)
          (plist-put plist :filter-final-output filter-final-output))
      (plist-put plist :filter-final-output '(jf/ox/filter-final-output/latex)))))
  plist)

(add-to-list 'org-export-filter-options-functions
             'jf/org-export-change-options)

The code is a bit chatty, but I wanted to ensure that I did not clobber other filter functions; instead appending to the list of possible filters.

A Bit of How I Got There

While working towards this solutions, I took a few different pathways. Ultimately, I leveraged the Emacs 📖 debugger and inline documentation of the ox package: the Org-Mode export “framework” package.

I allude to the revelatory moment in the change of the jf/denote/link-ol-abbr-with-property parameter name and doc string; changing the parameter name from protocol to info. I had carried protocol over from some other implementation; but a bit of debugging showed me that this was very much the information channel shared throughout the export.

With that knowledge, I added to the info parameter and set about finding a meaningful place to check the content.

It became a matter of reading through some source code to see how I might apply filters; something I knew existed based on previous reads of the ox package.

Why This “Vanity Project”?

Using a glossary of abbreviations is an endeavor that started with a two-fold goal:

• Registering as Back-Links
• Initial Descriptions of Abbreviations

And a third goal that has emerged over time: Untangling the Acronyms Used At Work.

Registering as Back-Links

When I was writing my posts in Markdown and directly into my Hugo project, I established back-link capabilities for glossary items. As I moved to Org-Mode for writing and first Org-Roam 📖 and then Denote 📖 , I wanted to continue to have access to back-links.

And since my abbr and abbr-plural conform to Denote ’s linking schema; using an abbreviation link registers as a back-link. Which means later on I can easily find written references to those glossary terms.

Initial Descriptions of Abbreviations

We live in a world of acronyms. Consider the three letters `RPG`:

• Rocket Propelled Grenade
• Role-Playing Game
• Report Program Generator

All of those are terms I’ve used when I’ve typed or spoken `RPG`.

When blogging about role-playing games, I don’t want to keep writing “Role-Playing Game” over and over. I want to use the abbreviation. But the first time I introduce an abbreviation, I want to provide the full name.

In using that strategy, I hope I’m improving the accessibility of my writing. Helping the reader acclimate to the terms I’m using, while also not maintaining a verbosity that might be off-putting for readers familiar with the domain.

Untangling the Acronyms Used At Work

Since I started blogging, I have changed jobs four times. I have joined:

• An academic library
• An early stage start-up
• A software consultancy
• A new-to-me tech domain of product development/engineering

At each of these, management and other folks would use new-to-me terms. Each time, I’d clarify then write down the acronym and its “title”.

This helped me remember what we were chattering about. And in some cases, depending on the documentation maturity of the organization, I’d contribute my glossary terms back to the organization’s glossary; or my team’s glossary.

Conclusion

Given that I write via Org-Mode documents for different audiences and that I have a glossary of abbreviated terms, I think it is useful to share at least a bit about that term.

For the future, I did note that the acronym package allows for me to include a description as a third parameter to rendering the list. My glossary also includes descriptions, so at some point, I’m certain I’ll be adding that to my LaTeX output.