Yet another short tip for today. (Don’t worry, longer, more involved posts will still happen here;-)!) A few years ago I wrote about a few useful debugging features of Emacs. A few days ago I learned that there are more of them now! About two years after I published that post, Emacs learned how to invoke the debugger every time some variable changes its value.
Update 2021-10-25: color-theme-tangotango removed, as it was mistakenly added.
Upcoming events:
Emacs Berlin (virtual, in English) https://emacs-berlin.org/ Wed Oct 27 0930 Vancouver / 1130 Chicago / 1230 Toronto / 1630 GMT / 1830 Berlin / 2200 Kolkata – Thu Oct 28 0030 Singapore
M-x Research (contact them for password): TBA https://m-x-research.github.io/ Fri Nov 5 0800 Vancouver / 1000 Chicago / 1100 Toronto / 1500 GMT / 1600 Berlin / 2030 Kolkata / 2300 Singapore
As I noted previously, John Kitchin has been busy making and posting videos about Scimax, his Emacs starter kit targeted at scientists and engineers involved in programming and publishing. There are so many videos that I can’t hope to comment on each one. Nonetheless, they’re all worth watching so I’m publishing a link to the Scimax channel so that you can subscribe or add it to your RSS feed.
Currently, Kitchin is concentrating on videos about Org-ref 3, the latest version of his excellent bibliographic and cross-reference tool for Org-mode. There’s still one or two videos to come in the series but after he’s done I’ll probably write a post on the whole series.
Until then, take a look at the channel’s video listings and pick any that look interesting to you. It will, believe me, be hard to go wrong,
I've been using Ledger to keep track of my finances for several years, and one
feature I really like is the ability to run reports from within Emacs via
ledger-mode.
Creating these reports is done by customizing the ledger-reports
variable. It's also possible to set them using the ledger-reports-add
function, but I prefer to set them all in one go during setup.
My current list of reports looks like this:
(setq ledger-reports
'(("bal" "%(binary) -f %(ledger-file) bal")
("bal this month" "%(binary) -f %(ledger-file) bal -p %(month) -S amount")
("bal this year" "%(binary) -f %(ledger-file) bal -p 'this year'")
("net worth" "%(binary) -f %(ledger-file) bal Assets Liabilities")
("account" "%(binary) -f %(ledger-file) reg %(account)")))
bal
Shows the balance of all accounts; this also includes individual
expenses (such as Expense:Car:Gas) so it's a pretty long report.
bal this month
As above, but only includes transactions from the
current month.
bal this year
Same as bal, but limited to the current year.
net worth
A short report that shows what I've got in each account, as
well as what I owe on loans and credit cards.
account
This prompts for an account name and then shows its
transactions for the current file. This is useful for quickly seeing how
much money has been spent on a specific category during the year.
Emacs reports that have the %(ledger-file) token are run against the current
ledger file. I have my Ledger files stored in a projectile project so I can
quickly jump to them to run reports or make adjustments.
I’ve been here before, but I find myself back here again. I think that many
people who start off with one of the big frameworks (like Spacemacs or Doom
Emacs) eventually circle around to thinking, “hey, why don’t I just build my own
Emacs config that includes the best bits of Spacemacs/Doom?”. Usually, that is
followed some time later (as happened in my case last time), by the realization
that those frameworks are really well crafted, and getting anything like that
degree of polish and sleekness yourself is very difficult without basically
replicating the entirety of those projects. However, I’m playing with
configuring from scratch again, just for kicks.
This is going to be a pretty short status update, as very little has happened
since my last “Meta Reduce” post. I was mostly enjoying the last weeks of my sabbatical
and I didn’t do almost any work on my OSS projects. That being said, now I’m finally back home and I’m ready for some action.
OSS Projects
My biggest short-term goals are new releases of nREPL and CIDER. Both are almost ready and I expect them
to go live in the next couple of weeks. A big update of clj-refactor.el is also on the horizon.
I also have to submit CIDER and clj-refactor.el to the NonGNU ELPA package repository.
I don’t expect any significant developments on my other projects, but I’d be happy to be surprised there.
Meta
I’ve spent the final weeks of my vacation mostly traveling in France and reading.1 I’m really glad that I finally found the time
to read “Only the Paranoid Survive” by Andy Grove. I’ve been meaning to read this book for years and it didn’t disappoint!
I also enjoyed reading the autobiography of Walmart’s founder “Sam Walton: Made in America”.
In the mean time I also finished re-watching Daniel Craig’s James Bond movies and I’m finally ready to see “No Time to Die”. And the new “Dune” movie!
I haven’t been to the cinema in ages, but I’ll make an exception for those two movies.
I also finished watching “Sex Education” season 3, and I have to say that this was the worst season of the show so far.
Yesterday I wrapped up “Squid Game” as well - a decent show, with plenty of flaws, that didn’t live up to the hype. Still,
at least now I understand all the “Squid Game” memes out there.
I was very excited the read about the partnership between Fastmail and 1Password, that allows 1Password to generate unique
e-mail aliases for each service that you sign up for. That’s a great step for privacy, and some very fortunate timing for me, as I switched to both services
just a few months ago. This also convinced me that for the time being 1Password fits better my needs than Bitwarden. I was really on the fence about which service
did I like more before the Fastmail news.
I continued to play with HEY World and I wrote a couple of short articles there:
I guess it’s safe to say I’m quite excited about the freshly announced MacBook Pros and quite upset about the COVID-19 situation in Bulgaria.
I like HEY World and I’ll probably keep using for sharing short musings, that don’t fit into a tweet or two.
I continue to favor the use of Jekyll (and Emacs) for most of my blogging.
Last, but not least - I’ve kept refining my StarCraf II game and I have the feeling I’m doing slightly better as of late.
With infinite choice of strategies that you and your opponents can go for, that’s one of the toughest and most mentally rewarding
computer games I’ve ever played. And one massive time sink!
Epilogue
And that’s a wrap. Now that I’m full of energy after my long vacation I hope to make some good progress on my OSS projects.
By the way - I can heartily recommend doing a long vacation to everyone, especially given how tough the last couple of years have been
on all of us.
And eating! French food is great and now I don’t fit into half my pants. ↩
You might have noticed that this year I’ve been promoting Discord
as the main chat for my bigger OSS projects like RuboCop and CIDER.
You may have been surprised by this move, as Discord originated
in the gaming community and is still mostly associated with it.
That’s why I decided to explain my reasoning in this short article.
I discovered Discord at the beginning of the year, and I like most people
I assumed it was just some chat for gamers.1 However, after playing with it
for a few months I figured out it’s a great fit for the needs of my projects
for several reasons:
You can create new Discord servers2 for each project, where you can create a nice channels structure like #support, #hacking, #general, #feature-requests, etc.3
The UI is simple and clean, and the client application is not a resource hog (unlike Slack). The UI assessment is obviously subjective, but a lot of people I’ve spoken with on the subject voiced the same opinion.
Even on a free plan you have unlimited chat history retention. That’s one of my main
issues with Slack and the famous Clojurians Slack, often the history there
disappears before I’ve had the chance to read something.
Discord’s voice channels are a nice way to quickly speak with someone on some topic. Admitted I use those mostly while gaming, but I definitely see the potential for
doing something like “office hours” with your community or whatever.
I like Discord’s version of threads a lot more that Slack’s. Slack threads are just too easy to miss.
Discord has very nice moderation tools.
It’s a cross-platform tool. Although I guess most of the famous chat apps are cross-platform apps these days.
It’s not Slack.
Discord reminds me a lot of the early days of Slack, where it was clear the target
audience where hackers (compared to big enterprises today) and the whole UX was
optimized for this. Compare this to all the buttons and menus that have replaced the old
days of markup and text commands.
Obviously Discord is not perfect (which tool is?), but I like it a lot and I’ve decided to place my bets on it. I’m well aware that it’s a proprietary tool, and that there were some privacy concerns about it in the past, but that’s fine with me, given the lack of good (in my subjective perspective) truly free alternatives. I never forget that perfect is the enemy of done.
There’s also the meta matter that chat never really took off for any of my projects, expect
CIDER’s Slack channel on the Clojurians Slack (#cider). I guess the majority of people simply
prefer to use only GitHub for project-related conversations and that’s perfectly fine for me. If I weren’t using Discord for other purposes, I doubt I would have bothered installing it just to chat about my projects. Same with Slack, really - the only reason I’m the Clojurians Slack is the fact that I use Slack for work.
I guess that’s also the reason why Gitter never took off, even if it was designed to be a chat for OSS projects - nobody was using outside of OSS, which really
limited its appeal.
Anyways, that’s all from me for today. I just wanted to encourage people looking to
add some chat to their OSS projects to consider Discord. I’m also curious to hear what chats
are you using to collaborate on OSS projects if any. Feel free to share your thoughts in the comments.
Truth be told, I kept confusing Discord with Discourse. ↩
You can think of Discord servers as an instance of Discord or an isolated namespace. They are basically the same as workspaces in Slack. ↩
I wrote a bit more about the structure of CIDER’s Discord here. ↩
WebDev Tory makes a point that many long time Emacs users can relate to:
Lately there has been a little chafing at #emacs appearance. I’m quite happy with my low-distraction working environment, though, and don’t care for file-trees or LSP helpers. pic.twitter.com/cykUeHRIoG
Power. That’s what Emacs gives us and really it’s all that matters. If you want bling—to channel DMR—you know where to find it. Head on over to VS Studio or whatever the latest hotness is. But if you want power, Emacs is the place to be.
In this video I showcase my new package, mct.el. It makes the default
completion experience feel closer to what you get with vertically
aligned third-party completion interfaces, by only re-using built-in
functionality and its own glue code.
Minibuffer and Completions in Tandem, also known as “MCT”, “Mct”, mct,
or mct.el, is a package that enhances the default minibuffer and
*Completions* buffer of Emacs 28 (or higher) so that they work
together as part of a unified framework. The idea is to make the
presentation and overall functionality be consistent with other
popular, vertically aligned completion UIs while leveraging built-in
functionality.
The main feature set that unifies the minibuffer and the
*Completions* buffer consists of commands that cycle between the
two, making it seem like they are part of a contiguous space.
MCT tries to find a middle ground between the frugal defaults and the
more opinionated completion UIs. This is most evident in its approach
on how to present completion candidates. Instead of showing them
outright or only displaying them on demand, MCT implements a minimum
input threshold as well as a slight delay before it pops up the
*Completions* buffer and starts updating it to respond to user
input.
Customisation options control the input threshold
(mct-minimum-input) and the delay between live updates
(mct-live-update-delay). Similarly, a blocklist and a passlist for
commands are on offer.
MCT is not available in any package archive for the time being, though I
do plan to submit it to GNU ELPA. In the meantime, you will have to
install it manually or with the help of straight.el, quelpa, or
similar. Its source code: https://gitlab.com/protesilaos/mct (a
Github mirror exists as well).
Those familiar with my dotemacs may
notice that mct.el is derived from my now-deprecated
prot-minibuffer.el library. The longer-term plan is to turn every
piece of my custom code into its own package or submit it as a patch for
emacs.git (as we did with the auto-renaming feature of EWW
buffers).
Too harsh? Let me rephrase: you’re barely using Avy. Still too broad? Okay, the noninflammatory version: Avy, the Emacs package to jump around the screen, lends itself to efficient composable usage that’s obscured by default.
Without burying the lede any further, here’s a demo that uses a single Avy command (avy-goto-char-timer) to do various things in multiple buffers and windows, all without manually moving the cursor:
Copy text, kill lines or regions, move text around, mark text, bring up help buffers, look up definitions, search google, check my spelling, the list goes on. I emphasize this again: Avy defines dozens of jump commands, but I’m only using one. This post breaks this down in detail so you can create your own version of this, but more importantly tries to explain why this is a neat idea.
This is the first of two parts in a series on Avy, an Emacs package for jumping around efficiently with the keyboard. Part 1 is about about supercharging built-in customization to do anything with Avy, or some approximation thereof. Part 2 will be a more technical (elisp-y) dive into writing more complex features for your individual needs. If you are interested in the short elisp snippets in this document, they are collated into a single file here.
Filter → Select → Act
We see the same pattern repeated in most interactions with Emacs whose primary purpose isn’t typing text. To perform a task, we Filter, Select and Act in sequence:
Filter: Winnow a large pile of candidates to a smaller number, usually by typing in some text. These candidates can be anything, see below for examples.
Select: Specify a candidate as the one you need, usually with visual confirmation. If you filter a list down to one candidate, it’s automatically selected.
Act: Run the task with this candidate as an argument.
Want to open a file? Invoke a command, then type in text to filter a list of completions, select a completion, find-file.
Switch buffers? Type to filter, select a buffer, switch.
Autocomplete a symbol in code? Type a few characters to narrow a pop-up list, choose a candidate, confirm.
As ever, this model is a simplification. Helm, Ivy, Dired & co let you select multiple candidates to act on, for instance. We will put this qualification aside while we explore this idea.
This observation leads to several interesting ideas. For one, the Filter → Select → Act process is often thought of as a single operation. Decoupling the filtering, selection and action phases (in code and in our heads) frees us to consider a flock of possibilities. Here’s minibuffer interaction:
The possibilities are, respectively: different completion styles (matching by regexps, flex, orderless), different selection interfaces (Icomplete, Ivy, Helm, Vertico, Selectrum and more by the day) and different action dispatchers (Embark, Helm/Ivy actions)1. A similar mini-Cambrian explosion has happened in the space of shell utilities since fzf arrived on the scene. In Emacs the minibuffer is where we’re used to selecting things, so it’s natural to think of these examples.
But this usage pattern extends to more than just minibuffer interaction. We often do things in regular buffers that involve the Filter → Select → Act pattern. In fact, you can look at most interactions with Emacs (whose focus isn’t typing text) this way: when you call a goto-definition command with point–the text cursor–in some text, the filtering stage is skipped, the selection is made for you by a function from the thing-at-pt library, and a preset action is called. How about a mouse interaction? scroll through a window full of icons, move the mouse cursor to an icon and click on it. These completely redundant examples prime us for the more interesting cases. Consider Isearch:
When you type (after C-s), you automatically filter buffer text and select the nearest match. You can select each subsequent match in turn, or jump to the first or last match. The Act here is the process of moving the cursor to the match location, but it can be one of a few things, like running occur or query-replace on the search string. Many Isearch commands simultaneously filter, select and act, so we’re fitting a square peg in a round hole here2.
If you’ve spent any time using Isearch, you can appreciate the tradeoff involved in dividing a task into these three independently configurable phases. Lumping two or all three into a single operation makes Isearch a wicked fast keyboard interaction. When I use Isearch my brain is usually trying to catch up to my fingers. On the other hand, the advantage of modularity is expressive power. The three phase process is slower on the whole, but we can do a whole lot more by plugging in different pieces into each of the Filter , Select and Act slots. To see this, you have only to consider how many disparate tasks the minibuffer handles, and in how many different ways!
But back to Isearch: what can we do to decouple the three stages here? Not much without modifying its guts. It’s all elisp, so that’s not a tall order. For example, Protesilaos Stavrou adds many intuitive actions (such as marking candidates) to Isearch in this video. But it turns out we don’t need to modify Isearch, because Avy exists, has a killer Filter feature, and it separates the three stages like a champ. This makes for some very intriguing possibilities.
Wait, what’s Avy?
Avy is authored by the prolific abo-abo (Oleh Krehel), who also wrote Ivy, Counsel, Ace-Window, Hydra, Swiper and many other mainstays of the Emacs package ecosystem that you’ve probably used. If you’re reading this, chances are you already know (and probably use) Avy. So here’s a very short version from the documentation:
avy is a GNU Emacs package for jumping to visible text using a char-based decision tree.
You can call an Avy command and type in some text. Any match for this text on the frame (or all Emacs frames if desired) becomes a selection candidate, with some hint characters overlaid on top. Here I type in “an” and all strings in the frame that match it are highlighted:
Typing in one of the hints then jumps the cursor to that location. Here I jump to this sentence from another window:
Play by play
Call avy-goto-char-timer
Type in “se”. This shows hints for all matches with “se”, including “sentence”.
Type in the hint char corresponding to “sentence”, which is g here.
Typical Avy usage looks something like this: Jump to a location that matches your text (across all windows), then jump back with pop-global-mark (C-x C-SPC). In a later section I go into more detail on jumping back and forth with Avy. Here is a demo of this process where I jump twice with Avy and then jump back in sequence:
Play by play
Call avy-goto-char-timer
Type in “demo”. There is only one candidate for this string, so Avy jumps to the other window.
Type in “jump”. This shows hints for all matches with “jump”.
Pick one of the matches. Avy jumps again, this time to the third window.
Call pop-global-mark (C-x C-SPC) to jump back to the previous location.
Call pop-global-mark (C-x C-SPC) again to jump back to the previous location.
At least that’s the official description. You can peruse the README for more information, but what I find mystifying is that…
…Avy’s documentation leaves out the best part
Avy handles filtering automatically and the selection is made through a char-based decision tree. Here’s how it fits into our three part interaction model.
Filter,
Before you call Avy every text character on your screen is a potential candidate for selection. The possibilites are all laid out for you, and there are too many of them!
You filter the candidate pool with Avy similar to how you would in the minibuffer, by typing text. This reduces the size of the pool to those that match your input. Avy provides dozens of filtering styles. It can: only consider candidates above/below point, only beginnings of words, only the current window (or frame), only whitespace, only beginnings of lines, only headings in Org files, the list goes on.
Filtering commands in Avy
Unit: Char
Unit: Word or Symbol
Unit: Line or other
avy-goto-char-timer
avy-goto-word-0
avy-goto-line
avy-goto-char
avy-goto-subword-0
avy-goto-end-of-line
avy-goto-char-2
avy-goto-symbol-1-above
avy-goto-whitespace-end
avy-goto-char-in-line
avy-goto-word-0-below
avy-org-goto-heading-timer
avy-goto-char-2-below
avy-goto-word-1-below
avy-goto-whitespace-end-above
avy-goto-char-2-above
avy-goto-symbol-1-below
avy-goto-line-above
avy-goto-word-1-above
avy-goto-whitespace-end-below
avy-goto-symbol-1
avy-goto-line-below
avy-goto-word-or-subword-1
avy-goto-subword-1
avy-goto-word-1
avy-goto-word-0-above
This is a crazy list to keep track of.
Filtering in Avy is independent of the selection method (as it should be), but there is a dizzying collection of filtering methods. I assume the idea is that the user will pick a couple of commands that they need most often and commit only those to memory.
Here’s the problem: We want to use our mental bandwidth for the problem we’re trying to solve with the text editor, not the editor itself. Conscious decision-making is expensive and distracting. As of now we need to decide on the fly between Isearch and Avy to find and act on things. If you use a fancy search tool like Swiper, Helm-swoop or Consult-line, you now have three options. Having a bunch of Avy commands on top is a recipe for mental gridlock. To that end, we just pick the most adaptable, flexible and general-purpose Avy command (avy-goto-char-timer) for everything.
(global-set-key (kbd"M-j") 'avy-goto-char-timer)
Further below I make the case that you don’t need to make even this decision, you can always use Isearch and switch to Avy as needed.
To be clear, this decision cost has to be balanced against the cost of frequent busywork and chronic context switching that Avy helps avoid. There is a case to be made for adapting Avy’s flexible filtering options to our needs, and the number of packages that offer Avy-based candidate filtering (for everything from linting errors to buffer selection) attests to this. We will examine this in depth in Part II.
But in this piece we are interested in a different, much less explored aspect of Avy.
Select,
Every selection method that Avy offers involves typing characters that map to screen locations. This is quite different from Isearch, where you call isearch-repeat-forward (C-s, possibly with a numerical prefix argument) or the minibuffer, where you navigate a completions buffer or list with C-n and C-p. Avy’s selection method is generally faster because it minimizes, by design, the length of the character sequences it uses, and we have ten fingers that can access ∼40 keys in O(1) time. The fact that we’re often looking directly where we mean to jump means we don’t need to parse an entire screen of gibberish. Unfortunately for this article, this means using Avy is much more intuitive than looking at screenshots or watching demos.
This excellent design leaves us with little reason to tinker with the selection phase: it’s sufficiently modular and accommodating of different filter and act stages. You can customize avy-style if you want to change the set or positions of characters used for selection. Here is an example of using simple words to select candidates:
We will pay more attention to the selection operation in part II as well.
Act!
This brings us to the focus of this article. The stated purpose of Avy, jumping to things, makes it sound like a (contextually) faster Isearch. But jumping is only one of many possibilities. Avy provides a “dispatch list”, a collection of actions you can perform on a candidate, and they are all treated on equal footing with the jump action. You can show these commands any time you use Avy with ?:
This means we are free to leverage Avy’s unique filtering and selection method to whatever action we want to carry out at any location on the screen. Our interaction model now ends in a block that looks something like this:
Additionally, Avy also defines a few commands that run different actions, like copying text from anywhere on screen:
Kill
Copy
Move
avy-kill-ring-save-whole-line
avy-copy-line
avy-move-line
avy-kill-ring-save-region
avy-copy-region
avy-move-region
avy-kill-region
avy-transpose-lines-in-region
avy-kill-whole-line
avy-org-refile-as-child
The problem with this approach is that it doesn’t scale. Each of these commands defines a full Filter → Select → Act process, and we quickly run out of headspace and keyboard room if we want any kind of versatility or coverage. They’re also not dynamic enough: you’re locked into the pipeline and cannot change your mind once you start the command.
Folks love Vim’s editing model for a reason: it’s a mini-language where knowing M actions (verbs) and N cursor motions gives you M × N composite operations. This (M + N) → (M × N) ratio pays off quadratically with the effort you put into learning verbs and motions in Vim. Easymotion, which is Vim’s version of Avy3, has part of this composability built-in as a result. We seek to bring some of this to Avy, and (because this is Emacs) do a lot more than combining motions with verbs. We won’t need to step into Avy’s source code for this, it has all the hooks we need already.
Avy actions
The basic usage for running an arbitrary action with Avy is as follows:
Call an Avy command. Any command will do, I stick to avy-goto-char-timer.
Filter: Type in some text to shrink the candidate pool from the entire screen to a few locations.
Act: Specify the action you want to run. You can pull up the dispatch help with ?, although you won’t have to if you set it up right, see Remembering to Avy.
Select: Pick one of the candidates to run the action on.
Here are some things I frequently do with Avy. Note that you can do this on any text in your frame, not just the active window!
First, taking the annoyance out of some common editing actions with Avy. If you use Vim and Easymotion, you get the first few actions below for free:
A note about these demos
For clarity, I set Avy to desaturate the screen and to “pulse” the line during a few of these actions. These are not enabled by default. I also slowed down the operations by adding a delay to make them easier to follow. In actual usage these are instantaneous.
The keys Avy uses to dispatch actions on candidates are specified in avy-dispatch-alist.
We will also need to ensure that these keys don’t coincide with the ones Avy uses as selection hints on screen. Consider customizing avy-keys for this.
Kill a candidate word, sexp or line
Killing words or s-expressions is built-in. I added an action to kill a line. In this demo I quickly squash some typos and delete a comment, then remove some code in a different window:
Play by play
Call avy-goto-char-timer.
Type in “is”. This shows hints for all matches for “is”.
Call avy-action-kill with k
Select one of the duplicate “is” occurrence. This deletes it.
Repeat steps 1-4 to delete a redundant “and”.
Call avy-Goto-Char-Timer.
Type in “key” to filter matches for “key”.
Call avy-action-kill-whole-line with K.
Select the commented line, this removes the line.
Repeat steps 1-4, this time selecting “(” in the other window and killing the function definition.
Repeat steps 7-9, this time selecting “ad” in the other window and killing the “(advice-add …)” line.
Note that Avy actually defines separate commands for this: avy-copy-line and avy-copy-region to copy lines and regions from anywhere in the frame. These are a little faster since they have the action stage baked into the function call. You might be better served by these. But we want to avoid the mental burden of remembering too many top level commands, so we work in two simpler stages: call avy-goto-char-timer (to filter and select) and then dispatch on our selected candidate as we see fit.
Move a candidate word, sexp or line
Avy calls this “teleport”, I call it “transpose”, either way it’s bound to t. In this demo I move text around the buffer… without moving (much) around the buffer:
Play by play
Make some room, type in a space.
Call avy-goto-char-timer.
Filter to candidates that begin with “(".
Press t to run avy-action-teleport
Select the candidate that says “(parametric forcing)". It is moved over (transposed) to where the point is.
Jump to where it says “DOWNLOADED” in the window with avy-goto-char-timer. This is the only match for the input “down”, so Avy jumps there automatically. You could also just isearch-backwards here.
Call avy-goto-char-timer.
Filter to candidates matching “the”.
Press T to run =avy-action-teleport-line~.
Select a candidate line (the one just below the image). It is moved over (transposed) to where the point is.
I use the dictionary package for Emacs, and I’m lazy about it:
Play by play
Call avy-goto-char-timer (or any other Avy jump command)
Type in “non”, this highlights matches for “nonpareil” (among others)
Press the dispatch key for avy-action-define, set to = here
Select the candidate, in this case one of the “nonpareil” matches.
This produces the buffer with the definition of “nonpareil”.
Call scroll-other-window (C-M-v) to scroll the dictionary window.
Call avy-goto-char-timer again.
Type in “fi”, this highlights the match for “finch” (among others). Note that this match is in another buffer, the one with the definition. We did not have to switch buffers.
Press the dispatch key for avy-action-define
Select the candidate, in this case the “finch” match.
This produces the buffer with the dictionary definition of “finch”.
;Replace your package manager or preferred dict package
(package-install'dictionary)
(defundictionary-search-dwim (&optionalarg)
"Search for definition of word at point. If region is active,
search for contents of region instead. If called with a prefix
argument, query for word to search."
(interactive"P")
(ifarg
(dictionary-searchnil)
(if (use-region-p)
(dictionary-search (buffer-substring-no-properties
(region-beginning)
(region-end)))
(if (thing-at-point'word)
(dictionary-lookup-definition)
(dictionary-search-dwim'(4))))))
(defunavy-action-define (pt)
(save-excursion
(goto-charpt)
(dictionary-search-dwim))
(select-window
(cdr (ring-refavy-ring0)))
t)
(setf (alist-get?=avy-dispatch-alist) 'dictionary-search-dwim)
Look up the documentation for a symbol
Play by play
Call avy-goto-char-timer
Type in text to filter with, in this case “pc”.
Press H to run avy-action-helpful.
Select a candidate phrase, in this case “pcase-lambda”. This pulls up a documentation buffer for this symbol.
scroll-other-window with C-M-v to scroll the help buffer.
call avy-goto-char-timer
Type in text to filter with, in this case “ma”.
Press H to run avy-action-helpful.
Select a candidate phrase, in this case “macroexp-parse-body”. Note that this is matched in the other (help) window. This pulls up the documentation for this symbol.
Repeat steps 5-9 to find the documentation of another symbol, in this case memq.
;Replace with your package manager or help library of choice
(package-install'helpful)
(defunavy-action-helpful (pt)
(save-excursion
(goto-charpt)
(helpful-at-point))
(select-window
(cdr (ring-refavy-ring0)))
t)
(setf (alist-get?Havy-dispatch-alist) 'avy-action-helpful)
You’ll need an Emacs feature that can search Google for you. There areseveral. I use a CLI program named Tuxi for this, and it’s pretty handy:
Play by play
Call avy-goto-char-timer (or any other Avy jump command)
Type in “ema”, this highlights matches for “Emacs” (among others)
Press the dispatch key for avy-action-tuxi, set to C-= here
Select the candidate, in this case one of the “Emacs” matches.
This produces the buffer with the Google’s description of Emacs.
Call avy-goto-char-timer again.
Type in “vi”, this highlights the match for “Vi” (among others). Note that this match is in another buffer, the one with the Google result. We did not have to switch buffers.
Press the dispatch key for avy-action-tuxi
Select the candidate, in this case the “Vi” match.
This produces the buffer with Google’s description of Vi.
Repeat steps 6-10 but selecting the string “POSIX” instead.
Now: We could continue populating avy-dispatch-alist with functions to do increasingly arcane contextual actions, but let’s take a step back. We want a list of easily callable actions on pieces of semantically classified buffer text… now where have we seen something like this before?
Avy + Embark: Any action, anywhere
Avy and Embark plug into each other like LEGO blocks. Here are a couple of examples:
Highlight occurrences
In this demo I highlight some keywords in a busy LaTeX document, then visit the bibliography entry of a citation key with Avy and Embark, without ever manually moving the cursor:
Play by play
Call avy-goto-char-timer (or any other Avy jump command)
Type in “flo” to filter matches that include “Floquet”
Run avy-action-embark with o.
Select one of the matches for “Floquet”. This runs Embark on the match.
Select the embark-toggle-highlight action with H.
Repeat 1-5 to highlight “Parametric”.
Call avy-goto-char-timer again.
Type in “na” to match one of the citation keys (among others)
Run avy-action-embark with o.
Select the citation key match. This runs Embark on it.
Choose the bibtex-action to visit the Bib file, bound to e by the bibtex-actions package.
Run a skein through Emacs’ help systems
In this demo I explore my way through a package with Avy and Embark, threading help, apropos and customization buffers, again without manually moving the cursor.
Play by play
Call avy-goto-char-timer.
Type in text to filter with, in this case “root”.
Run avy-action-embark with o.
Select “project-root”, one of the matches. This runs Embark on the match.
Press h, which makes Embark run describe-symbol on the match. This opens up a help buffer for the function. (Note: we bound a help command to Avy earlier, we could have used that.)
Press C-M-v (scroll-other-window) to scroll the help buffer.
Call avy-goto-char-timer again.
Type in text to filter with, in this case “proj”.
Run avy-action-embark with o.
Select “project-x”, which is one of the matches. This runs Embark on the match.
Call embark-cycle to change the target from a file (named “project-x”) to a library (named “project-x”)
Press h, which makes Embark run finder-commentary on the project-x library. This opens a buffer with some commentary.
Repeat the previous steps to run Embark on “project-x” again. This time, run the apropos-library action with a in Embark. This opens an apropos buffer.
Repeat the previous steps to run Embark again, this time on the symbol “project-x-local-identfier”.
Choose the customize-variable action with u in Embark. This opens a customization buffer for the variable project-x-local-identifier.
A division of responsibility
We save ourselves a lot of redundancy and reuse muscle memory here. Avy provides its unique means of filtering and Embark does what it does best, run actions! The intermediate job of candidate selection is shared between Avy and Embark: Avy specifies the general location of the candidate, and Embark figures out the semantic unit at that position on which to act. The fact that the Filter → Select → Act process is helpfully chunked this way by Avy makes the elisp required to integrate the two completely trivial:
Isearch and Avy have different strengths: Avy jumps quickly to any visible element in any window, Isearch to any matching candidate in this buffer. Avy is faster when you want to cover some distance in a jump, Isearch when you’re moving a small distance or a very large one. Avy is useful when your eyes are already on a target, Isearch when you’re looking for one. But you don’t have you choose. You can handily combine the two by restricting Avy’s candidate pool to Isearch candidates: now you can start Isearch and finish with Avy:
Again, consciously deciding which of the two commands to call every time is a bad idea. It’s not a bad idea to always Isearch and switch to Avy when necessary:
Play by play
Start Isearch with C-s. In the video I switched to isearch-regexp with M-r.
Type in a phrase to search for. In the video I typed in a regexp that ends in “-region”.
Navigate Isearch matches with C-s, recentering the screen with C-l if necessary.
Call avy-goto-char-timer. The candidate pool limits to the Isearch matches.
Pick an Isearch match to jump to as before.
At least, that’s the usual pitch.
For us, however, “jump” in that description is replaced with “act”. We can act on any visible Isearch candidate with one of the above actions. Kill text between two isearch matches? Copy the previous line that contains a word to the current location? Check. Essentially we filter with Isearch and select and Act with Avy, indirectly decoupling Filter from the other two actions in Isearch.
When Avy is too clever
This usage pattern has a failure mode. When there’s a single match, Avy jumps to the location and does not offer any actions. Oops.
While it’s possible to force Avy to show a selection char/hint for a single match, the default DWIM behavior is usually desirable. There are two options:
Filter candidates conservatively, for example by typing in a single character. Using avy-goto-char or avy-goto-char-2 will almost always result in more than one match, preventing this problem. If you use one of the timer-based Avy commands, you can vary how much text to filter by on the fly.
Carry out the action the old-fashioned way after jumping, then jump back by popping the mark Avy sets. You can do this with the default set-mark-command (C-u C-SPC)6. You can do this for most commands that cause the point to jump, including Isearch. Vim users have the jumplist, accessed with C-o, and the changelist, accessed with g;.
In this demo I jump twice with Avy to edit text and then chain jump my way back to where I started:
Play by play
Call avy-goto-char-timer and jump to a candidate (or end up there by accident)
Make your edits (or not).
Call set-mark-command with a prefix arg (C-u C-SPC) to jump back. You can chain these jumps.
What’s the Point, anyway
This section is for the pedants.
I’ve been using “point” and “cursor” interchangeably in this article. Yes, I’m aware of the distinction.
One of the illustrated advantages of using Avy to filter and select text to run actions on is that you can do it without moving the cursor. But as the above code snippets make clear with their save-excursion blocks, we do move the point, mostly just invisibly. The point is where the “gap” in Emacs’ gap-buffer data structure is located, so Emacs commands are all oriented around acting on the point, and usually more efficient when doing so.
Yes: it’s much faster to run an Avy action on some text in a different window than it is to call other-window, then Isearch to the text, run an action and switch back. But to me, the Point (har) is primarily a useful abstraction for writing elisp. The real advantage of Avy is in how it lets me think about the contents of the entire frame in the powerful Filter → select → Act paradigm. The fact that you can do this without the mental context switch involved in expressly navigating around the window or frame is a bonus.
Remembering to Avy
In some ways, using Avy to jump around the screen is like using a mouse. You could make the case, quite successfully, that the mouse is faster here and thus preferable. This unfavorable comparison evaporates when you add your dispatch actions into the mix. Yanking a line from another window or running goto-definition on a symbol on the other edge of the screen is much faster with Avy than with the mouse selection/right-click business. And this is without taking into account the disruptive effect of frequent context switching, the reason to prefer all keyboard navigation (or all mouse, when possible) in the first place.
The smell test
However, using Avy actions is a new way of interacting with text on your screen even if you already use Avy to jump around. To remember to use Avy actions or find new ones, I look for “smells” in my day-to-day Emacs usage:
Switching windows multiple times to land my cursor on some text
Isearching through more than three matches to jump to the right one
Moving the point a long distance to run a lookup command
Activating the mark manually (C-SPC) all the time
Jumping to locations to delete single words
Buried in keymaps
The other sense of “remembering to Avy” is that piling a new abstraction onto simple text editing means you have to learn a new keymap. Emacs already has too many of those!
This is true. But the effort is greatly mitigated by a choice of keys that is sensible to you. In the above code snippets, I made choices that mimic my Emacs’ keybindings (which are close to the default ones) so I don’t have to remember anything new:
Action
Avy keybinding
Emacs keybinding
Emacs Default?
Kill
k, K (line)
C-k
Yes
Copy
w, W (line)
M-w
Yes
Yank
y, Y (line)
C-y
Yes
Transpose
t, T (line)
C-t, M-t etc
Yes
Zap
z
M-z
Yes
Flyspell
;
C-;
Yes
Mark
m
m in special buffers
Yes
Activate region
SPC
C-SPC
Yes
Dictionary
=
C-h =
No
Google search
C-=
C-h C-=
No
Embark
o
C-o
No
You can go beyond the mnemonic and simply reuse the same keybindings you use in regular editing, trading off a slightly longer key sequence for maximally reusing your muscle memory. If you’re an Embark user, you don’t even need the above keys, just one to call embark-act.
The missing pieces
There are two common editing actions that still require manually moving the point elsewhere, perhaps to another window:
Searching or jumping to the contents of other windows beyond the confines of the screen. This has a simple solution:
In keeping with C-M-v to scroll the other window, you can Isearch the other window with C-M-s without switching to it7. If you’re feeling adventurous, replace (other-window next) in the above functions with (ace-window).
You can call Avy from Isearch as before, to run actions on essentially any text in the other-window’s buffer.
Copying regions of text. This has an Avy-based solution: avy-copy-region. I promised at the beginning, however, that you would only need to call one Avy command. For now you would do this the boring way, using Avy only to make shorter jumps or using Isearch+Avy. The more elispy solution will have to wait until part II of this series.
This post primarily concerned itself with the Act part as it connects with the ideas in the the previous one about ways to use Embark. But Avy is composed of modular pieces that makes it suitable for a wide variety of Filter → Select applications as well. In part II of this series, we will dig into the Avy API and see how to create unique commands.
Technically there’s also sorting the candidates, which I lump in with filtering in this breakdown. ↩︎
Isearch combines filter/search/act into a single command for efficiency. Even so, it’s a very cleverly designed library for Emacs with support for many actions. Folks who replace Isearch entirely with Swiper or similar commands are missing out. ↩︎
Or perhaps Avy is Emacs’ version of Easymotion. ↩︎
Why would you want to google an s-expression? The most common reason is when the sexp is a hyphenated phrase, like Fink-Nottle. ↩︎
Technically this is false, since we use save-excursion in all the avy action functions. The real accomplishment here is that I got almost 4000 words into this piece before mentioning Embark. ↩︎
If you end up in a different window you can pop the global mark instead with pop-global-mark (C-x C-SPC). There are also external packages for this now, like dogears. ↩︎
C-M-s is bound to isearch-forward-regexp by default. You can continue to access it with M-r in the Isearch map, i.e.C-s M-r. ↩︎
This year’s Emacs conference (EmacsConf 2021) will take place on
November 27 and 28. I will be among the speakers. My talk’s
description:
How Emacs made me appreciate software freedom
The theme will be “how Emacs empowered my software freedom”. I will
outline the key moments in my transition to a GNU/Linux operating
system and mark those which eventually contributed towards me becoming
an Emacs user, maintainer of a—dare I say—popular package, and
contributor to upstream Emacs (among others). By alluding to personal
experiences, I will draw generalisable insights and connect them to
what I believe are irreducible qualities of Emacs qua software and
Emacs as a community of like-minded people. The talk will be
theoretical in nature: there won’t be any code-related demonstration
nor technical references that only people with a background in
computer science would likely recognise. Personal anecdotes shall be
tangential to the point and considered as ancillary to the thesis of
what Emacs represents from the standpoint of software freedom and user
empowerment. The presentation is intended for a general audience that
is interested in GNU software in general and Emacs in particular. My
formal educational background as a social scientist (i.e. not a
programmer) and later as a philosopher informs my approach to this
topic.
The presentation shall be 40 minutes long. Its text will be in essay
form and shall be supplied as complementary material to the video.
The notation will be in Org mode.
Backstory
I had no intention to participate in the event, even though I was well
aware of it and despite the fact that I had followed the previous two
conferences. My reasoning was that (i) I probably had no technical
insight to share that wasn’t already known to the members of—and
likely participants in—such a setting, (ii) I already have my website
with a dedicated section on Emacs where I can publish any article or
video I want, and (iii) EmacsConf provides a platform for people who are
otherwise not disseminating their own stories to reach a wider audience.
Some or all of these assumptions may have been fallacious. They still
represent what I believed at the time.
At around the end of September, a few days before the deadline for the
submission of proposals, someone knowledgeable of the conference’s
logistics, whose name shall not be divulged, contacted me and asked me
to deliver a talk. I was eventually persuaded to make an attempt at
taking a 20-minute slot, but it was not feasible to condense my thoughts
to such a short video, especially given the tight deadline (I was
preoccupied with other issues, including the release of modus-themes
version 1.6.0
that was made just in time for the creation of the emacs-28 branch in
emacs.git). So I replied, perhaps with some relief, that I would not be
available for this year’s event and would consider trying again the year
after.
To cut the long story short, the 40-minute slot was considered a viable
option and I made a last-ditch effort to submit an abstract before the
deadline. It happened on the 30th of September.
There are lots of talks in this year’s conference. All seem very
interesting and I am looking forward to watch them. Hopefully mine will
be worth it as well.
Recently, John Kitchin tweeted a pointer to the draft of a paper by
Timonthy Johnson that makes the case for Emacs as the tool for scientists to organize and report their research. The paper, Emacs as a Tool for Modern Science, contrasts the “normal” way or organizing and conducting research to using Emacs for the same job.
Whereas the normal procedure involves the use of multiple pieces of software, the Emacs method uses only a single program: Emacs. It’s easy to take the attitude of ‘Meh, so what?’ but with Emacs, there is only one set of keystrokes to learn and there are no context switches as you move between different phases of a research project.
Johnson also stresses the importance of persistence. When you use proprietary software, you are at the mercy of the vendor. They may go out of business, discontinue the product, or make non-backward compatible changes. With open source software, the user can always recover—although perhaps not easily—because the source is available. More importantly, with Emacs, everything is in plain text and readable with any editor.
Finally, there is Org-mode. With Org, data and the calculations upon it can be embedded in the manuscript. That’s a huge win for reproducible research and ensures that the results in the paper also reflect the latest version of the data and the calculations on it. It’s also an easy way of generating LaTeX for submission to journals.
All of these benefits are well-known to developers who use Emacs but many scientists are focused on their field of research and don’t consider their tools. Mostly they use whatever tools they learned in grad school regardless of the tools’ shortcomings. If you follow Emacs, you know there are exceptions like Kitchin and Eric Fraga—to name the first two to come to mind—who have embraced the Emacs way in their research. Indeed, Kitchin has a similar paper in ACS Catalysis that makes analogous points.
I’ve written before about the use of Emacs in the humanities. Most of the same points apply. Really, if you’re wrangling text or doing research, you should consider Emacs.
The Task An old (ie 10+ years old) desktop application needed to be converted to a web application for modernization purposes. The program was a simple display of textbook-style information with links to audio content. We decided to build this with as simple a technology as possible, with hopes for longevity to exceed what it has had before: raw HTML and CSS, with as little Javascript as possible, optimizing compatibility1, longevity, and load time.
I can use C-h m to check the bindings of my current major mode, if it’s one where I can rest my cursor. But how can I check the bindings that are present in, eg, isearch mode? I can set up such bindings, but I’m not sure how to review them.
Answer: describe-keymap In emacs 28 there seems to be created the perfect function for this: (describe-keymap)1
Footnotes 1 Thanks to the answer on Reddit for this one: https://www.
Emacs FFM: Emacs FFM Meetup! October 2021 https://www.meetup.com/emacs-ffm/events/281293223/ Wed Oct 20 1000 Vancouver / 1200 Chicago / 1300 Toronto / 1700 GMT / 1900 Berlin / 2230 Kolkata – Thu Oct 21 0100 Singapore
Emacs APAC (virtual, in English) https://emacs-apac.gitlab.io/ Sat Oct 23 0130 Vancouver / 0330 Chicago / 0430 Toronto / 0830 GMT / 1030 Berlin / 1400 Kolkata / 1630 Singapore
Emacs Berlin (virtual, in English) https://emacs-berlin.org/ Wed Oct 27 0930 Vancouver / 1130 Chicago / 1230 Toronto / 1630 GMT / 1830 Berlin / 2200 Kolkata – Thu Oct 28 0030 Singapore
Too many people in the tech world make too much of a fetish of “minimalism”
Too many people in the tech world vastly over-complicate things.
Now, presenting these two things as opposites is something of a false dichotomy – it’s entirely possible (and maybe even probable) that a surfeit of minimalism at one level may lead to over-complication at another – but the representatives of the two camps as broadly outlined are in generally opposition to each other.
We might think of it as the vi/go/musl school versus the vscode/electron/kubernetes school.
One group holds that the problem with technology is over-complicated (and I agree!) and that the solution is to pare things down, remove excess complexity, go back to the good old days, and tends to have a somewhat elitest point of view that the problem is new developers being stupid.
The other position seems to be that the problem with technology is that we’re demanding more than ever out of it (and I agree!) and that the solution is to improve tools, build more complete, more user-friendly abstractions, that ultimately delegate the complexity to a elite caste of toolmakers.
In both cases I largely agree with the premise and some of the conclusions, but tend to oppose the schools as a whole.
Minimalisim
I’ve written about my issues with the “cult of minimalism” before, but in short my problem with it is that it tends to heap blame on the programmer, assert that failures of the tools are failures of the wielder, and seems to me to have a generally reactionary, conservative tenor.
It may be true that “it’s the poor craftsperson that blames their tools”, but if the tools are aggressively difficult to use and resources for actually learning how to use said tools are desperately wanting, it’s not especially surprising that people get discouraged and look elsewhere.
This point of view ultimately seems to argue for doing less, because it’s easier to do and results in more aesthetically-pleasing programs.
While I certainly can appreciate how nice it is to have a beautiful program, almost mathematical or poetic in its simplicity and directness, if you want to write programs real people actually use, that doesn’t last long.
Handling different locales and character sets is often a glaring weakness of minimalist exemplars, not to mention other forms of accessibility.
Other programmers may like looking at your code – and that is certainly a valuable thing, especially if you want to encourage contributions! – but users that aren’t programmers will probably be discouraged; both by the lack of features of the program and the condescending attitude such projects tend to display to non-programmer users.
In short, the “minimalist” programming methodology seems to me to be for programmers over users:
Make programs that are simple to write and easy to change, rather than trying to anticipate needs.
Again, I don’t disagree with that in principle, but in practice people’s needs are complex and addressing that essential complexity will make the code no longer simple; something that tends to be bitterly resisted.
Maximalism
On the other hand, we have giant, complex systems that are used to solve giant, complex problems.
Three immediately come to mind for me: SystemD, Electron, and Kubernetes.
The first two are roundly despised by not only many in the minimalism camp but by many programmers generally, while the last is (as far as I see) generally tolerated, seen as a necessary evil.
SystemD has been explained and debunked and counter-debunked many, many times already and fortunately seems to now be more broadly accepted by most non-partisans.
I won’t really dwell on that too much:
My perspective as a long-time Linux user and sort-of sysadmin, I like systemd because it unifies what was previously a bunch of arcane shell scripts and greatly simplifies many tasks.
I have written a lot of shell script in my time, I’m comfortable with lots of Linux esoterica, but systemd is just easier and doesn’t really have any downsides to me.
Pretty much all of the “downsides” I see people complain about tend to be of a more philosophical nature (the old “Unix way!” canard comes up a lot), but it’s a philosophy I don’t subscribe to.
I can understand why people dislike Electron.
Many professional programmers, especially those that started programming some time ago, tend to have a visceral dislike of Javascript and Electron seems to represent Javascript taking over the world of “native apps”.
Again, I sympathize – Javascript is not my favourite language, it has many warts, and I really dislike its package ecosystem.
However, people complain about Electron apps replacing native apps, how it results in so much RAM consumption, CPU usage, is inefficient and represents “laziness” on the part of developers.
My perspective is generally that it’s not “electron vs native app”, it’s usually “electron vs nothing” – especially for Linux users.
It’s a huge pain to write multiple native apps, not to mention that it’s not exactly clear what “native” means for Linux anyway – GTK (3 or 4?)? Qt (with it’s weird licensing issues)? X11? Wayland?
No matter what choice you make, people are going to get mad at you
I’m certainly not against native apps.
I also develop for iOS, but I can completely understand why one wouldn’t do that!
It’s a lot of effort and expense to get someone to learn Objective-C and/or Swift (languages that are essentially only useful for Apple’s platforms), learning UIKit and/or AppKit (and/or Catalyst, and/or SwiftUI), and then wrestling with Apple’s god-awful documentation and IDE, and that’s just for one platform!
If you also want a Windows app, you have another gigantic amount to learn, and Linux support is at that point pretty unlikely.
Users can derive a lot more benefit from an electron app that is in front of them now, not three years from now, and works on all their friend’s computers as well.
I sincerely doubt many users that aren’t programmers know or care what electron is because it doesn’t affect them.
There is a part of electron I dislike though, and that’s the Javascript ecosystem that comes along with it.
Much digital ink has been spilled lambasting that, so suffice to be say that it’s frustratingly over-complicated, fragile, constantly changing, and generally a sad state of affairs.
Kubernetes is something that, every time I read about it, I just get frustrated.
It seems to be the apex of a certain sort of programmer fashion trend, the exact opposite of the “extreme minimalist” that says “abstract absolutely everything” and “cloud scale”.
The notion that we need to be using whatever the FAANG companies are using, to deal with the problems of our massive scale.
These things always seem to me like just creating your own problems, making increasingly complicated tools to fix the problems, then layer even more complexity on top to address the complexity of the previous set of tools.
Even Docker seemed somewhat ridiculous to me and the extent to which it has metastasized into this leviathan system of orchestrators and containers and oceans of YAML seems nightmarish to me.
This is where I start to come down on the side of the minimalists – this is complexity that is unneeded, if people could just learn their tools better!
If were actual sysadmins with expertise hadn’t been off-shored and then fired, maybe it would be possible to just deploy applications on a server without it needing a zillion layers of tools and config files.
These massive systems tend to lead to a sort of bifurcation of programmers into the plebians that just use the pretty tinker-toys and the “real” programmers that build the tools for them.
Sure, there are going to be different levels of skill and ability, to I fear that this sort of “just use the monolith” approach really discourages learning and makes new programmers learn that the implementation is on another level from them, one they’ll never be able to work at.
Synthesis
That’s the solution then?
Unfortunately, complex, gray-array “it-depends”.
It’s very unsatisfying to say “do as little as possible, but no less”, because that’s such a subjective, hard-to-nail-down thing.
Ultimately though, that’s why programming is a craft that takes time to develop, not just a cookie-cutter approach that can be blindly applied everywhere.
The desire for simple answers is, of course, a much broader problem than just in programming.
We live in a complex world and have limited resources, so anything we can do to replace difficult reasoning with easy-to-apply rules is welcome.
Of course, those “easy-to-apply rules” are what biases are; something that we generally are taught are “bad”.
Even that dismissal as bias as bad is itself an over-simplification though!
The bias against biases can be replaced with nuance, realizing that some biases are useful (e.g. if food smells rancid, being biased against eating it is probably a good trait), while some are not.
Who has time to carefully examine all their biases though?
One of the big advantages of living in a society is that we’re taught biases that save us having to learn bitter lessons ourselves.
A senior developer teaching a new hire to “always put curly-braces on your if-statements” is giving them a rule that they’ve learned the hard way, in the hope that the fledgling coder won’t make that mistake again.
This sort of cultural knowledge is how we develop and hone our collective craft over time.
However, maturity is realizing that your mentors might not have been wrong, per se, but that they were giving you a simplification that was appropriate.
To develop further, you need to learn the reasons behind the biases you were taught.
Instead of just pursuing minimalism for the sake of it, try to think about the why:
They wrote minimalist code, because it was a time where things were changing rapidly and every user was also a programmer, so making things easier on programmers also paid off for the users.
Similarly, giant tools like Kubernetes are made to deal with the problems of scale beyond what almost anyone actually needs to do with:
Abstraction has a cost and one should only accept that when it’s a net benefit.
I think the commonality in what I dislike is that programming becomes a fashion statement rather than a means to an end.
Both the ultra-minimalists and the sprawling over-engineering maximalists are both trying to use their way of programming to indicate what kind of person they are:
Intelligent, frugal, crafty lone-geniuses, or modern, professional, best-practice-gurus.
I certainly enjoy programming for its own sake, but when making something that’s actually for other people, I need to make sacrifices.
I sometimes need to throw away the beautiful, elegant code because it wasn’t solving the problem users actually had.
By the same token, I can serve users better if I think carefully about what is actually needed now and solve that problem in the most direct way, not the way that’s going to make me look the best.
Of course, I’m quite privileged in that regard – I run my own company and had the access to learn many different languages and frameworks in the past, so I’m aware my options aren’t available to everyone.
This isn’t meant to be prescriptive, but more of an attempt for me to attempt to reconcile my opinions to myself.
I just want more people to be able to both feel the satisfaction of making something genuinely useful and being able to fully understand how the whole thing works.
As I’ve said before, John Kitchin is producing so many videos that it’s hard to keep up. One of his recent videos is about Scimax Notebook, his implementation of digital notebooks based on Projectile, Bozhidar Batsov’s project interaction package. In my Zamansky-79: Project.el post, I mentioned that I’ve never felt the need for a package like Projectile but Kitchin reimagined it as an engine for digital notebooks.
The idea is pretty simple: every notebook is a project in the Projectile sense, and every Projectile project can be thought of and acted on like a notebook. The Scimax Notebook library is simply a series of functions that operate and search the project’s file. All of that is, of course, leveraged on Projectile.
The library and Kitchin’s explanation of it is interesting. It’s clear that the library grew organically with functions being added as there was a need for them. Kitchin says that he rarely uses some of them indicating a failed experiment. In other words, the library grew according to his needs and will doubtless continue to evolve.
The source code, as a literate programming file, is available here. The video provides a quick exegesis of many of the functions and provides an idea of what’s going on. The video itself is just over 37 and a half minutes so you’ll definitely have to schedule some time. If, like me, the idea of an Emacs-based notebook infrastructure appeals to you, be sure to spend the time to watch his video. The source code, being a literate file, should be easy to adapt for your own purposes and needs.
Starting with commit 171de3eee4 to the Emacs master branch, users
can now opt-in to a feature that automatically renames EWW buffers to
something more usable than the default *eww*.
The point of entry is the customisation option eww-auto-rename-buffer.
When given a title value, it will use the web page’s title as the name
of the buffer. When the value is url, the page’s URL shall be used
instead. Otherwise it is possible to give it the symbol of a function
that runs without parameters and returns a string. So users can
configure this however they like.
Detail of the commit:
commit 171de3eee459ed64388a8ced7d07fa031ea025a6
Author: Protesilaos Stavrou <info@protesilaos.com>
Date: Fri Oct 15 14:12:32 2021 +0200
Add new option to rename eww buffers
* etc/NEWS: Document the new user options.
* lisp/net/eww.el (eww-auto-rename-buffer, eww-buffer-name-length):
Add new user options.
(eww--rename-buffer): Introduce new function that performs the
renaming of buffers.
(eww--after-page-change): Add new wrapper function which calls
'eww-update-header-line-format' and 'eww--rename-buffer'.
(eww, eww-render, eww-tag-title, eww-readable, eww-restore-history):
Include eww--after-page-change.
Fix bug#51176.
Co-authored-by: Abhiseck Paira <abhiseckpaira@disroot.org>
Co-authored-by: Protesilaos Stavrou <info@protesilaos.com>
doc/misc/eww.texi | 8 +++++++
etc/NEWS | 10 +++++++++
lisp/net/eww.el | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
3 files changed, 78 insertions(+), 5 deletions(-)
Thanks to everyone who participated in the discussion around
bug#51176.
Thanks to the Emacs co-maintainer, Lars Ingebrigtsen, for the feedback
and for merging the patch. And special thanks to Abhiseck Paira, my
co-author for this patch, for all the work in making this happen.
I am very happy to see this merged into core. I have been using a
variant of it for several months now. It had been one of the most
useful additions of mine in making EWW my main web browser.
Going forward, the plan is to extract more out of my prot-eww.el,
refine it, and push it to emacs.git. Same principle for everything else
that goes into my dotemacs.
John Kitchin has a short video in his Scimax channel that considers another Elisp construct. This video talks about let and its siblings. One of the first non-trivial things a new Lisper learns is how to deal with local variables using let.
If that didn’t make sense to you, Kitchin’s video explains it all. If you’re familiar with any of the Lisp dialects, you’ll also be familiar with let. It turns out, though, that there’s still a bit to learn. Elisp has additional let constructs that are reminiscent of Common Lisp’s destructuring-bind.
The first, let-alist, binds each car of an alist to its corresponding cdr. If you have to deal with several entries in an alist, it’s a way of referring to an alist value by name instead of the normal assoc/car/cdr dance.
The second, seq-let, is similar except that it takes two lists. The first is a list of symbols that are bound to the corresponding items in the second list. This is basically a scaled down version destructuring-bind.
Finally, Kitchin mentions pcase-let, which binds patterns to values. It’s much more complex than the others so Kitchin doesn’t discuss it in depth.
This is a short video, 12 minutes, 12 seconds, but it’s really useful for Elisp n00bs and those not familiar with let-alist, seq-alist, and pcase-let. If you fall into either category, be sure to take a look.
Leuven
For emacs custom framework used to implement the following emacs themes refer to this post:
How to Configure Emacs Themes
Screenshot Leuven
In following, in terms of clarity, it's easier to structure blocks of code syntax with opening and closing blocks.
Any code blocks between opening and closing code blocks translates into code between those sections in the script.
Open Use-Package Color-theme-modern Configuration Code Block With ':commands' directive
Mike Zamansky, on the heals of his last video, has another offering in his Using Emacs Series. This time it’s about the new Project library built into the latest Emacs. If you’re familiar with Bozhidar Batsov’s Projectile package, you can think of Project as a pared down, built-in Projectile.
Zamansky is refactoring his Emacs configuration and trying to keep down the bloat. Since he doesn’t use the Project/Projectile functionalities very often it makes sense for him to see if he can replace Projectile with Project.
The video takes us through the Project workflow and concludes that, at least for Zamansky’s needs, it’s a decent replacement for Projectile. There’s no doubt that Projectile is more complete and polished at this point but Project will no doubt evolve to meet the needs of its users.
Projectile has long been a popular project and many people swear by it. I’ve never felt the need for what it offers so I have nothing to offer in the way of informed advice. By way of uninformed advice, I’d say try out Project and if it works for you, investigate Projectile.
The video is 11 minutes, 39 seconds so you can probably fit it into a coffee break. The good news is Zamasnsky is promising more videos in the near future.
Problem As laid out on reddit1, I arrange my orgmode notes as single files with lots of entries. This works great for me because emacs is more buffer-oriented than directory-oriented, so all the emacs search commands optimize for single-file approaches (eg search all headlines in my tech/blog file for the keyword of that Linux function I’ve forgotten how to use).
The single problem I have is footnotes. From the documentation you have two options: inline or in a file-wide section.
Intro This is an issue that others are experiencing too.1
Just pulled and evalled latest bufler moments ago.
Have multiple frames/windows running, eg C-x 2
M-x bufler
choose one of the buffers from the list, to move to it
See it close one of your other windows.
I discovered this when it was eliminating my exwm workspaces upon choosing something, but I found that it is by no means limited to exwm things.
At work we have an effort called the ‘return to new normal’ which is an admission that while we are returning to something that looks like pre-pandemic life, everything has changed and what is normal now is entitely different from before.
I (many others I know) are “exiting” the pandemic with new babies. If a global catatrophy doesn’t change your whole life, a baby certainly will. Some things become more important (like having a job that supports your new reality) and some things less so (like whatever you thought your 5-year career plan should be).
As my baby exits her first year, I am finding myself returning to a new normal. My hobbies, reading and knitting, have restarted after a long lockdown and pregnancy pause. In an effort to avoid storing my data in someone else’s system, I’ll be adding some reading logs and knitting project posts here.
John Kitchin has been vlogging up a storm. He’s producing so many videos, Irreal is having a hard time keeping up. One of his recent offerings is a really great1video on one of my favorite Emacs packages: Elfeed. Elfeed, of course, is the Emacs RSS reader. My use of it is pretty much out-of-the-box: it had everything I needed and in any event there weren’t any obvious ways of customizing it short of digging into the code. Kitchin, of course, showed that for the nonsense it is. He’s made numerous tweaks without touching the Elfeed source code.
The first thing that struck me was that he set up a timer to update his feed database automatically, much like email. Elfeed doesn’t have that built in but it’s a trivial one command addition to the Elfeed config to implement it.
He also sets some font faces so that article headlines display in a different color depending on their subject matter. The latest Elfeed README has some guidance on how to do this. If this appeals to you, it’s easy to do and Kitchin explains one implementation.
For most people, Kitchin’s biggest tweak is probably his implementation of scoring. He used to have a home-grown solution for that but now uses the elfeed-score package. It provides Gnus style scoring so that articles that are of the most interest to you are at the top. If you have lots of feeds, seldomly check them, or have limited time to deal with them, scoring is an excellent way of concentrating on the ones most important to you.
Being a researcher, Kitchin also has several functions to do things like email an RSS entry to a colleague, capture an entry into his bibliography database and store links for later use.
If you aren’t an Elfeed user, you should be. If you are an Elfeed user, you should definitely watch Kitchin’s video; it has a ton of useful ideas to make your workflow better and easier.
Mike Zamansky has come up for air and after a long hiatus, he’s back with another video in his Using Emacs Series. This video is about his project grading workflow. That may not seem to interesting to some folks but, as usual, he leverages Emacs and some new packages that he hasn’t discussed previously.
If you haven’t been a teacher, you probably have no idea what a chore grading is. If you want to evaluate your students fairly, you have to bear down and can’t just phone it in. Therefore, anything that eliminates routine chores and makes the process run more smoothly is a real win.
To a first approximation, Zamansky use GitHub and GitHub Classroom to handle his homework and project assignments and their grading. GitHub provides a very nice infrastructure for teaching and coordinating with students. That was true before COVID-19 and is probably more true now. The problem with it is that it’s Web based and involves a lot of clicking when you’re dealing with several students and their repositories.
Zamansky deals with this by cloning all the student repositories to his local machine. That’s a win because he can then deal with everything from inside Emacs. For example, if he wants to compile and run some student code, that’s easy from within Emacs.
A typical workflow is grading a project. Zamansky first runs consult-find to find all student programs implementing the project. He can direct all the paths to a separate buffer from which he can visit each in turn. There’s no navigating to each student, then to the proper project, and finally to the code. Everything is right there so he can simply step through the list. This is handled through the consult package.
Likewise, he can communicate with a student by leveraging the forge package to raise an issue in the student’s repository. Similarly, a student can raise an issue that can be downloaded with forge. Finally, Zamansky can use the Embark package to render PDFs or link in a separate utility, although he usually prefers to render it directly in Emacs.
If you’re interested in this sort of thing, you can also take a look at this 2014 blog post and video from John Kitchin on how he handled things then: blog post, video. I’m sure he’s evolved his system subsequently but it was already pretty powerful and completely Emacs based.
Zamansky’s video is 18 minutes, 39 seconds long so plan accordingly. It’s interesting even if you aren’t a teacher. In any event, it’s nice to have Zamansky back.
One of the comments on my last video talked about the new project
management support Emacs includes in it's latest version -
project.el. I remembered reading about it when it rolled around but
then forgot and never checked it out.
Up until recently when working in projects I used Projectile, a great
package by Bozhidar Batsov, also known as Bug. I only used it when
doing development work - that's when I find myself jumping around
within a group of related files. At other times, I'm pretty much in
one file, do my thing, then move on.
As I mentioned in the last video, I decided to rebuild my Emacs config
from scratch and I'm trying to be a bit more minimalist so with that
reminder I got, I figure it's time to check out the now built in
project.el. If it's as good as projectile, or even, good enough, it
could be a nice alternative.
At it's core, project.el works similarly to projectile. There's a
prefix key an then an action which is restricted to your project. For
projectile, it was C-c p and the default for project.el is C-x
p. No big difference there. Projectile seems to have many many more
options but in general, I find that all I need is
Switch to a project - C-x p p
Find or open a file in a project - C-x p f
Switch buffers in a project - C-x p b
Delete all the project buffers - C-x p k
List the project buffers
Project.el has all except listing all the buffers. It also supports a
few other features that I think I'll find useful:
project dired - C-x p d
project shell (or eshell) C-x p s
project regex search and variants C-x p g
I'm not sure if projectile has these. It probably does and I could see
them being useful. I probable won't use the regex searches much
because I'll likely use other tools for those like consult-ripgrep.
So, it certainly seems like project.el has enough coverage for my
purposes. Another difference between the two projects is that
project.el only supports a couple of project definitions - things that
are supported by Emacs' VC system ad things supported by Emacs EDE
thing (which I've never really played with). I seem to recall
Projectile being more flexible.
In any event, project.el looks like it's a nice simple alternative at
least for me. Check out the video for the runthrough:
Using an RSS reader instead of visiting Youtube or other social media has many advantages, including that you have to deal with much less energy-sapping, will-eroding clickbait. Unfortunately feed entries can still have clickbaity titles that can DESTROY your patience! So I wrote a dumb-as-rocks declickbaiter for Elfeed. You WON’T BELIEVE the results!
elfeed-title-transform can be replaced with something much better, like a grammar aware title formatter (AI or otherwise), or with information extracted from the content or video description of the entry. But I’ll settle for this pareto solution for what was half an hour of live coding in the dark:
Okay, the above screenshots were chosen as a joke. Here’s a more representative version of the declickbait effect:
The next issue I ran into was more complex. After fixing the packages, I wanted to browse my home folder on the Mac with Dired. I got an error that stated: "Listing directory failed but 'access-file' worked". Huh? What’s that about?
When contents of a folder fails to list on a Mac, especially on more recent macOS versions, it usually means there’s a permission issue. In Catalina and Big Sur, even the Native Terminal app needs explicit access to the hard drive from the user. I always found this odd, but I guess I can see why: if you don’t know what the Terminal is and why it needs access (many users don’t, unfortunately), it’s a good idea to keep it locked. Giving Emacs explicit access to the hard drive and folders was a no-brainer, However, that didn’t help. I still received the same error message.
After some research on Stackexchange, I learned that the list (“ls”) command in macOS is different than the one in Linux. They both do the same thing and both historically date back to Unix, but they’ve developed somewhat differently. the Linux ls program is the GNU ls command, which is the one Emacs “knows” how to use. This version of ls is bundled with other command line tools inside a package called coreutils, which can be installed with Homebrew. That was my next step, but I still encountered the error.
The same place online that suggested installing coreutils also gave instructions to change the path for Bash on macOS, so it can be found. This made sense; after all, if Emacs can’t find the ls program, it would give the same error as if it’s not installed. So that was the next step. Zilch. The error persisted.
I was puzzled until a co-worker looked at this with me and noticed the path instructions are for bash (in ~/.bash_profile). Bash, as the Mac users among you know, is not the default shell on macOS since macOS Catalina; it is now ZSH. The file to configure ZSH is ~/.zshrc. That was a mistake that someone like me, who’s coming from Linux and used to Bash, would not know to change.
With the path changed in the ZSH configuration file, Emacs was finally able to find the ls program and list the directories. Well, most of them. The Desktop and the Documents folder were giving me permission denied error when I tried to list their contents. This was strange since I already gave Emacs full access to the hard drive. More online research followed. What I found was interesting.
On macOS, Emacs runs inside a Ruby wrapper (this detail is buried on the Emacs wiki). So, while it’s nice that I gave Emacs access to the hard drive, Ruby, which is (from my understanding) the “messenger” Emacs sends out to use the ls command in Dired, was denied access. The fix was to add Ruby to the access list under Security in macOS settings1.
Only after Ruby had access I was able to see all the local directories I wanted. It’s odd just how alienated macOS “feels” Emacs is. I understand the logic of what’s going on, but still, the notion that I need to customize so many things to let something as free and good as Emacs is something I’d grown to expect from Windows, not macOS. Oh well. C’est la Vie.
Footnotes
This doesn’t explain why other directories worked fine without giving Ruby explicit access. For example, I could view my Downloads folder just fine on the Mac. From what I read, I understand this has to do with the integration of iCloud, which the Desktop and the Document folders are a part of. My guess is that Ruby needs access to iCloud, or, that you don’t access these directories at all, but some sort of cloud integration that just seems like folders, when in fact they are not. Whatever the case may be, that was the fix. ↩︎
M-x Research (contact them for password): TBA https://m-x-research.github.io/ Fri Oct 15 0700 Vancouver / 0900 Chicago / 1000 Toronto / 1400 GMT / 1600 Berlin / 1930 Kolkata / 2200 Singapore
Emacs FFM: Emacs FFM Meetup! October 2021 https://www.meetup.com/emacs-ffm/events/281293223/ Wed Oct 20 1000 Vancouver / 1200 Chicago / 1300 Toronto / 1700 GMT / 1900 Berlin / 2230 Kolkata – Thu Oct 21 0100 Singapore
Emacs APAC (virtual, in English) https://emacs-apac.gitlab.io/ Sat Oct 23 0130 Vancouver / 0330 Chicago / 0430 Toronto / 0830 GMT / 1030 Berlin / 1400 Kolkata / 1630 Singapore
They all have advantages and disadvantages. A hard reset is fast but the discarded commits are deleted. The other two methods are safer but involve more typing. You can get all the details by watching the video.
Using either reset can seem like a scary proposition but Whatacold shows that they are both straightforward and easy to understand. If they still seem too fraught, you can always use revert. It generates more commits but doesn’t seem as scary.
The video is 9 minutes, 25 seconds so it should be pretty easy to fit in.
Grading sucks. Even for a well crafted assignment it takes time and
rarely does it give you the same insights into your students as you
can get from just observing them and working with them, at least when
the environment supports you doing so.
This leads lots of teachers to go to auto graders. I can't bring
myself going along with that. While assignments are imperfect and
tedious to grade, they do provide some insight to your students and
that's an important part about being a teacher.
So, I always keep my eye out for ways to improve my project gradiing
workflow.
In the video below, I step through my process. It's mostly Emacs but I
also use GitHub classroom and the terminal along the way.
If you're here as for the Emacs, I'd any thoughts on alternatives or
improvements.
If you're here as a teacher, I'd love to hear what your flow is like
and what tools you use.
