Irreal: Visualizing Data With Emacs And Gnuplot

I really like Gnuplot. It can produce most of the types of graphs that you’d actually want to produce and it’s reasonably flexible. That said, I hardly ever use it. It’s sort of a chicken/egg problem: on the one hand, I don’t use gnuplot very often because I can’t remember how to use its (semi-arcane) command language. On the other hand, I can’t remember the language because I seldom use it.

I just ran across a very short video from Anand Tamariya that shows the user selecting a rectangle of data, calling gnuplot-rectangle, and having a graph of the data be rendered by gnuplot. The video is only 35 seconds and doesn’t have any explanation. I was bitterly disappointed because it seemed like an easy way to use gnuplot for many of the cases I care about.

Serendipitously, I had another tab open in my browser by the author of the video that showed the code for gnuplot-rectangle and explained how to use it. There’s not much to do. You have to have gnuplot installed on your system, of course, and you also need to have the Emacs gnuplot package installed. That package is available from the Melpa and NonGNU elpa repositories so it’s easy to install. If you’re like me, you probably already have them installed.

The final piece is the gnuplot-rectangle function that Tamariya gives in his explanatory post. I haven’t had a chance to try it yet but it seems just perfect for folks like me that don’t need to draw graphs often enough to learn the intricacies of gnuplot.

-1:-- Visualizing Data With Emacs And Gnuplot (Post jcs)--L0--C0--March 28, 2024 03:42 PM

Irreal: An Explanation Of The Mark Ring

Arialdo Martini has a very informative post about the Emacs mark ring and how it works. On the surface, it’s pretty simple. The mark ring is a ring buffer that records positions inside a buffer—or several buffers in the case of the global mark ring—and allows you to return to or otherwise use those positions.

When you dig a little deeper, you find that there’s a lot more to it. For example, the mark helps define a region in an Emacs buffer and that fact can be leveraged to perform all sorts of useful tasks.

The mark ring actually stores markers to a given position so they are automatically adjusted if the absolute position of a place changes. Thus, you can, for example, push a position onto the mark ring, add text above that position, and still jump to where the mark was set even though its absolute position has moved.

In addition to allowing you to push positions onto the mark ring, Emacs will also record positions for you. For example, if you jump to a new position, your old position is pushed onto the mark ring. That allows you to do things like search for a symbol and then jump back to where you started. The exact rules for when Emacs pushes the mark onto the mark ring or global mark ring are a bit complex. This Irreal post discusses a video the explains some of those complexities.

The simple mark ring mechanism enables all sorts of useful functionality. Take a look at Martini’s post for some examples.

-1:-- An Explanation Of The Mark Ring (Post jcs)--L0--C0--March 27, 2024 04:13 PM

Protesilaos Stavrou: Emacs: consult-denote developer preview

I just published the prototype of consult-denote: https://github.com/protesilaos/consult-denote.

The idea is to use Consult across the various prompts of Denote to get the helpful features of previewing and narrowing. In addition, we can leverage Consult to perform searches in the denote-directory.

At this stage, I want to get a better sense of what we can do with Consult to help users of Denote. If this is something people are interested in, then we can check how to extend it further and eventually produce a formal package for inclusion in GNU ELPA.

What consult-denote does as of this writing:

  • Provides a drop-in replacement for the denote-file-prompt. This is used in a variety of commands, such as those that link between Denote files.
  • Defines a source for consult-buffer which lists buffers that visit Denote files. We call these “Denote buffers”.
  • Introduces thin wrappers to search by files contents or name in the denote-directory. The commands are consult-denote-grep and consult-denote-find. The user options consult-denote-grep-command and consult-denote-find-command control which Consult function is called.
  • Adds the consult-denote-mode to set up Consult-powered prompts in place of the generic ones that Denote offers. Currently, this is not done in the way I intend to do it: it uses the advice mechanism, though my plan is to modify denote.el (or others) to run the function prompts via a variable, thus giving packages like this one the option to cleanly define their own replacements.

I repeat, this is a prototype. I am not distributing a user manual at this stage, nor am I doing any video demonstration. Let us first have a fully fledged package. Then we go with the flow.

-1:-- Emacs: consult-denote developer preview (Post)--L0--C0--March 27, 2024 12:00 AM

M. Rincón: Eat Evil

Eat is a terminal emulator for Emacs available in NonGNU ELpa. Eat is much slower than foot, but faster than term.el. It has some nice features like sixel support, and being able to send messages to Emacs. It also has some surprising idiosyncrasies which makes adding Evil key bindings and other modifications challenging.

I think that the default state, semi char mode, captures too many key combinations. This can be changed by modifying eat-semi-char-non-bound-keys and reloading eat. But doing that inside a whith-eval-after-load block creates an infinite loop. To prevent the loop, the reloading should be done inside a let block:

(with-eval-after-load 'eat
 (setq eat-semi-char-non-bound-keys (list [?\C-\\]
 [?\C-w]
 [?\C-h]
 [?\C-x]
 [?\e ?x]))
 (eat-update-semi-char-mode-map)
 (let ((after-load-alist nil)
 (after-load-functions nil))
 (eat-reload)))

Though eat doesn't work well with Evil, eat is a modal terminal emulator. And these modes can be leveraged to make it a bit more evil. This requires starting eat with a new function, my-eat-start:

(defun my-eat-end ()
 "Kill and close the eat buffer."
 (interactive)
 (eat-kill-process)
 (kill-this-buffer))

(defun my-eat-emacs-mode ()
 "Help create a normal state with `eat-emacs-mode`."
 (interactive)
 (eat-emacs-mode)
 (evil-define-key* 'normal eat-mode-map (kbd "[[") 'eat-previous-shell-prompt)
 (evil-define-key* 'normal eat-mode-map (kbd "]]") 'eat-next-shell-prompt)
 (evil-define-key* 'normal eat-mode-map (kbd "i") 'my-eat-semi-char-mode)
 (evil-define-key* 'normal eat-mode-map (kbd "gO") 'browse-url)
 (evil-define-key* 'normal eat-mode-map (kbd "q") 'my-eat-end)
 (evil-normal-state))

(defun my-eat-semi-char-mode ()
 "Create bindings for `eat-mode` and switch to Emacs state."
 (interactive)
 (eat-semi-char-mode)
 (keymap-set eat-mode-map "C-w h" 'evil-window-left)
 (keymap-set eat-mode-map "C-w l" 'evil-window-right)
 (keymap-set eat-mode-map "C-w k" 'evil-window-up)
 (keymap-set eat-mode-map "C-w j" 'evil-window-down)
 (keymap-set eat-mode-map "C-w +" 'evil-window-increase-height)
 (keymap-set eat-mode-map "C-w -" 'evil-window-decrease-height)
 (keymap-set eat-mode-map "C-w <" 'evil-window-decrease-width)
 (keymap-set eat-mode-map "C-w >" 'evil-window-increase-width)
 (keymap-set eat-mode-map "C-w =" 'balance-windows)
 (keymap-set eat-mode-map "C-w R" 'evil-window-rotate-upwards)
 (keymap-set eat-mode-map "C-w r" 'evil-window-rotate-downwards)
 (keymap-set eat-mode-map "C-w W" 'evil-window-prev)
 (keymap-set eat-mode-map "C-w w" 'evil-window-next)
 (keymap-set eat-mode-map "C-w s" 'evil-window-split)
 (keymap-set eat-mode-map "C-w v" 'evil-window-vsplit)
 (keymap-set eat-mode-map "C-w _" 'evil-window-set-height)
 (keymap-set eat-mode-map "C-w |" 'evil-window-set-width)
 (keymap-set eat-mode-map "C-w c" 'evil-window-delete)
 (keymap-set eat-mode-map "C-w n" 'evil-window-new)
 (keymap-set eat-mode-map "C-w o" 'delete-other-windows)
 (keymap-set eat-mode-map "C-w p" 'evil-window-mru)
 (keymap-set eat-mode-map "C-w q" 'evil-quit)
 (keymap-set eat-mode-map "C-w N" 'my-eat-emacs-mode)
 (keymap-set eat-mode-map "C-\\ C-n" 'my-eat-emacs-mode)
 (keymap-set eat-mode-map "C-\\ C-p" 'eat-send-password)
 (keymap-set eat-mode-map "C-w \"" 'eat-yank)
 (keymap-set eat-mode-map "C-x b" 'switch-to-buffer)
 (keymap-set eat-mode-map "C-x k" 'my-eat-end)
 (keymap-set eat-mode-map "C-x p" 'eat-send-password)
 (evil-emacs-state)
 (goto-char (point-max)))

(defun my-eat-start ()
 "Start Eat in the current project root directory in another window."
 (interactive)
 (require 'project)
 (require 'eat)
 (let* ((proj (project-current nil)))
 (if proj (let* ((default-directory (project-root proj))
 (eat-buffer-name (project-prefixed-buffer-name "eat")))
 (eat-other-window)
 (switch-to-buffer eat-buffer-name)
 (my-eat-semi-char-mode))
 (progn (eat-other-window)
 (switch-to-buffer eat-buffer-name)
 (my-eat-semi-char-mode)))))

Eat also provides great color support; however, if .bashrc is making some color settings based on the terminal name, the file needs to be modified.

### Color support
case "${TERM}" in
 xterm-color|*-256color|foot|*-truecolor)
 echo Making color settings...
 ;;
 ,*)
 echo :(
 ;;
esac

if [ -n "$EAT_SHELL_INTEGRATION_DIR" ]; then
 if [ -r "$EAT_SHELL_INTEGRATION_DIR/bash" ]; then
 # shellcheck source=/dev/null
 source "$EAT_SHELL_INTEGRATION_DIR/bash"
 fi
fi

One nice thing about eat is that it can communicate with Emacs. With a few modifications, eat can open a file inside the host Emacs. With this change the command _eat_msg open will open a file in Emacs.

(defun my-eat-open (file)
 "Helper function to open files from eat terminal."
 (if (file-exists-p file)
 (find-file-other-window file t)
 (warn "File doesn't exist")))

(add-to-list 'eat-message-handler-alist (cons "open" 'my-eat-open))

And for ease of use, the command can be aliased toe emopen in .bashrc.

alias emopen='_eat_msg open'
-1:-- Eat Evil (Post)--L0--C0--March 27, 2024 12:00 AM

Irreal: Casual: A Porcelain For Calc

Those of you who have been reading Irreal for a while know that I’m a huge fan of Emacs Calc. It’s tremendously flexible and comprehensive and it’s built into Emacs. But that comprehensiveness comes at a price: it’s very hard to learn its commands and they always prove evanescent if you don’t use them regularly. I’ve been through the manual a couple times and I still can’t remember how to use most of Calc. I almost always bring up the cheat sheet when I need to do anything more than the basic calculator functions.

Charles Choi to the rescue. We haven’t heard from Choi for a while and now we know why. He’s been busy building a porcelain for Emacs Calc. It’s a transient menu that helps you remember many of the esoteric Calc commands. When you get stuck, you can bring up the menu and navigate your way to the proper command.

Choi says that he’s not trying to provide full coverage of the Calc commands, just the most—for some value of “most”—used commands. You can read about his goals and non-goals at his post.

The package is on MELPA and although Choi doesn’t give the use-package recipe for installing it, here’s what I used:

(use-package casual
  :ensure t
  :bind (:map calc-mode-map ("C-o" . 'casual-main-menu)))

I just installed it so I don’t have a lot of experience with it yet but so far I really like it. The menu doesn’t appear until you invoke it and it disappears after you choose a command so it doesn’t get in the way. I can see this package making Calc more approachable and increasing its usage. If you’re a Calc user or would like to be, you definitely want to take a look at Choi’s post.

-1:-- Casual: A Porcelain For Calc (Post jcs)--L0--C0--March 26, 2024 03:59 PM

Ben Simon: out-of-band is officially out-of-bounds | Reviving emacs oauth2 with Google APIs

For months, I happily edited blog posts in Emacs using a tiny Blogger client I wrote in Elisp called hbo-blogger.el. hbo-blogger.el was easy to write, thanks in part to oauth2.el, which simplifies authentication against Google's Blogger API.

Things all came crashing down one day when I needed to re-authenticate using Google's OAuth2 server. I executed the following Elisp code, expecting it to open a browser, ask me a few questions, and show me a code I could paste back into Emacs.

(oauth2-auth-and-store
       "https://accounts.google.com/o/oauth2/v2/auth"
       "https://oauth2.googleapis.com/token"
       hbo-blogger-oauth-scope
       hbo-blogger-client-id
       hbo-blogger-client-secret)

However, this time I got an error announcing "Access blocked: Blur's request is invalid." Digging deeper took me to this page: Access blocked: Third-party app is using an unsupported OAuth method.

The issue is that oauth2.el sets up authentication using the redirect URL urn:ietf:wg:oauth:2.0:oob. The 'out-of-band' (oob) redirect URL handler works great when you are performing OAuth in a command-line or, in this case, an Elisp context. However, Google now denies this type of redirect URL.

Without a way to authenticate against the Blogger API, hbo-blogger was useless, and I was forced to go back to editing posts over at blogger.com.

What I needed were two things: support for a redirect URL that Google would be happy with, and a way to call oauth2.el using that new URL.

Good news: I now have both of these components, and if you want, you can have them too.

Setting up a Kosher OAuth2 Redirect Handler

First off, I built out oauth2handler. This is a bit of PHP code that runs on a publicly exposed Linux box. It allows you to set up a 'real' redirect URL without having to write any custom code. It serves as a sort of universal OAuth2 redirect URL.

Annoyingly, setting up oauth2handler takes about 12 steps. However, none of the steps are very hard; you just need to be patient.

With oauth2handler, I had a redirect URL. However, I needed a way to integrate this URL into oauth2.el. You can find the code for doing this over at oauth2handler.el.

Like oauth2handler (and heck, all things OAuth2), there's a heap of things to set up. But, once you've got everything properly defined, performing OAuth authentication is easy. Here's a function I can call to initialize the OAuth process:

;; Grab these values from the Google developer console.
(defvar hbo-blogger-client-id nil)
(defvar hbo-blogger-client-secret nil)

;; These can remain set to their defaults for blogger
(defvar hbo-blogger-oauth-scope
  "https://www.googleapis.com/auth/blogger")
(defvar hbo-blogger-oauth-auth-url
  "https://accounts.google.com/o/oauth2/auth")
(defvar hbo-blogger-oauth-token-url
  "https://www.googleapis.com/oauth2/v3/token")

;; Point this to your oauth2handler URL. For me, this is:
;; (setq hbo-blogger-oauth-redirect-url
;;      "https://code.benjisimon.com/oauth2handler/emacs_hbo")
(defvar hbo-blogger-oauth-redirect-url nil)

(defun hbo-blogger-auth-start ()
  "Start the Oauth2 authentication process in the browser"
  (interactive)
  (oauth2handler-start-auth
   hbo-blogger-oauth-redirect-url
   hbo-blogger-oauth-auth-url
   hbo-blogger-oauth-token-url
   hbo-blogger-oauth-scope
   hbo-blogger-client-id
   hbo-blogger-client-secret))

Running M-x oauth2handler-start-auth opens up a new web browser, and just like the 'oob' style of authentication, it prompts me to grant permission. The final page of oauth2handler displays a blob of JSON, similar to the code that the oob URL produces. I copy that JSON into an Emacs buffer that's been opened for me and hit C-c C-c.

Hitting C-c C-c generates the oauth2.plstore file that oauth2.el expects. From there, oauth2 and by extension hbo-blogger.el work again as they should.

In my .emacs.d/init.el file, I have the following settings and function call:

(require 'hbo-blogger)
(setq hbo-blogger-client-id  "<from the google developer's console>")
(setq hbo-blogger-client-secret "<from the google developer's console>")
(setq hbo-blogger-oauth-redirect-url
      "https://code.benjisimon.com/oauth2handler/emacs_hbo")

(hbo-blogger-auth-setup)

hbo-blogger-auth-setup uses oauth2.el's oauth2-auth-and-store function to set up a valid OAuth2 token. This token is what hbo-blogger.el uses to communicate with Blogger's API.

This whole process has been frustrating and exceedingly delicate. However, now that I've got oauth2handler and the corresponding Elisp code, life is good again. It feels good to be back home (editing in Emacs).

-1:-- out-of-band is officially out-of-bounds | Reviving emacs oauth2 with Google APIs (Post Ben Simon (noreply@blogger.com))--L0--C0--March 26, 2024 10:14 AM

Manuel Uberti: Food for Profit

Last night, as I was walking out of the cinema where I had just watched Food for Profit,1 I could not help but think about the people around me. It is pretty easy to say that maybe I am not the kind of person Innocenzi’s and D’Ambrosi’s documentary aims at. Why would I need an investigation such as this to move me towards ethical choices that by now I have already been following for over ten years? And yet, beside supporting this much too necessary endeavor, I was curious to sit in a room with other people and sense the impact its images would have on eyes possibly not accustomed to the reality of intensive farming.

Let me start by simply saying that if one cares in any way about the world we live in, then they should not ignore this work. Furthermore, at the moment in Italy it is nearly impossible not to find a screening. Its authors have been literally reaching every corner of this country to make sure everyone has a chance to see the results of their five-year long effort. Considering that this is an independent film which relies on word of mouth, because big producers and giant distributors have no interest in alienating the meat industry, Innoncenzi’s and D’Ambrosi’s willingness to share what they believe in is absolutely remarkable.

However, I had my misconceptions about the film. Somehow I was expecting it to be solely criticism about European agricultural politics, as if the whole point of the documentary was to expose politicians not behaving as one would suppose, which would have been barely a surprise nowadays. This is only a part of what Innoncenzi and D’Ambrosi do, though. They cannot and do not want to hide the inevitable consequences of what some inept members of the European Parliament agree upon. Animal suffering, diseases, environmental disasters, exploitation of workers: it is all here in its vivid, horrific evidence.

After what I learnt from Earthlings2 and Dominion3 I should be used to mindless violence, but the shock always manages to unsettle me, just as it always takes me a moment to recover from the fact that we seem incapable of treating animals and other people with the same principles of humanity we believe to have perfected over the course of history. Food for Profit acts as a powerful reminder of how we constantly and comfortably forget those principles in order to enjoy privileges that we choose not to question. As Innocenzi said to us at the end of the screening, we usually expect documentaries such as this from the US. It is about time Europe comes to terms with the same atrocities that we like to think are far way from us.

This is what one should really focus on as they watch the film. I can hardly trust critics who pick at Innoncenzi for posing herself in a sort of egocentric, Michael Moore-like kind of way.4 On the contrary, I believe that she needs to put herself right within these images and back them as strongly as she can. She has every right to use her public persona to engage with viewers, because I do not see this work as art for art’s sake, nor I see any point in approaching it with the dissecting eye of a film critic. There is simply too much at stake in this documentary to look at it with questionable, rational detachment or reductionist arguments.

And this is why last night I found people’s reactions more honest and important than any critique. We were all shaken by the cruelty and the indifference we had witnessed, but I was genuinely surprised by the sincere concern many showed at the end. Innocenzi kindly replied to all the questions, most of which dealt with what can we do now, what is actually feasible and what, if any, are the hopes for a better world. If one, like me, finds it really difficult to believe that the planet and whoever lives on it can put up with human actions any longer, people’s response to Food for Profit will be enough to placate the heart for a while.


  1. See: Food for Profit↩︎

  2. See: Earthlings↩︎

  3. See: Dominion↩︎

  4. See Matteo Marelli’s “review” on FilmTv. The cheap scepticism used to dismiss the actual point of Food for Profit tells me more about Marelli than the film itself. ↩︎

-1:-- Food for Profit (Post Manuel Uberti (manuel.uberti@inventati.org))--L0--C0--March 26, 2024 06:50 AM

Jiewawa: Blogging with Denote and Hugo

A few months ago, I replaced Org-Roam with Prot’s Denote package and have been using it for a while now. I have mostly been using it to manage my collection of personal notes, but I also want the option to publish any note as a blog post quickly and easily.

Since starting to use Denote, I am trying to think of notes and public blog posts not as two separate entities, but as a part of a larger knowledge base. Unless I’m writing something specific like notes on a book, which would go in the /books subdirectory, most notes start their life in the root of my denote-directory before being moved to a relevant subdirectory. Blog posts are moved to the /blog subdirectory.

With ox-hugo, you can write all posts in a single Org file, or like I do, you can have a single file for each post. ox-hugo also provides a really useful feature for auto-exporting on save. I enabled it by adding the following in the .dir-locals.el file in the /blog subdirectory.

((org-mode . ((eval . (org-hugo-auto-export-mode))
(org-hugo-section . "blog"))))

The three stages of a blog post

I make use of Denote’s file-naming scheme to easily distinguish between each of the following states that a note can be in:

  • Private note: Located in the root of my denote-directory

  • Draft blog post: Located in the /blog subdirectory, with the keyword draft

  • Published blog post: Located in the /blog subdirectory, without the keyword draft

    Below is truncated tree view of my denote-directory:

├── 20230708T141810--books__metanote.org
├── ........
├── blog
│   ├── ........
│   ├── 20231104T134226--a-dwim-fullscreen-function-for-emacs-and-sway__emacs_linux.org
│   └── 20240323T143034--hacking-on-denote-and-hugo__blogging_denote_draft_emacs.org

Linking

Prot introduced a new set of link parameters specifically for Denote. The file’s Denote identifier is used to refer to the file you want to link to [[denote:20230708T141810][Books]] and the title of the file is the description. I have seen some Emacs users complain that these links are not portable, but for me they are very useful as the links continue to work even when I move files to different subdirectories.

I modified the included denote-link-ol-export so that links to private notes or drafts are not exported, but links to public notes are exported as Hugo relative links. There is only one change from the original function, in which I use my/denote-markdown-export to determine which type of note the link refers to and format it accordingly.

(defun my/denote-link-ol-export (link description format)
" Modified version of `denote-link-ol-export'.
Replace markdown export with `my/denote-markdown-export'

Original docstring below:
Export a `denote:' link from Org files.
The LINK, DESCRIPTION, and FORMAT are handled by the export
backend."
(let* ((path-id (denote-link--ol-resolve-link-to-target link :path-id))
(path (file-relative-name (car path-id)))
(p (file-name-sans-extension path))
(id (cdr path-id))
(desc (or description (concat "denote:" id))))
(cond
((eq format 'html) (format "<a href=\"%s.html\">%s</a>" p desc))
((eq format 'latex) (format "\\href{%s}{%s}" (replace-regexp-in-string "[\\{}$%&_#~^]" "\\\\\\&" path) desc))
((eq format 'texinfo) (format "@uref{%s,%s}" path desc))
((eq format 'ascii) (format "[%s] <denote:%s>" desc path)) ; NOTE 2022-06-16: May be tweaked further
((eq format 'md) (my/denote-markdown-export link desc))
(t path))))
(defun my/denote-markdown-export (link desc)
"Format the way Denote links are exported to markdown.
 If LINK is considered private or a draft, return DESC.
 If LINK is considered a public note, format it as a Hugo relative link. "
(let ((path (denote-get-path-by-id link)))
(if (not (string-match "/blog" path))
(format "%s" desc)
(if (string-match "_draft" path)
(format "%s" desc)
(format "[%s]({{< relref \"%s\" >}})"
desc
(denote-sluggify-title
(denote-retrieve-filename-title path)))))))
(org-link-set-parameters "denote" :export #'my/denote-link-ol-export)

With that, I am able to covert the following Org-Mode links:

This is a private note: [[denote:20230708T141810][Books]]
This is a draft blog post: [[denote:20240323T143034][Hacking on Denote and Hugo]]
This is a published blog post: [[denote:20231104T134226][A DWIM fullscreen function for Emacs and Sway]]

Into this Markdown in which only public notes are exported:

This is a private note: Books
This is a draft blog post: Hacking on Denote and Hugo
This is a published blog post: [A DWIM fullscreen function for Emacs and Sway]({{< relref "a-dwim-fullscreen-function-for-emacs-and-sway" >}})

Moving between the three stages

The following code requires Denote version 2.3.0 as my/insert-hugo-export-file-name makes use of the denote-sluggify-title, if you are using an earlier version of Denote, you can replace that function with denote-sluggify.

Changing a note from a draft is quite simple. my/denote-convert-note-to-blog-post is a wrapper for simple functions that do the following:

  1. Move file to /blog subdirectory
  2. Add Denote keyword draft
  3. Add #+hugo_draft: t to file metadata
  4. Sluggify Denote title and add #+export_file_name to file metadata
(defun my/insert-hugo-draft-status ()
"Add metadata to current org-mode file marking it as a Hugo draft."
(save-excursion
(goto-char 0)
(search-forward "filetags")
(end-of-line)
(insert "\n#+hugo_draft: t ")))
(defun my/insert-hugo-export-file-name ()
"Add metadata to current org-mode file containing export file name.
 Export File Name is returned by `denote-retrieve-title-value'."
(save-excursion
(goto-char 0)
(search-forward "filetags")
(end-of-line)
(insert (format
"\n#+export_file_name: %s.md"
(denote-sluggify-title
(denote-retrieve-title-value buffer-file-name 'org))))))
(defun my/move-current-buffer-file-to-subdirectory (subdirectory)
"Move file of current buffer to SUBDIRECTORY seamlessy.
 There should be no discernable difference in the buffer's appearance."
(let ((input-file (buffer-file-name))
(output-file (concat (denote-directory)
subdirectory
(file-relative-name (buffer-file-name))))
(window-pos (window-start))
(cursor-pos (point)))
(save-buffer)
(rename-file input-file output-file)
(find-file output-file)
(set-window-start (selected-window) window-pos)
(goto-char cursor-pos)
(kill-buffer (get-file-buffer input-file))))
(defun my/denote-convert-note-to-blog-post ()
"Mark file of current Denote buffer to be marked as a draft blog post."
(interactive)
(my/move-current-buffer-file-to-subdirectory "blog/")
(denote-keywords-add '("draft"))
(my/insert-hugo-draft-status)
(my/insert-hugo-export-file-name))

To change a note from a draft to a public note, my/denote-publish-hugo-post is another wrapper around simple functions that do the following:

  1. Format current date and add #+hugo_publishdate to file metadata
  2. Remove Org-Mode draft tag
  3. Remove #+hugo_draft: t from file metadata
  4. Rename file and remove draft from Denote keywords
 (defun my/insert-hugo-published-date ()
"Format the current date and add it to Org-Mode metadata."
(save-excursion
(goto-char 0)
(search-forward "filetags")
(end-of-line)
(insert (concat "\n#+hugo_publishdate: "(format-time-string "%Y-%m-%d")))))
(defun my/denote-remove-draft-tag-from-metadata ()
"Remove \"draft\" tag from Org-Mode metadata."
(save-excursion
(goto-char 0)
(search-forward "filetags")
(when (search-forward-regexp ":draft\\|draft:" (line-end-position) t)
(replace-match "")
(when (and (looking-at ":$\\|: ") (looking-back " "))
(delete-char 1)))))
(defun my/denote-remove-hugo-draft-status ()
"Remove Hugo draft entry from Org-Mode metadata."
(save-excursion
(goto-char 0)
(when (search-forward "#+hugo_draft: t")
(beginning-of-line)
(kill-line)
(kill-line))))
(defun my/denote-remove-keyword-from-filename (keyword)
"Remove Denote keyword \"draft\" from filename of current file."
(denote-rename-file buffer-file-name
(denote-retrieve-filename-title buffer-file-name)
(delete keyword
(denote-retrieve-keywords-value
buffer-file-name 'org))
nil))
(defun my/denote-publish-hugo-post ()
"Mark file of current `denote' buffer to be published as a blog post."
(interactive)
(my/insert-hugo-published-date)
(my/denote-remove-hugo-draft-status)
(my/denote-remove-draft-tag-from-metadata)
(my/denote-remove-keyword-from-filename "draft"))

Conclusion

Are these tweaks going to make me a prolific blogger? Probably not, but they gave me a good opportunity to craft a comfy setup while also practising some Emacs Lisp.

-1:-- Blogging with Denote and Hugo (Post)--L0--C0--March 26, 2024 12:00 AM

Evan Moses: Representing State as interfaces in Go

I made up a neat little pattern in Go the other day. It’s a way to represent a state change in a system by exposingdifferent APIs for different states, while only holding state in a single underlying struct. I’m sure I’m not the firstperson to invent this, and it may already a name, so please let me know if you know of one [Update:apg on Lobsters pointed out thename “typestate”, which I like]. I’m goingto show an instance of the pattern first and the motivation after.

The Pattern

// You start with a Resolver, and incrementally feed it data to be looked uptype Resolver interface {    // Collect collects some piece of data and maybe extra information about it that you    // need to do resolution    Collect(someId string, someData string)    // Maybe you also need some global contextual data to do the resolve    AddContextualData(data SomeContext)    // When you're done, the resolve can send the bulk query to the executor to perform    // the lookup that you want to do    Execute(context.Context, Executor) (Resolved, error)}// Resolved is what you get back after you Execute, and it lets you access the resolved datatype Resolved interface {    // Resolve returns the result. If the result doesn't have that id, either because it    // wasn't looked up successfully during execution or because you never [Collect]ed it,    // it will return ("", false)    Resolve(id string) (string, bool)}// Executor is capable of doing the db lookup/cache lookup/service request/http request// that actually gets the data you need to gettype Executor interface {    DoTheThing(        ctx context.Context,        idsToResolve map[string][]string,        contextualData SomeContext,    ) ([]ResolveResult, error)}

So far, so boring. You’ve got a Resolver that batches up query data, an Executor that, well, executes the query,and a Resolve that lets you access the result. What’s neat about this is that Resolver and Resolved can beimplemented by the same struct:

type idResolver struct {collected map[string][]stringcontextData SomeContextresolved map[string]string}func (r *idResolver) Collect(someId string, someCategory string) {r.collected[someCategory] = append(r.collected[someCategory], someId)}func (r *idResolver) AddContextualData(data SomeContext) {r.contextData = data}func (r *idResolver) Execute(ctx context.Context, executor Executor) (Resolved, error) {result, err := executor.DoTheThing(ctx, r.collected, r.contextData)if err != nil {return nil, err}r.resolved = transformResult(result)/*** * 🪄 THE MAGIC 🪄 * Ooh look I'm just returning this struct as a Resolved! */return r, nil}func (r *idResolver) Resolve(id string) (string, bool) {res, ok := r.resolved[id]return res, ok}

Ok but why?

I was building a system to bulk-lookup names in my system by ID. But if you called Resolve before it had executed,you’d have an invalid result. And if you Collected after you executed, you’d never look up the id. So I added aboolean hasExecuted to IdResolver so that if you called Resolve before you had called Execute or Collectafter, it would check that flag and panic.

func (r *StatefulIdResolver) Collect(someId string, someCategory string) {    if r.hasExecuted {        panic("Collect called after Execute")    }r.collected[someCategory] = append(r.collected[someCategory], someId)}// Execute executes and changes state but doesn't return anything newfunc (r *StatefulIdResolver) Execute(ctx context.Context, executor Executor) error {    if r.hasExecuted {        panic("Execute called twice")    }    ...    r.hasExecuted = true    ...}func (r *StatefulIdResolver) Resolve(id string) (string, bool) {    if !r.hasExecuted {        panic("Resolve called before execute")    }res, ok := r.resolved[id]return res, ok}

This is kind of a mess, harder to read and maintain, and much easier to mess up the logic in.

When to use it

Of course the Resolver and the Resolved could be represented by different structs with their own methods. But inthis case it felt like we were really working with a single object: it collects data, operates on it, and manages theresult.

The crux of the matter is that the object has different operations that are valid in different states, and Go interfacesare a perfect way to expose that.

Update: Is this just a Builder?

A few folks on the internet have pointed out that this is very similar to the Builder pattern that you see prettyoften in Java/C# (and to a lesser extent in Go), and especially a multiphase Step Builder.

I don’t think there’s a fundamental difference, but Builders that I’m familar with generally have a state-change/executestep (Build()) that produces an immutable object for use somewhere else, rather than doing any sort of effectfulexecution. I think this pattern is both more general (you could certainly use it as a Builder) and has a differentpurpose.

-1:-- Representing State as interfaces in Go (Post Evan Moses (webmaster@emoses.org))--L0--C0--March 26, 2024 12:00 AM

Alvaro Ramirez: Emacs 29.3 emergency release

25 March 2024 Emacs 29.3 emergency release

It was only last week when I upgraded to Emacs 29.2. Yup, I was late to the party. This week, we have a new release.

Emacs 29.3 is an emergency bugfix release, so this time I've upgraded promptly. I'm on macOS using the great Emacs Plus so upgraded via Homebrew using:

brew reinstall emacs-plus@29 --with-imagemagick --with-no-frame-refocus --with-native-comp --with-savchenkovaleriy-big-sur-3d-icon --with-poll

29.3.png

ps. Like this splash screen? Check out the Emacs eye candy post.

-1:-- Emacs 29.3 emergency release (Post)--L0--C0--March 25, 2024 10:28 PM

Irreal: Emacs 29.3 Is Available

Eli Zaretskii writes to tell us that Emacs 29.3 has been released. It’s the first time in a long time—or maybe ever—that I can recall Emacs releasing a security update. It’s an editor, after all, what security issues could it have. The problem is that Emacs is not just an editor: it’s a complete operating environment and some of those files it works on can have active elements.

You can read about the specific issues that the update addresses in the NEWS file but they’re just what you’d think they would be. The fixes mainly tighten down on existing safeguards that prevent Emacs from executing possibly malicious Elisp.

As I always say, thanks to Eli and the other developers for their work on this release. It’s not the type of thing that gets a lot of notice and hosannas. It’s a simple security update: boring but nevertheless vital. That makes it all the more worthy of our thanks.

Afterword

Ihor Radchenko announced that there’s a corresponding update to Org mode so be sure to update them both.

-1:-- Emacs 29.3 Is Available (Post jcs)--L0--C0--March 25, 2024 04:21 PM

Sacha Chua: 2024-03-25 Emacs news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, kbin, programming.dev, lemmy, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

-1:-- 2024-03-25 Emacs news (Post Sacha Chua)--L0--C0--March 25, 2024 12:24 PM

Troy Hinckley: Bump Allocation: Up or Down?

Back in 2019, Nick Fitzgerald published always bump downwards, an article making the case that for bump allocators, bumping “down” (towards lower addresses) is better than bumping up. The biggest reasons for this are bumping up requires 3 branches vs 2 for bumping down and rounding down requires fewer instructions than rounding up. This became the method used for the popular bumpalo crate. In this post, I want to go back and revisit that analysis.
-1:-- Bump Allocation: Up or Down? (Post)--L0--C0--March 25, 2024 12:00 AM

Charles Choi: Announcing Casual - An opinionated porcelain for Emacs Calc

Casual Main Menu Screenshot

Announcing the first public release of Casual - an opinionated Transient-based porcelain to support the casual usage of Emacs Calc.

It’s now on MELPA and requires Emacs 29.1+.

Install

Add the following lines to your Emacs initialization file.

(require 'casual)
(define-key calc-mode-map (kbd "C-o") 'casual-main-menu)

Motivation

While Emacs Calc has an embarrassingly rich feature set, for many users this capability is inaccessible due its dependence on an overwhelming number of keybindings to access them. These keybindings have a steep learning curve that is quickly lost if not in constant use.

Menus are a user interface (UI) affordance that offer users discoverability and recall. Providing a hierarchical menu UI over Calc greatly improves its casual use.

Goals

  • To provide a keyboard-driven UI to Calc that is menu based.
  • To allow users experienced with scientific calculators to casually use Calc without having to reference its Info pages.

Non-Goals

  • Full coverage of all Calc commands. Casual is not intended to be a power user tool.
  • Strict adherence to default Calc keybindings. Calc’s prompt-first interface resulted in a command keybinding design that embeds context in the key sequence (e.g. v for vector, b for binary). Hierarchical menus make this context implicit, allowing for key reuse in different contexts.
  • Strict adherence to Calc command naming. While Casual is mostly in alignment with Calc’s command naming, there are cases where it will make an opinionated change if the name is deemed too vague or idiomatic.
  • UX Stability (for now). Given that Casual is early in its life-cycle, expect changes to its user experience in terms of menu hierarchy and keybinding choices in future releases.

Usage

  1. Invoke M-x calc to launch Calc.
  2. When the point is in the Calc window, invoke C-o (or a binding of your choosing) to launch the Casual menu.

Calc Basics

It helps to know some basics about Calc.

  • Calc is a stack-based calculator that supports both RPN and algebraic style entry.
    • By default it uses RPN entry, but this can be changed to algebraic.
  • Stack based operations are always RPN-style.
  • Undo has the keybinding U, redo is D.
  • The top of the stack is referred to as 1:
  • Calc vectors are punctuated with [ and ] (e.g. [2 3]) Matrix values are represented as vectors within a vector. For example, [[1 0] [0 1]] is a square diagonal matrix.
  • Calc vector indexes are 1-offset.
  • Intervals
    • Inclusive intervals are represented as [𝑛..𝑚], where 𝑛 < 𝑚.
    • Exclusive intervals are represented as (𝑛..𝑚), where 𝑛 < 𝑚.
    • Any combination of lower and upper bounds set to be inclusive or exclusive is supported.
  • Complex numbers are entered as (𝑟, 𝑖), where 𝑟 is the real part and 𝑖 is the imaginary.
  • Radix numbers are entered as 𝑏#𝑛 where 𝑏 is the base value and 𝑛 is the number. For example entering 2#0101 will put 5 on the stack.
  • H:M:S values are default entered as ℎ@ 𝑚“ 𝑠’.
  • Org-mode active timestamps can be entered into Calc.
  • The top of the stack (1:) can be edited by pressing the ` key.
  • Entering a single quote (’) will prompt you for an algebraic entry.

Closing Thoughts

Back in the early 90’s I had the happy circumstance of owning a HP-28S. This calculator heavily influenced the feature set of Emacs Calc, but despite that Calc was frustrating to use because its primary interface (a prompt) was exactly the opposite of the interface provided by the HP-28S: an array of buttons to discover and press.

HP-28S photo

With keyboard-driven menus, I’d assert that Casual goes a long way towards fulfilling Calc’s promise to provide a HP-28 at your fingertips with the added bonus of integration into your digital life because it all lives in Emacs.

As Casual is new, I’m looking for early adopters! Your feedback is welcome as it will likely impact Casual’s evolution, particularly with regards to UI.

If you really like Casual, consider supporting its development and maintenance though buymeacoffee.

Thanks for reading and happy calculating!

-1:-- Announcing Casual - An opinionated porcelain for Emacs Calc (Post Charles Choi)--L0--C0--March 24, 2024 10:00 PM

tobli: A static Backloggery clone with Emacs org-mode and Python

Like many internet users, I enjoy obsessively tracking my activities, and there are a lot of websites around that make that easy. On the other hand, for various reasons, I don’t enjoy uploading my data to third-party sites. At the beginning of this year I used Emacs to put together an offline system that I think works well, based on org-capture, which prompts me for progress with reading a book/playing a game/etc. and then saves the information to a dedicated file for that day. The same files also contain my notes to self, DW posts, records of concerts I perform in, older sporadic diary/journal/blog entries going back to 1998, and (when I periodically export the data) scrobbles to last.fm.

My intention is to write a series of Python scripts that will read the data from these org files and output selected parts as nice HTML, potentially for uploading to my website if I want to share the data publicly. Backloggery is undergoing a drastic redesign/modernisation at the moment and I like the look of the new site, so decided to use this as a basis for building a new static site generator that shows some details about the games I’ve been playing based on my org files.

More )

Now that this is in working condition, I need to think about what I’d like to track in other domains, and how I want to do it … I’d like to have something Goodreads-esque for books but I’m not sure how feasible that is when a lot of that is based on server-side metadata and I’m also terrible at remembering to track my reading progress. I’ve also wanted to build something in a kind of bullet journal style for HTML/CSS for a long time, and could potentially do so for tracking household tasks, but that’s a very vague idea in my mind for the moment.



comment count unavailable comments
-1:-- A static Backloggery clone with Emacs org-mode and Python (Post)--L0--C0--March 24, 2024 09:56 PM

Irreal: Prot On Note Taking

Protesilaos Stavrou (Prot) has published an excerpt from a conversation he had with someone asking his advice on Emacs and note taking. The TL;DR is that his correspondent is a Vim user who is considering changing to Emacs for the advantages it offers for note taking. He is asking Prot how to proceed.

Prot’s advice is eminently sensible: Start slow. Use Org mode to begin with and add other packages only when you find you need them. Specifically, he advises using a single notes.org file with a heading for each note. Each note can have, or accumulate, additional subheadings as required. He also recommends setting the file to open with only the headings displayed. Folding is one of the strengths of Org mode and helps keep you from being overwhelmed by the available information.

His best advice—in my opinion—is to concentrate on the notes themselves rather than on the mechanics of taking them. He encourages note takers to spend time on writing purposeful and coherent prose so that when you read them later you will understand what they’re about and what your thinking was at the time. He also suggest linking new notes with previous notes where it makes sense. That, of course, is the basis of the Zettelkasten method, which Prot says he hasn’t studied and doesn’t necessarily recommend unless you, too, are genius workaholic like Niklas Luhmann. Better, he says, to start slow and find a system that works for you.

If you take notes, in Emacs or elsewhere, you should definitely take a look at Prot’s post. It has a lot of good advice without insisting that you use a particular piece of software.

-1:-- Prot On Note Taking (Post jcs)--L0--C0--March 24, 2024 04:44 PM

Alvaro Ramirez: Emacs: Toggling the continuation indicator

23 March 2024 Emacs: Toggling the continuation indicator

By default, Emacs typically displays curly arrows when wrapping lines. While likely a handy feature to some, I didn't really find much use for it. At the same time, I never looked into their removal until now.

Turns out, there's a continuation entry in fringe-indicator-alist variable that handles this. Removing this entry also removes the curly arrows.

(setq-default fringe-indicator-alist
              (delq (assq 'continuation fringe-indicator-alist) fringe-indicator-alist))

Alternatively, one could write a simple function to toggle displaying the continuation indicator.

curly.gif

(defun  toggle-continuation-fringe-indicator ()
  (interactive)
  (setq-default
   fringe-indicator-alist
   (if (assq 'continuation fringe-indicator-alist)
       (delq (assq 'continuation fringe-indicator-alist) fringe-indicator-alist)
     (cons '(continuation right-curly-arrow left-curly-arrow) fringe-indicator-alist))))

That's it for this post. A tiny tip. Perhaps there's a better way to handle it. If you know, I'd love to know too ( Mastodon / Twitter / Reddit / Email).

-1:-- Emacs: Toggling the continuation indicator (Post)--L0--C0--March 24, 2024 10:02 AM

Protesilaos Stavrou: Emacs: Denote version 2.3.0

Denote aims to be a simple-to-use, focused-in-scope, and effective note-taking and file-naming tool for Emacs.

Denote is based on the idea that files should follow a predictable and descriptive file-naming scheme. The file name must offer a clear indication of what the contents are about, without reference to any other metadata. Denote basically streamlines the creation of such files or file names while providing facilities to link between them (where those files are editable).

Denote’s file-naming scheme is not limited to “notes”. It can be used for all types of file, including those that are not editable in Emacs, such as videos. Naming files in a constistent way makes their filtering and retrieval considerably easier. Denote provides relevant facilities to rename files, regardless of file type.

Below are the release notes.


Version 2.3.0 on 2024-03-24

This release brings a host of user-facing refinements to an already stable base, as well as some impressive new features. There is a lot to cover, so take your time reading these notes.

Special thanks to Jean-Philippe Gagné Guay for the numerous refinements to parts of the code base. Some of these are not directly visible to users, but are critical regardless. In the interest of brevity, I will not be covering the most technical parts here. I mention Jean-Philippe’s contributions at the outset for this reason. Though the Git commit log is there for interested parties to study things further.

Check out the denote-explore package by Peter Prevos

This package provides several neat extensions that help you make better sense of your knowledge base, while keeping it in good order. The denote-explore package has commands to summarise the usage of keywords, visualise connections between notes, spot infrequently used keywords, and jump to previous historical entries.

Now on to Denote version 2.3.0!

Link to a heading inside a Denote Org file

Denote creates links to files by using their unique identifier. As Org provides the CUSTOM_ID property for per-heading identifiers, we now leverage this infrastructure to compose links that point to a file and then to a heading therein. This only works for Org, as no other plain text major mode has a concept of heading identifiers (and it is not Denote’s job to create such a feature).

I demonstrated the functionality in a video: https://protesilaos.com/codelog/2024-01-20-emacs-denote-link-org-headings/

Technically, the denote: link type has the same implementation details as Org’s standard file: and has always had this potential to jump to a section inside the given file.

The denote-org-store-link-to-heading user option

The user option denote-org-store-link-to-heading determines whether org-store-link links to the current Org heading (such links are merely “stored” and need to be inserted afterwards with the command org-insert-link). Note that the org-capture command uses the org-link internally if it has to store a link.

When its value is non-nil, org-store-link stores a link to the current Org heading inside the Denote Org file. If the heading does not have a CUSTOM_ID, it creates it and includes it in the heading’s PROPERTIES drawer. If a CUSTOM_ID exists, org-store-link use it as-is.

This makes the resulting link a combination of the denote: link type, pointing to the identifier of the current file, plus the value of the heading’s CUSTOM_ID, such as:

  • [[denote:20240118T060608][Some test]]
  • [[denote:20240118T060608::#h:eed0fb8e-4cc7-478f-acb6-f0aa1a8bffcd][Some test::Heading text]]

Both lead to the same Denote file, but the latter jumps to the heading with the given CUSTOM_ID. Notice that the link to the heading also has a different description, which includes the heading text.

The value of the CUSTOM_ID is determined by the Org user option org-id-method. The sample shown above uses the default UUID infrastructure.

If denote-org-store-link-to-heading is set to a nil value, the command org-store-link only stores links to the Denote file (using its identifier), but not to the given heading. This is what Denote was doing in all versions prior to 2.3.0.

Thanks to Kristoffer Balintona for discussing with me how org-capture interfaces with org-store-link. I updated the documentation accordingly. This was done in issue 267: https://github.com/protesilaos/denote/issues/267.

Insert link to an Org file with a further pointer to a heading

As part of the optional denote-org-extras.el extension that comes with the denote package, the command denote-org-extras-link-to-heading prompts for a link to an Org file and then asks for a heading therein, using minibuffer completion. Once the user provides input at the two prompts, the command inserts a link at point which has the following pattern: [[denote:IDENTIFIER::#ORG-HEADING-CUSTOM-ID]][Description::Heading text]].

Because only Org files can have links to individual headings, the command denote-org-extras-link-to-heading prompts only for Org files (i.e. files which include the .org extension). Remember that Denote works with many file types.

This feature is similar to the concept of the aforementioned user option denote-org-store-link-to-heading. It is, however, interactive and differs in the directionality of the action. With that user option, the command org-store-link will generate a CUSTOM_ID for the current heading (or capture the value of one as-is), giving the user the option to then call org-insert-link wherever they see fit. By contrast, the command denote-org-extras-link-to-heading prompts for a file, then a heading, and inserts the link at point.

Refinements galore to minibuffer prompts

All commands that affect file names conform with denote-prompts

The scope of the denote-prompts user option is broadened to make it more useful. In the past, this variable would only affect the behaviour of the denote command. For example, the user would make the command prompt for a subdirectory, then keywords, then a title. But all other commands were not following this setting, as they were hardcoding the prompts for title and keywords.

Take the denote-subdirectory command as an example. It would first prompt for a subdirectory to place the new note in, then for a title, and then for keywords. Whereas now, it prepends the subdirectory prompt to the list of denote-prompts. So if the user has configured their denote-prompts to, for example, ask for a signature and a file type, the denote-subdirectory will do just that with the addition of the subdirectory prompt.

Same idea for all commands that either create or modify file names, wherever conformity with denote-prompts makes sense. For example, the denote-rename-file will never ask for a subdirectory because our renaming policy is to always rename in place (to avoid mistakes—you can always move the file afterwards).

This also means that the denote-rename-file and its multi-file counterpart, denote-dired-rename-files, will only prompt for a signature if it is part of the denote-prompts. Whereas in the previous version this was unconditional, thus burdening users who do not need the SIGNATURE file name component (more about renaming further into the release notes).

Lots of Git commits went into this redesign, per my initiave in issue 247: https://github.com/protesilaos/denote/issues/247. Thanks to Vedang Manerikar for the changes to the convenience wrappers of the denote command (like denote-subdirectory), which were done in pull request 248: https://github.com/protesilaos/denote/pull/248.

Vedang has assigned copyright to the Free Software Foundation.

Also thanks to Max Brieiev for joining the technical discussion therein.

The renaming commands are more intuitive now, which addresses a discussion point raised by user babusri in issue 204: https://github.com/protesilaos/denote/issues/204.

A simple tweak for more informative minibuffer prompts

The text of each prompt now has all capital letters for the word referencing its scope of its application, like TITLE, KEYWORDS, SIGNATURE. The idea is to make it easier to quickly scan the text, especially while working through multiple prompts. For example, the prompt for a title now reads:

New file TITLE:

This paradigm is followed by all prompts. It is a small yet effective tweak to get a better sense of context.

The file prompt uses relative names once again

In previous versions of Denote, the minibuffer prompt to pick a file (such as a file to link to) would show relative file names: the name without the full file system path. The functionality depended on the built-in project.el library, which did not allow us to do everything we wanted with our prompts, such as to have a dedicated minibuffer history or to easily enable the workflow of commands like denote-open-or-create.

In the previous version, I made the decision to remove the project.el dependency and the concomitant presentation of relative names in order to add the functionality we want. I did it with the intention to find a better solution down the line. Et voilá! Relative file names are back. We now have all the functionality we need. Sorry if in the meantime you had to deal with those longer names! It was a necessary intermediate arrangement for the greater good.

For the technicalities, refer to the source code of the function denote-title-prompt.

Completion using previous inputs is now optional

All our minibuffer prompts have their dedicated history (you can persist histories with the built-in savehist-mode). They store previous values, giving the user easy access to their past input values. Some of our commands not only record a history, but also leverage it to provide completion. These commands are named in the variable denote-prompts-with-history-as-completion. As of this writing, they are:

  • denote-title-prompt
  • denote-signature-prompt
  • denote-files-matching-regexp-prompt

Users who do not want to use completion for those can set the new user option denote-history-completion-in-prompts to a nil value.

Renaming files got better all-round

One of the pillars of the denote package is its ability to rename any file to use the efficient Denote file-naming scheme (makes file names predictable and easy to retrieve even with rudimentary tools). To this end, we provide several commands that affect file names, beside the commands that create new files.

As noted above, the commands which rename files to follow the Denote file-naming scheme now conform with the user option denote-prompts, but there is more!

A broadened scope for the denote-rename-no-confirm option

The implementation of this user option is redone (i) to save the underlying buffer outright if the user does not want to provide their confirmation for a rename each time and (ii) to cover all relevant commands that perform a rename operation. The assumption is that the user who opts in to this feature is familiar with the Denote renaming modalities and knows they are reliable.

The default is still the same: Denote always asks for confirmation before renaming a file, showing the difference between the old and new names, as well as any changes to the file’s contents. In this light, buffers are not saved to give the user the chance to further inspect the changes (such as by running diff-buffer-with-file).

Commands that will now skip all confirmation prompts to rename the file and, where relevant, save the corresponding buffer outright:

  • denote-rename-file
  • denote-dired-rename-files
  • denote-dired-rename-marked-files-with-keywords
  • denote-rename-file-using-front-matter
  • denote-rename-add-keywords
  • denote-rename-remove-keywords
  • denote-rename-add-signature (new, more below)
  • denote-rename-remove-signature (new, more below)

Rename a file by adding or removing a SIGNATURE component

The SIGNATURE is an optional free-form field that is part of a Denote file name. A common use-case is to write sequence notes with it, though Denote does not enforce any particular convention (you may prefer to have it as a special kind of keyword for certain files that simply stands out more due to its placement).

[ Besides, the denote-sort-dired command lets you filter and sort files while putting them in a fully fledged Dired buffer, so manually sequencing notes via their signature may not be needed. ]

We now provide two commands to add or remove a signature from file names:

  • The denote-rename-add-signature prompts for a file and a signature. The default value for the file prompt is the file of the currently open buffer or the file-at-point in a Dired buffer. The signature is an ordinary string, defaulting to the selected file’s signature, if any.

  • The denote-rename-remove-signature uses the same file prompt as above. It performs its action only if the selected file has a signature. Otherwise, it does nothing.

Files that do not have a Denote file name are renamed accordingly. Though for such cases it is better to use denote-rename-file or denote-dired-rename-files as they are more general.

Use the denote-after-rename-file-hook for optional post-rename operations

All renaming commands run the denote-after-rename-file-hook after a successful operation. This is meant for users who want to do something specific after the renaming is done.

More optional features of the denote-org-extras.el

I already covered the denote-org-extras-link-to-heading, though the file denote-org-extras.el has some more optional goodies for those who work with Org files.

Create a note from the current Org subtree

In Org parlance, an entry with all its subheadings and other contents is a “subtree”. Denote can operate on the subtree to extract it from the current file and create a new file out of it. One such workflow is to collect thoughts in a single document and produce longer standalone notes out of them upon review.

The command denote-org-extras-extract-org-subtree (part of the optional denote-org-extras.el extension) is used for this purpose. It creates a new Denote note using the current Org subtree. In doing so, it removes the subtree from its current file and moves its contents into a new file.

The text of the subtree’s heading becomes the #+title of the new note. Everything else is inserted as-is.

Read the documentation string of denote-org-extras-extract-org-subtree or consult the manual for further details.

Convert denote: links to file: links

Sometimes the user needs to translate all denote: link types to their file: equivalent. This may be because some other tool does not recognise denote: links (or other custom links types—which are a standard feature of Org, by the way). The user thus needs to (i) either make a copy of their Denote note or edit the existing one, and (ii) convert all links to the generic file: link type that external/other programs understand.

The optional extension denote-org-extras.el contains two commands that are relevant for this use-case:

  • Convert denote: links to file: links: The command denote-org-extras-convert-links-to-file-type goes through the buffer to find all denote: links. It gets the identifier of the link and resolves it to the actual file system path. It then replaces the match so that the link is written with the file: type and then the file system path. The optional search terms and/or link description are preserved.

  • Convert file: links to denote: links: The command denote-org-extras-convert-links-to-denote-type behaves like the one above. The difference is that it finds the file system path and converts it into its identifier.

The Denote Org dynamic blocks are now in denote-org-extras.el

As part of this version, all our dynamic blocks are defined in the file denote-org-extras.el. The file which once contained these block definitions, denote-org-dblock.el, now only has aliases for the new function names and dipslays a warning about its deprecation.

There is no need to require the denote-org-extras feature because all of Denote’s Org dynamic blocks are autoloaded (meaning that they work as soon as they are used). For backward compatibility, all dynamic blocks retain their original names as an alias for the newer one.

We will not remove denote-org-dblock.el anytime soon to avoid any potential breakage with people’s existing notes. Though if you are new to this functionality, you better avoid the deprecated symbols.

Org dynamic block to only insert missing links

The denote-missing-links block is available with the command denote-org-extras-dblock-insert-missing-links. It is like the denote-links block (documented at length in the manual), except it only lists links to files that are not present in the current buffer. The parameters are otherwise the same:

#+BEGIN: denote-missing-links :regexp "YOUR REGEXP HERE" :sort-by-component nil :reverse-sort nil :id-only nil

#+END:

Remember to type C-c C-x C-u (org-dblock-update) with point on the #+BEGIN line to update the block.

This brings back a feature that was deprecated in version 2.2.0, but makes changes to it so that (i) it is more limited in scope and (ii) available as a standalone Org dynamic block.

Thanks to Stephen R. Kifer, Peter Prevos, and Elias Storms for the discussion which made it clear to me that users do have a need for such functionality. This was done in the now-defunct mailing list: https://lists.sr.ht/~protesilaos/denote/%3C1db2104e-70bd-47f9-a7ed-b8d4bb370a7f%40app.fastmail.com%3E.

Also thanks to Vedang Manerikar for fixing an edge case bug. This was done in pull request 260: https://github.com/protesilaos/denote/pull/260.

Org dynamic blocks are a powerful feature which also showcases how far we can go with Denote’s efficient file-naming scheme.

Quality-of-life improvements

Here I include other changes we made to existing functionality.

BREAKING User-defined sluggification of file name components

In the previous version, we introduced the user option denote-file-name-letter-casing. This was used to control the letter casing of file name components, but was ultimately not flexible enough for our purposes. We are thus retiring it and replacing it with the more powerful, but also more advanced, user option denote-file-name-slug-functions.

For existing users of the deprecated functionality, you can still preserve the input of a prompt verbatim with something like this:

(setq denote-file-name-slug-functions
      '((title . denote-sluggify-title)
        (keyword . identity)
        (signature . denote-sluggify-signature)))

The manual explains the details and shows ready-to-use code samples.

Remember that deviating from the default file-naming scheme of Denote will make things harder to use in the future, as files will have permutations that create uncertainty. The sluggification scheme and concomitant restrictions we impose by default are there for a very good reason: they are the distillation of years of experience. Here we give you what you wish, but bear in mind it may not be what you need. You have been warned.

Thanks to Jean-Philippe Gagné Guay for introducing this variable, among other tweaks, in pull request 217: https://github.com/protesilaos/denote/pull/217. Jean-Philippe has assigned copyright to the Free Software Foundation.

Option to automatically save the buffer of a new note

The user option denote-save-buffer-after-creation controls whether commands that create new notes save their buffer right away.

The default behaviour of commands such as denote (or related) is to not save the buffer they create. This gives the user the chance to review the text before writing it to a file. The user may choose to delete the unsaved buffer, thus not creating a new file on disk.

If denote-save-buffer-after-creation is set to a non-nil value, such buffers are saved automatically and so the file is written to disk.

The denote-menu-bar-mode and the placement of the Denote submenu

The command denote-menu-bar-mode toggles the inclusion of the submenu with the Denote entries in the Emacs menu bar (which is on display when menu-bar-mode is enabled).

This submenu is now shown after the Tools entry.

Thanks to Joseph Turner for sending me the relevant patches. Joseph has assigned copyright to the Free Software Foundation.

The C-c C-o works in markdown-mode for Denote links

In files whose major mode is markdown-mode, the default key binding C-c C-o (which calls the command markdown-follow-thing-at-point) correctly resolves denote: links. This method works in addition to the RET key, which is made available by the buttonization that we also provide. Interested users can refer to the function denote-link-markdown-follow for the implementation details.

Thanks to user pmenair for noting a case where this was breaking general Markdown linking functionality. This was done in issue 290: https://github.com/protesilaos/denote/issues/290.

More fine-grained control of Denote faces for dates/identifiers

We now define more faces for fine-grained control of the identifier in Dired. Thanks to mentalisttraceur for suggesting the idea in issue 276: https://github.com/protesilaos/denote/issues/276.

Before you ask, no, none of my themes will cover those faces because extra colouration is something only the user can decide if they want or not. In the above link I provide a sample with a screenshot (apart from the modus-themes, my ef-themes and standard-themes have similar functionality):

(defun my-modus-themes-denote-faces (&rest _)
  (modus-themes-with-colors
    (custom-set-faces
     `(denote-faces-year ((,c :foreground ,cyan)))
     `(denote-faces-month ((,c :foreground ,magenta-warmer)))
     `(denote-faces-day ((,c :foreground ,cyan)))
     `(denote-faces-time-delimiter ((,c :foreground ,fg-main)))
     `(denote-faces-hour ((,c :foreground ,magenta-warmer)))
     `(denote-faces-minute ((,c :foreground ,cyan)))
     `(denote-faces-second ((,c :foreground ,magenta-warmer))))))

(add-hook 'modus-themes-post-load-hook #'my-modus-themes-denote-faces)

New convenience command for users of the optional denote-journal-extras.el

The command denote-journal-extras-link-or-create-entry links to the journal entry for today or creates it in the background, if missing, and then links to it from the current file. If there are multiple journal entries for the same day, it prompts to select one among them and then links to it. When called with an optional prefix argument (such as C-u with default key bindings), the command prompts for a date and then performs the aforementioned. With a double prefix argument (C-u C-u), it also produces a link whose description includes just the file’s identifier.

Thanks to Alan Schmitt for contributing this command, based on previous discussions. It was done in pull request 243: https://github.com/protesilaos/denote/pull/243.

For developers or advanced users

These has new parameters or are new symbols altogether. Please read their respective doc string for the details.

  • Function denote-convert-file-name-keywords-to-crm.
  • Function denote-valid-date-p.
  • Function denote-parse-date.
  • Function denote-retrieve-title-or-filename.
  • Function denote-get-identifier.
  • Function denote-signature-prompt.
  • Function denote-file-prompt.
  • Function denote-keywords-prompt.
  • Function denote-title-prompt.
  • Function denote-rewrite-front-matter.
  • Function denote-rewrite-keywords.
  • Function denote-update-dired-buffers.
  • Function denote-format-string-for-org-front-matter.
  • Function denote-format-string-for-md-front-matter.
  • Variable denote-link-signature-format.
  • Function denote-link-description-with-signature-and-title.
  • Variable denote-link-description-function.

Miscellaneous

  • The denote-sort-dired function no longer errors out when there is no match for the given search terms. Thanks to Vedang Manerikar for the initial patch! This was done in the now-defunct mailing list: https://lists.sr.ht/~protesilaos/denote/patches/47625. Further changes by me.

  • The denote-keywords-sort function no longer tries to sort keywords that are not a list. Thanks to Ashton Wiersdorf for the patch. The change is small. As such, Ashton does not need to assign copyright to the Free Software Foundation.

  • Documented in the manual that custom convenience commands can be accessed by the denote-command-prompt. Thanks to Glenna D. for clarifying the language.

  • The denote-user-enforced-denote-directory is obsolete. Those who used it in their custom code can simply let bind the value of the variable denote-directory. Thanks to Jean-Philippe Gagné Guay for making the relevant changes (the Git history is not direct here and I cannot quickly find the pull request—the commit is a48a1da).

  • The denote-link-return-links no longer keeps buffers around. Thanks to Matteo Cavada for the patch. This was done in pull request 252: https://github.com/protesilaos/denote/pull/252. The change is small and so Matteo does not need to assign copyright to the Free Software Foundation.

  • Thanks to user jarofromel (recorded in Git as “random” author) for fixing a mismatched parenthesis in denote-parse-date. This was done in pull request 258: https://github.com/protesilaos/denote/pull/258.

  • The denote-rename-buffer-mode now works as expected with non-editable files, like PDFs. Thanks to Alan Schmitt for bringing this matter to my attention and then refining the implementation details in pull request 268: https://github.com/protesilaos/denote/pull/268.

  • All the Denote linking functions can be used from any file outside the denote-directory (links are still resolved to files inside the denote-directory). Thanks to Jean-Philippe Gagné Guay for the contribution in pull request 236: https://github.com/protesilaos/denote/pull/236.

  • We removed all glue code that integrated Denote with the built-in ffap, xref, and project libraries. We may reconsider how best to organise such features in the future. Thanks to Noboru Ota (nobiot), who originally contributed those extensions, for suggesting their removal from our code base. We did this by evaluating all use-cases. The discussion with Noboru happened in issue 264: https://github.com/protesilaos/denote/issues/264. Also thanks to Jean-Philippe Gagné Guay and Alan Schnmitt for checking the impact of this on how we generate backlinks. The latest iteration of this was done in pull request 294, by Jean-Philippe: https://github.com/protesilaos/denote/pull/294.

  • While renaming files, signatures no longer lose consecutive spaces. Thanks to Wesley Harvey for the contribution in pull request 207: https://github.com/protesilaos/denote/pull/207. The change is within the ~15 line limit and so Wesley does not need to assign copyright to the Free Software Foundation.

  • All of the above and lots more are documented at length in the manual. This is a big task in its own right (as are release notes, by the way), though it ensures we keep a high standard for the entire package and can communicate all our knowledge to the user.

No more SourceHut

Development continues on GitHub with GitLab as a mirror. I explained my reasons here: https://protesilaos.com/codelog/2024-01-27-sourcehut-no-more/.

This is a change that affects all my Emacs packages.

Forward guidance for Denote version 3.0.0

We will not any new features until mid-April or a bit later if necessary. This gives users enough time to report any potential issues with version 2.3.0. If there are any bugs, they will be fixed right away and new minor releases will be introduced (though without release notes).

Once we are done with this release cycle, we want to prepare for the next major version of Denote. The plan is to make the placement of file name components entirely customisable, among many other power user features. Though the defaults will remain intact.

For the immediate future, please prioritise bug reports/fixes. Then see you around for another round of hacking. The Denote code base is a pleasure to work with due to how composable everything is. I happy to make it even better for developers and users alike.

Git commits

Just an overview of what we did. Thanks again to everyone involved.

~/Git/Projects/denote $ git shortlog 2.2.0..2.3.0 --summary --numbered
   246	Protesilaos Stavrou
    46	Jean-Philippe Gagné Guay
     6	Vedang Manerikar
     3	Joseph Turner
     2	Alan Schmitt
     2	Max
     2	Peter Prevos
     1	Ashton Wiersdorf
     1	Glenna D.
     1	Matteo Cavada
     1	mattyonweb
     1	random
     1	wlharvey4

All contributions are valuable

I encourage you to provide feedback on any of the functionality of the Denote package. You do not need to be a developer or indeed an expert in Emacs. When you have an idea in mind on how you use Denote, or you think something could be done differently, please speak your mind. I do listen to feedback and am interested in further improving this package. Everybody is welcome!

-1:-- Emacs: Denote version 2.3.0 (Post)--L0--C0--March 24, 2024 12:00 AM

Irreal: Follow Mode

Marcin Borkowski has a nice post on Emacs’ follow mode. I’ve known about follow mode for a long time but never saw the point of it. That’s because I always thought of it as being used only on prose buffers. But as Borkowski points out, it’s really useful for code buffers where you want to see all or most of a large (or, perhaps more than one) function. By using follow mode you get two side-by-side windows that act as a single large window as far as scrolling is concerned.

When you scroll one window, the other scrolls too just as if they were a single window. This effectively doubles the height of your screen. But it gets better.

If you have enough horizontal real estate, you can split a window into not just two but three (or more) windows and link them together with follow mode so that they act as a single window tripling (or more) the height of you screen.

As I said at the top, I’ve never run across a case where doing this for a prose buffer was worth the effort of invoking follow mode but I agree with Borkowski that it can be useful for code buffers where surrounding context can be critical. Now that my eyes have been opened, perhaps I’ll make follow mode a part of my everyday toolkit.

If you aren’t already a follow mode user, take a look at Borkowski’s post. It’s reasonably short and you’ll learn some things that may be useful for your own workflow.

-1:-- Follow Mode (Post jcs)--L0--C0--March 23, 2024 03:08 PM

Mickey Petersen: Combobulate: Interactive Node Editing with Tree-Sitter

Recently, I talked about the challenges around improving Combobulate, my package that adds structured editing and movement to Emacs, so navigation is intuitive and works the way a human expects. Briefly, instead of relying on elaborate guesswork that breaks easily, there is now a little mini-language that captures how to pluck the right node from the concrete syntax tree provided by tree-sitter, a library that makes it a snap to parse and access said tree of any language for which there is a supported grammar.

Intuitive navigation is hard because if you ask a human developer to navigate up, down, left or right in their code, they can do so effortlessly, as there’s little ambiguity around, say, what constitutes the next “line of code”. Good luck asking a computer to make the right choice when it’s got a baker’s dozen of wildly different nodes to contend with. Hence the mini-language.

But I’ve talked about that already. You should read that article first, if you haven’t. So today I’d like to talk about a related issue: when you ask Combobulate to do something – say, cloning the node at point, and in effect duplicating a piece of code – and Combobulate can’t decide which node is the right one, because there’s more than one legitimate node to clone at point.

The problem space is similar to intuitive navigation, but with a twist. When we navigate we have a limited number of sensible directions we can move (up, down, left, right) and the bindings to go with it – C-M-n to move to the next sibling, C-M-u to move up to a parent, etc. – and when we invoke them we’d expect Emacs to just, you know, go to the right place. You don’t really want annoying minibuffer prompts, banners, billboards, smoke signals, crop circles or much of anything else to petition you and impede your train of thought. You just want to scoot away and not have Emacs stop you to ask for directions.

Fair enough. Combobulate goes to a lot of trouble to try and pick the right thing for you in most cases, whether you’re editing or moving. Sometimes, though, you must ask the user to make a choice.

And what’s the best way to do that?

Decisions, decisions…

The indentation 'carousel' in action. Tapping TAB at the start of a statement and Combobulate will calculate all possible indentation levels and let you choose the one you want interactively.

It’s a quagmire, really, because I’d had this issue multiple times where I really wanted some way of asking the user to make a choice. I had played around with the obvious contenders: some form of completing-read (Emacs’s way of doing minibuffer completion) and a hook to update the buffer as you browse through the candidates. It didn’t really work all that well as it assumes you have a completion system that lets you cruise through the candidates interactively. Not everyone uses that; default Emacs has you TAB endlessly to scratch out a completion like some sort of neolithic farm hand. So that method wouldn’t work well for people who use that type of completion, and it also has the annoying problem of feeling like it was the wrong tool for the job. What if someone uses Helm or any number of other completion systems?

I brainstormed some other methods like a Magit transient popup-like thing that you’d pick nodes from, but it also felt cumbersome and I couldn’t figure out how to get transient to do what I wanted it to do.

So I did what I always do: I got bored and went off in search of something else to do.

Right around that time I’d added a primitive version of “expand region”. It is a simple concept, really: given successive key presses, expand the region to incorporate larger and larger structural elements, starting from point. It’s a nifty way of picking things that ordinary Emacs methods struggle to do well at, though I never cared much for it pre-tree-sitter as I found it too imprecise. Lots of people love it, though, and I figured that it’d be super handy with tree-sitter, as it’s so granular, and I wanted to support a wide range of workflows. So I added a basic version in about 30 minutes, and off it went to Github.

Combobulate can expand the region with M-h. It highlights the item ahead of the current region, to give you an idea of where you are going. A customizable feature to let you select by number is also enabled here, and Combobulate will show you how much it has selected in the tree view as well.

A few days later I got a polite and totally obvious in hindsight request to make possible to shrink the region. It’s all too easy to overshoot your mark as you’re tapping away and, having done so, you’d have no recourse but to start from scratch. Argh. Terrible UX!

So I put on my thinking cap and started wondering how I could crowbar something into combobulate-mark-node-dwim (bound to M-h) so it can toggle the direction and, I… I realized I should just re-use another feature I’d added some months prior: a node / region-based indentation command in Python that lets you interactively browse and select the indentation level you want your node or region to have. (Believe it or not, but Emacs’s crummy Python mode can’t cycle all possible indentations of a region. It’s madness. The crux of the code to build it is right there, too. No TS required at all.)

The python indentation command (see above) acts like a carousel: keep moving in one or the other direction and you’ll wind up back where you started. That’s great for a wide range of things, particularly when you’re not sure how many finger taps away something is: is it two or three indentations I want? Tap a bit and find out, as your buffer updates automatically.

The indentation command was also a bit ham fisted in some ways, but it was a huge improvement over the ghastly python-mode indentation logic you had before which involved manual tedium and C-x TAB to indent by column.

I gambled that if I could file off the rough edges, and build some tooling around it into Combobulate, it could work well for a wide range of tasks. I sat down and spent a fair bit of time hammering out a system of rules and concepts that makes for, I think, a pleasant user experience. Carousel seems like as good a name for it as anything, so that’s what I’ll call it from now on.

If you’re already using Combobulate you’ll of course be familiar with the carousel concept already as it’s been in Combobulate for quite a while.

Reading a Key vs Minibuffer Prompting

Let me explain. There’s a philosophical view (and some who disagree with this) that certain user-facing actions in Emacs don’t require a full-blown minibuffer prompt and that reading a key is enough. Which one is the right choice is rarely too contentious, but in a few instances you could lean towards either method, and that’s usually where tempers flash.

Reading a key is mechanically simple: you call read-key (or one of its close relatives) and Emacs will patiently wait for a key; any key or key sequence, really. There is no minibuffer history to contend with; recursive minibuffers don’t apply, so you can’t switch out of the minibuffer window, either. You can display a prompt, but the main focus is really about user anticipation: once you know how the command works, and you know it expects a key, you just type it and get on with it. It is about as zero-effort as requesting user input could ever be. It’s used in a wide range of contexts in Emacs today, and it’s perfect when all you need from a user is a key.

The alternative is minibuffer prompting which is the all-singing, all-dancing prompt experience you know already.

I want Combobulate’s carousel to read a key: the reason is that it means I can capture the key you typed and, if I decide I have no use for it, I can put the key back on the unread-command-events. I could in theory still do that with the minibuffer prompt, but I still have to deal with the fact that it’s a complex system designed for non-trivial user interactions.

Reading a key is simpler, and that’s important because it meshes well with the idea that the carousel is there to offer a seamless transition in to, and out of it.

Seamless transition

If you want to expand the region with M-h, you probably want to follow that up with another key. Perhaps C-w to kill the region you just expanded. By reading the key I can separate TAB and S-TAB (to cycle to the next and previous choice in the carousel) from irrelevant keys the carousel does not care about. Then, by putting the key the carousel does not care about back into the event loop, you can have Emacs carry out what ever it was going to do as though the carousel wasn’t there at all.

End result? You can hit M-h and tap, tap, tap and press any other key that is not recognized by the carousel, and it’ll just execute the key as though you’d never had the carousel at all. No transition; no annoying in-your-face “are you really sure?” prompting; and no thinking required. It behaves the way an experienced Emacs user would expect it to work, and as if you never had the carousel active, even though, there it is, indicating in the echo area that it’s active and awaiting your input.

Maintaining your tempo

I mentioned that TAB and S-TAB cycle nodes. But if you tap, say, M-h for the first time, the carousel interface appears, and you can then repeatedly type M-h again which is the same as pressing TAB. That way you don’t have to move your fingers away from the triggering key and that helps preserve tempo. It makes for a smoother and easier user experience as you don’t have to context shift: oh I hit M-h and now I have to TAB to expand.

Because I look up the key in a boring, old keymap it’s easy for anyone to come along and modify it if they want the carousel to use other keys. It also means I can add additional keys that only apply to specific commands: Combobulate’s expand region functionality lets you cycle between the next and previous candidates with M-h and M-S-h, respectively.

The third reason why it’s useful to have the same key is that some operations are incredibly destructive and may leave your buffer in a “broken” state: the syntax is invalid, and tree-sitter may struggle to glue it back together. Having a cohesive view of the buffer at the beginning and operating on it from a clean slate is crucial.

Cohesion

Let’s say you’re deleting code as part of a refactoring operation you’re doing. As you go about doing it, you’re definitely going to leave your code in a broken state at some point. That’s all well and good: you’re a human, and you can fix it.

Tree-sitter has error-correction built into its parser, but just because it can mend, and partially recover from, broken syntax, does not mean it leaves the resulting tree in a state where an automated tool like Combobulate can make sensible decisions.

That’s particularly true of highly destructive operations like Combobulate’s splicing where you’re eliding text as you try to snip and glue two pieces of code back together: the code you’ve decided to keep, and the code around the code you’re keeping that the splice deleted. Think of HTML where you want to delete the outer tag but keep the things inside it - that’s one part of what you can do with splicing.

The problem is, maybe you want to splice two times, but the first splice irrevocably breaks your code in such a way that you’d never be able to make it to the second splice. And how do you delete the “outer” something of any old random piece of code, anyway? Take a look at the figure below to see what I mean.

M-<up> splices up, keeping the siblings, which in this case is one line of code, and then deletes successive parents until it's at the root of the buffer. Note that along the way it leaves the tree in a broken state.

Combobulate can splice nearly anything into something else. But that doesn’t mean it makes syntactic sense to do so. Maybe you do want it in a broken state; perhaps you want to tweak something to make it legal syntax again. Combobulate can’t read your mind so it has to calculate all possible paths.

If the splice up command – as it used to do – only went up one level, you could easily break the tree in such a way that you wouldn’t be able to splice again. A broken tree often begets an even more broken tree.

To work around that, the carousel virtualizes editing and computes everything on the fly, starting from the clean slate your buffer is (presumably!) in when you first initiate the command.

Virtualized Editing

C-c o c clones a node and, if there are ambiguities, Combobulate will let you cycle through all the choices and interactively preview the change it'll make to your buffer.

This was one of the harder things to build.

Tree-sitter generates your tree on-the-fly as you type. Every key press compels tree-sitter to rebuild all or parts of the tree. That is its main benefit, and a lot of work was put into making it fast enough for even the fastest typist to not experience any lag or latency.

Unfortunately, if you ask for a node and subsequently edit the buffer, that node is invalidated and marked outdated. If you try to do anything with it – anything at all, even asking it for its type or where it was in the buffer – it throws an error in your face.

So you can’t collect the node(s) you want at the start and then modify the buffer in-situ, like the indentation example above shows. The first tap would kill the old nodes (we changed the indentation!), and render that approach useless.

Combobulate constructs proxy nodes, and almost every part of Combobulate will accept these proxy nodes in lieu of the real deal. They are slimmed-down versions of the real things, but they reify the most important things we’d want to cling to: the point range in the buffer; the type; the text contained therein; etc.

So when you ask Combobulate to present a carousel it actually virtualizes the nodes before any sort of change can take place. It neatly skirts most issues and lets you write code that can in theory modify the buffer without worrying about your nodes expiring when you touch the buffer. Of course, modifying the buffer means you hold on to outdated information, and Combobulate is no oracle, so if you make substantial changes, the proxy nodes might be thoroughly useless.

Luckily, that’s usually not a problem because of Combobulate’s refactoring display system.

Part of the challenge around the proxy node thing is that most commands do small, localized editing operations: indent some code; expand an envelope; splice some code; clone a node. You get the idea.

As you tap through the carousel’s list of valid nodes, it should show you what would happen if the transformation you asked for is carried out on the currently selected node.

For this to work well, the carousel works in unison with Combobulate’s refactoring system. The latter sounds fancier than it really is: all it lets you do is describe simple transformations, and visual ones to aid users, to make to a buffer. Add in the carousel’s ability to use undo change groups to revert buffer changes between choices and you can have visible modifications made to your buffer that is properly restored as you cycle through the choices.

I think it’s important that, if you ask a command to make changes to your buffer, and if there’s more than one way to make that change, that you can preview all possible options. Combine it with the carousel interface and you’re afforded a fair amount of slop when it comes to point positioning.

And if you don’t like the change? You can hit C-g, as with most things in Emacs, to abort the command.

So that’s the carousel interface. I think it makes it much easier to visualize what’s going to happen in a non-committal way, and at the same time scroll through the valid node choices available to you.

-1:-- Combobulate: Interactive Node Editing with Tree-Sitter (Post)--L0--C0--March 23, 2024 02:04 PM

James Dyer: Eliminating Key Chords in Emacs under Linux with Sticky Keys

I spend many hours living in emacs and a large proportion of this time is using a laptop. Recently I have been thinking about my hands and how to protect them from any future pain or RSI.

Emacs out of the box can be a little, lets say tortuous awkward. For example I’ve always found the default keybindings for switching windows (`C-x o`) and saving buffers (`C-x C-s`) to be particularly uncomfortable, a feeling exacerbated by their frequent use. Also, key presses in Emacs typically involve key chording, this means that not only do you have to contort your hand into awkward positions, but you also have to press keys simultaneously thus reducing the possibility for natural hand movement during key activation.

The concept of C-x, especially for a Control key not mapped to the Caps Lock can really take its toll over a number of years; as of yet I am fortunate not to have yet experienced any hand pain but from what I’ve read, it’s not uncommon for discomfort to start to set in after many years of daily keyboard use, so I really want to take measures now.

In a previous post I had started to discuss this topic :

Sticky Keys on Sway using kmonad

I attempted to use the cross platform tool kmonad to apply Sticky Key functionality thus obviating the need for multiple key chord activation in emacs. For the most part a basic latched sticky concept works well and is easy to set up, however when using emacs I think a locking sticky modifier would be a much better fit, which is where a modifier will be locked on a double tap and then unlocked with a single tap.

For instance, if I could lock the Control key, navigating a buffer could be done with single key presses using ’n’, ‘p’, ‘b’, and ‘f’. Deleting multiple lines, a task I frequently do, would then require only individual presses of the ‘k’ key. To mark a selection, I could simply tap the space-bar and then use single navigation keys to expand the selection. A single tap of the Control key would then deactivate the lock.

Unfortunately kmonad doesn’t yet support this kind of sticky key locking functionality, however my attempts at finding a workaround led me to stumble into the murky arcane world of the x keyboard extension!

An XKB (X Keyboard Extension) file format is essentially a descriptive language that allows you to define the behaviour of the keyboard on linux. This includes the mapping of physical keys to symbols (characters or functions), modifying the action of keys depending on other keys that are held down (modifiers like Shift, Ctrl, Alt) and was primarily developed for the X11 windowing system.

I run a Wayland compositor called Sway as my daily driver and it just so happens that Wayland is compatible with the x keyboard extension standard!

Below are some instructions on how to set up Sway with Sticky keys to help eliminate the need for key chords in emacs. These instructions have been applied to the Sticky Keys section in the emacs wiki : https://www.emacswiki.org/emacs/StickyModifiers#h5o-8

Note that the following instructions can also work for X11 desktop environments as the x keyboard extension has been the de facto standard for many years on X11 and in fact I have successfully set up Sticky Keys in the same manner for the i3 window manager.


Setting up Sticky Keys through XKB

Firstly run the bash script below to create two sticky keyboard variant files:

#!/bin/bash
DST=$HOME/.config

# Reset keyboard layout (to your preferred language)
setxkbmap gb

# Apply sticky modifiers to a file
xkbcomp $DISPLAY -xkb - | \
sed 's|SetMods|LatchMods|g' > \
$DST/keymap_with_sticky_modifiers.xkb

# Reset keyboard layout (to your preferred language)
setxkbmap gb

# Apply locked modifiers to a file
xkbcomp $DISPLAY -xkb - | \
sed 's|SetMods|LatchMods|g' | \
sed 's|,clearLocks);|,clearLocks,latchToLock);|g' > \
$DST/keymap_with_locked_modifiers.xkb
  • keymap_with_sticky_modifiers.xkb - a latched sticky modifier setup in which a modifier is activated by a following key press.

  • keymap_with_locked_modifiers.xkb - a locking sticky modifier setup in which a modifier will be locked on a double tap and then unlocked with a single tap.

At this point you can activate Sticky Keys by loading the new xkb keymap in the Sway config as follows:

input "type:keyboard" {
    xkb_file $HOME/.config/keymap_with_locked_modifiers.xkb
}

Note that for X11 you can use the xkbcomp command as thus:

xkbcomp $HOME/.config/keymap_with_locked_modifiers.xkb $DISPLAY

Optionally modify the xkb file directly after generation to use the “Caps Lock” keyboard indicator LED for indicating a sticky modifier is active, like so :

indicator "Caps Lock" {
    !allowExplicit;
    whichModState= locked;
    modifiers= Control+Mod1+Shift;
};

This turns on the Caps Lock LED whenever a Control, Alt (Mod1) or Shift key is sticky locked giving a visual hint to any locked keys. Although useful on a laptop if you are using a full sized keyboard each modifier can be linked to its own indicator LED, for example:

indicator "Caps Lock" {
    !allowExplicit;
    whichModState= locked;
    modifiers= Control;
};
indicator "Num Lock" {
    !allowExplicit;
    whichModState= locked;
    modifiers= Mod1;
};
indicator "Scroll Lock" {
    whichModState= locked;
    modifiers= Shift;
};

As part of this keyboard redefinition it might also be useful to remap the Caps Lock key to the Control modifier. One way to achieve this in Sway is to modify the config file as follows:

input type:keyboard {
   xkb_file $HOME/.config/keymap_with_locked_modifiers.xkb
   xkb_options ctrl:nocaps
}

However, as we are redefining our keyboard layout we can directly modify the xkb configuration file in the following manner:

replace:

    key <CAPS> {         [       Caps_Lock ] };

with

    key <CAPS> {         [       Control_L ] };

replace:

    modifier_map Lock { <CAPS> };

with

    modifier_map Control { <CAPS> };

For bonus points, and to ensure that the right Alt modifier functions like a regular Alt thereby distributing the keys more evenly between the left and right hands, perform the following changes:

add:

    modifier_map Mod1 { <RALT> };

set the definition of key <RALT> to:

    key <RALT> {         [           Alt_L,          Meta_L ] };

With this setup, I can now effectively press single keys in succession to trigger functionalities in Emacs that would normally require key chording. Even pressing C-g has become naturally easier; I’m now pressing Ctrl, then not having to stretch my index finger across at the same time as my pinky is now released from the control key.

-1:-- Eliminating Key Chords in Emacs under Linux with Sticky Keys (Post James Dyer)--L0--C0--March 22, 2024 09:39 PM

Alvaro Ramirez: The Org bundle

22 March 2024 The Org bundle

Plain Org / plainorg.com

My more generic solution to access org files on the go and away from Emacs.

plainorg-600x0w.png

Flat Habits / flathabits.com

My take on frictionless habit tracking truly respecting user privacy and their time (absolutely no distractions).

flathabits-600x0w.png

*scratch* / App Store

Sure, we have tons of note-taking apps but most require more steps than desirable to write something down ASAP. Launch the app and you're good to write. No new note creation, bring keyboard up, etc.

scratch-600x0w.webp

Common denominator

In addition to being offline-first, no cloud, no login, no ads, no tracking, no social… each app targets a specific purpose, sharing an important common denominator: they all use org markup as the underlying storage.

The Org bundle / App Store

While you can still get each of my apps individually, you now have the option to get them all as a single bundle: The Org bundle.

bundle.png

Journelly joining the bundle soon…

Continuing on the org storage theme, I got another app in the works. Also joining The Org bundle, maintaining its privacy-first approach: offline, no cloud, no login, no ads, no tracking, no social… this time in the journaling space.

Journelly is currently in beta, want to join?

journelly-demo.gif

-1:-- The Org bundle (Post)--L0--C0--March 22, 2024 08:05 PM

Irreal: 🥩 Red Meat Friday: Changing Emacs Defaults

I’ve written before about messing around with Emacs defaults. The argument from those favoring changes is that Emacs’ conventions are not congruent with modern day standards and thus confuse n00bs. This, is turn, inhibits Emacs uptake by new users.

The other side of the argument is that long term users should not have their workflows and muscle memory disrupted on the nebulous basis that changing the default may result in new users. After all, most of those long term users learned Emacs long after today’s CUA standards were in place and can be forgiven for expecting others to do the same.

It’s not that the needs and expectations of new users should be ignored. It’s just that the way to accommodate them is not as easy as some claim. Some of the problems are discussed in the above post and some suggestions for addressing them are in the comments.

In the mean time, here’s a small rant from the Emacs Elements guy. He points out, correctly, that if you don’t like the Emacs defaults, you’re free to change them—indeed, that’s the whole point of Emacs configurability—just don’t insist that everybody else should change them too to suit your preferences. Sadly, the majority of comments to the rant are of the type that insist all those troglodytes from the PDP-10 era need to get on board with the new VS Code compliant conventions. The fact that those troglodytes can edit rings around them and their VS Code seems to escape them.

On the plus side, some of the comments suggest ways of addressing the problem. While I have limited sympathy for the whining complaining that “the defaults aren’t what I’m used to”, I’m glad to see other folks embracing the Tao of Emacs and suggesting ways of adapting Emacs to the problem.

-1:-- 🥩 Red Meat Friday: Changing Emacs Defaults (Post jcs)--L0--C0--March 22, 2024 05:40 PM

Protesilaos Stavrou: Re: Advice regarding note-taking in Emacs

What follows is an excerpt from a private exchange that I am sharing with the permission of my correspondent. I am not disclosing their identity. The topic is about how to start using Emacs to take notes. Part of what I cover involves tooling, while I also comment on matters of method.


After my last failed attempt to switching to emacs, I’ve decided to once again venture towards switching, with my primary motivation at the time being a cohesive framework for note-taking and implementing a second brain.

I was wondering, what your recommendations for the same would be. All I know about emacs-specific note-taking framework is org-mode, in addition to your denote package.

There are many other packages as well. It does not really matter. Without a clear workflow, no tool will magically fix your problems. Put differently, the first brain dictates the utility of the second one; the latter is no substitute for it.

I will not tell you about Denote. Just pick standard Org by creating a single file called notes.org. In that file, create new notes as top-level headings. Each note can have subheadings. Org handles those nicely as it can “fold” (show or hide) the contents of a heading and its subheadings.

To navigate this notes.org, set the file-specific option of showing only the headings by default when first opening the file. Add this to the top of your file:

#+startup: content

You can navigate the file using search methods, but do not worry about that at this initial stage. Your goal is to get in the flow of writing notes, which should eventually be of a high quality.

For this early stage, focus on producing content. Try to elucidate your thoughts and provide sufficient contextual information. Remember that writing notes is not the same as bookmarking. It has to be more involved, so that you have to put in some effort to understand/explain the subject matter.

Write with intent and spend the quality time this activity requires. The “I need to do it quickly” mindset contradicts the concept of a second brain, indeed of mindfulness altogether. If you are in a hurry, then you are not focused on retaining knowledge anyway. Whatever you write is, at best, a sloppy entry on a matter that needs to be re-learnt. If you never have time, then work on that first and then revisit the project of taking notes.

Again, it is the first brain that sets you up for success. The inverse narrative is a marketing gimmick. Get in the habit of writing regularly. Express your thoughts in a way that another person can also understand. This is important for your future self who, once withdrawn from the specific factors that informed the given note, is also a stranger to the topic. After a while, we forget what we were thinking or trying to achieve, so the notes we store must be done in such a way as to be helpful to us long-term.

Once you are doing well with this simple Org setup, you are ready to consider more packages that will complement or otherwise enhance your workflow (or just stick with what you have, which is fine). I will not tell you what those packages are because you will know what your needs are once you go through this process. You will figure out what works for you and what does not. You will thus have a better sense of what is missing.

Also, do you have any specific philosophy in mind when taking notes? I frankly find your method of disposing knowledge on your website and videos, quite methodological, of which I would like to implement in my own note-taking and knowledge sharing.

I recently came across this “Zettelkasten Method” of note-taking. Do you have any thoughts about it?

I have heard about Zettelkasten but have not studied it. I do not follow it, although I recognise there is an overlap with some concepts covered by my Denote package (but Denote’s cornerstone is the file-naming scheme, which is a computer-centric arrangement to optimise search even when using rudimentary tools—having separate ideas/format in individual files is how I have always known personal computing). What makes me sceptical about the hype surrounding the Zettelkasten method is that its proponents quote the rich corpus of its author’s work as proof that this method will suffice for you. But what if Luhmann was a genius and a workaholic? How much weight do those character traits carry? Will Zettelkasten grant me those too? Of course not!

Maybe Zettelkasten is good for you, but I encourage you to keep things simple at this early stage, as I outlined above. Using that method means that you require some extra tooling, which will put you down the path of figuring out which packages you should add, how to configure them, and so on. This is a major distraction for you at this point. Your goal is to write notes, not procrastinate using productivity as a pretext. Do start writing and then you can revisit this topic once you are prepared to improve on what you already have.

Even if you do not need some extra tools and you go with the original method of Luhmann (using paper slips), you still need to think like Luhnmann. Do you? I think not. You can train to do so, though this too is a lateral task to the one you are asking me about.

My approach to this and other matters of workflow is to listen to what people have to say but to ultimately proceed through experimentation. Start small so that your experiments will not be too costly should they fail. Expand or adjust your workflow as the need arises and the solution becomes obvious. Whatever method emerges from this organic process will be aligned with how your brain works, what sort of person you are, how you intuit about things, and so on.

P.S. In regards to switching to emacs, I’m in a similar boat currently as you were about 5 years ago i.e. my current text editor of choice is vim. I was wondering if had any tips to learn and not lose motivation in switching to emacs for a person such as myself. Would you recommend I learn the stock emacs keybindings for text-editing? (I tried those before it felt very unnatural compared to (what I personally feel) superior vim keybindings.

I used to think Vim keys are the superior paradigm. But I kept an open mind and decided to give Emacs keys a fair chance. After a while, everything felt natural and I am happy with what I have. I write a lot of prose, so the modal paradigm does not do much for me anyway. While I was using Vim, I felt it made me faster, though I realised it was a placebo effect. How fast I edit text is not the deciding factor in what I do. But I digress.

To be successful with Emacs, you want to take it slow. Add a package or snippet of code when you have a clear need for it and at least have a basic understanding of what it is doing. What guarantees the infamous “Emacs bankruptcy” is the bad habit of copy-pasting code without an overarching objective.

Take it slow and read the official documentation. It is the best source on the matter. Then you can complement that study with other sources, such as blog posts or video tutorials. As you piece together your own configuration, you gain insight into your choices and requirements, such that you have a better sense of the system you are piecing together.

Notice here the parallel with what I wrote above about discovering the particularities of your case organically. It is easy for me to tell you to just use this or that. It also is presumptuous, as one size does not fit all. Start with the basics and proceed through trial and error: it is just a computing environment and it is okay to play around as you explore it. You will eventually discover what others in the community consider best practices or modern standards, though knowing the “why” they are considered such will do you good as it will empower you to judge matters from a position of knowledge.

Good luck!

-1:-- Re: Advice regarding note-taking in Emacs (Post)--L0--C0--March 22, 2024 12:00 AM

Irreal: PSA: Mu 1.12.1

This is just a short PSA. As most of you know, I’m a huge fan of Mu/Mu4e and use it as my main mail client. Lately, I’ve wimped out and rather than building it from source, I’ve been installing it from Homebrew. That’s mainly because building it requires Cmake and some other tools that I have no other use for.

I’d been using the 1.10 version, which, sadly, had a few problems. Mostly it lost the marks of previously read emails when a new one came in before the mail listing buffer had been dismissed. Happily, that problem has gone away with version 1.12 and it now seems perfect. You can also choose whether emails should display as straight text or if links and images should be active. The default Homebrew install is now 1.12.1 so it’s easy to upgrade.

If you’re still running an older version, I strongly recommend that you upgrade. I’m sure you’ll be as pleased as I am.

-1:-- PSA: Mu 1.12.1 (Post jcs)--L0--C0--March 21, 2024 03:22 PM

Maryanne Wachter: Building OSS Map Apps With Observable Framework

import bldgMap from './media/bldg-map.gif'; ## Motivation I've been a pretty heavy user of [Observable](https://observablehq.com) since 2021. It was my initial entry point to learning and working with D3, and I still use the notebook interface heavily for prototyping and exploring public datasets. When I saw the announcement for [Observable Framework](https://observablehq.com/blog/observable-2-0), it seemed like a no brainer that would support a [lot](https://mclare.blog/visuals/ec3epds/) [of](https://mclare.dev/soft-stories/) [the](https://observablehq.com/@m-clare/sustainable-design-in-communities) [visualizations](https://bridge.watch/) I like to build (and help me do so quickly). I've been working with open source mapping lately, and I saw in the initial documentation that there was [Mapbox support](https://observablehq.com/framework/lib/mapbox-gl). I wanted to see how difficult it would be to build maps with Observable Framework using open source libraries (my preferred stack of [Maplibre](https://maplibre.org) and [Protomaps](https://protomaps.com/)), so I prototyped this [interactive map of potential energy efficient retrofits of Boston buildings](https://m-clare.observablehq.cloud/boston-building-retrofits/) over the past few weekends. ![Boston Building Retrofits](/media/boston-retrofits-example.png) ## Dataset Selection The city of [Boston](https://data.boston.gov/) has a number of great datasets about its buildings, including an aggregated view around its efforts in sustainability and energy efficiency, called the [Boston Building Inventory](https://data.boston.gov/dataset/boston-buildings-inventory). From the docs: > "This dataset pulls from many different data sources to identify individual building characteristics of all buildings in Boston. It also identifies high-potential retrofit options to reduce carbon emissions in multifamily buildings, using the best available data and assumptions from building experts." There are a few different components included with the dataset, including explanations of the potential retrofits, as well as a presentation providing further motivation for the project. I first took a look at the raw data using [Visidata](https://www.visidata.org/) before deciding to focus on mapping a subset of the data, only looking at the envelope retrofit strategies (which I have more familiarity with as a structural engineer). ## Starting an Observable Framework Project I started my Observable Framework project with default settings to launch the initial template application (I chose `npm` for my package management setup because I have very few opinions on the js package management hellscape). I did appreciate that Observable allows you to opt in on telemetry data given that the Framework library is open source. ## Data Loaders The key differentiator for me with Observable Framework vs. other static site generators is the [data loader](https://observablehq.com/framework/loaders) functionality. Most of the datasets that I work with are around building data, which is updated at most once a day (and in some cases, like [Bridge.watch](https://bridge.watch/?plot_type=percent_poor), only once a year!) The language-agnostic implementation means that I can easily build loaders in whichever programming language I need to (maybe even Rust 😈...), and then make a cron job on my server that would run the loader and deploy the static site once a day. For this visualization, I could just use NodeJS for my loader, but quite often I'll do data cleaning in Visidata and export the list of commands used within Visidata to generate a Python/Pandas script. For the Boston Buildings Inventory, I needed to stitch two data sets together. The retrofit potential data only had parcel numbers, not actual latitudes and longitudes to impose markers on buildings on the map. I utilized the [parcel map from 2022](https://data.boston.gov/dataset/parcels-2022/resource/023f4a54-9a73-492a-8489-3ae0f99dbf92) for Boston with its shapefiles (unfortunately 2023 is not available yet from ARCGis). I wanted to keep the map a bit lighter, so I used the [polylabel](https://www.npmjs.com/package/polylabel?activeTab=code) library to find a "center" of a building that was within its boundaries, rather than centroid which could exist outside. I wrote a [loader script](https://github.com/m-clare/boston-map/blob/main/docs/data/buildings_data.csv.js) that zipped the parcel lat/long data together with the subset of fields I wanted from the buildings data set, and output the new dataset as a zip file. Also, don't be like me and [read the docs](https://observablehq.com/framework/routing) that say that the loader file name needs to match the `FileAttachment` name you put into your components or Markdown doc. ## The Markdown File Within `index.md`, I ran afoul of some async commands and had to split my javascript across a few cells. I'd like to go back and figure out exactly what triggered the bad behavior, or if it's more of a hot reloading symptom/issue because of the map. The aforementioned issue with the loader file name did introduce some questions about what you'd do if your data loader needed to have multiple output files. As far as I could tell, you would need to have a single output zipped archive and then unpack each individual file with `${unzipped_object}.file(${filename}).csv()`. Because I had zipped the loader data as a csv to make it lighter than JSON or GeoJSON, I needed to handle transformation to GeoJSON client side and load it as a data source within my map. ```js const geoBuildingData = { type: "FeatureCollection", name: "geoBuildings", crs: { type: "name", properties: { name: "urn:ogc:def:crs:OGC:1.3:CRS84" } }, features: [], }; const bldData = await buildingData.file("buildings_data.csv").csv(); const buildingTypologies = new Set( bldData.map((feature) => feature.building_typology) ); bldData.map((feature) => { const { point: rawPoint, ...rest } = feature; const point = rawPoint.split(",").map((value) => parseFloat(value)); geoBuildingData.features.push({ type: "Feature", properties: { ...rest }, geometry: { type: "Point", coordinates: point }, }); }); ... map.on("load", () => { map.addSource("bld-data", { type: "geojson", data: geoBuildingData, }); }) ``` I also got *real* sloppy with some CSS to try and recreate the look and feel of some of the [other](https://mclare.dev/nyc-building-complaints/) [maps](https://mclare.dev/soft-stories/) I've built. It was a bit painful to figure out what to target and how to override the styles. I know that when you create an Observable Framework project, you can select a style, but I do wish it was a bit easier to customize after that first step. For this prototype I used project hosting with Observable because I wanted to see what the deploy process would look like. It was easy! The only hiccup I ran into was that I couldn't figure out how to set access to the map as public once I had deployed (it defaulted to private), but it turns out a newer version of Observable Framework allows you to set the project up as public when you're initially configuring the project from the CLI. The Observable team is also responsive, so I had an answer on the [forum](https://talk.observablehq.com/t/no-option-for-public-access-provided-upon-deploy/8950/2) less than two hours after I asked! <div style={{ maxWidth: "600px" }}> <img src={bldgMap} alt="Boston Building Inventory Map" /> </div> <p style={{ textAlign: "center" }}> [Boston Building Retrofits](https://m-clare.observablehq.cloud/boston-building-retrofits/) </p> <p style={{ textAlign: "center" }}> [Github](https://github.com/m-clare/boston-map) </p> ## Things This Prototype is Not - optimized for performance - in a world where I have an infinite amount of time for side projects, I'd build my loader to directly make map tiles based on the building data layer. - the cleanest code - I could probably separate out all the map data into a component. Maybe I will do that later (I took a stab at it and had some issues with the Runtime library) - optimized for mobile - I didn't extend zoom levels far enough for the basemap (only to level 15) to be able to use fat fingers to select individual markers easily. - bug-free - The heads up display seems to duplicate or come in the wrong order on some devices/browsers so it becomes difficult to read the text (#worksonmymachine) ## Takeaways I could definitely see myself moving away from [Gatsby](https://www.gatsbyjs.com/) (which is what I currently use for this blog) and rewriting using Observable Framework. I'm currently working off of a custom fork of my [Gatsby theme](https://github.com/alxshelepenok/gatsby-starter-lumen) to support mdx specifically because I want to be able to load in React components and visualizations both from Observable notebooks, as well as local components. I break things a lot with Gatsby, and more often than not, I get some new whack-a-mole issue with GraphQL when I `gatsby develop` a new blog post after a few months. I think Observable Framework might make this easier for me to do, but I'd have to give some thought as to how to change over the existing work which utilizes React to manage some state for the interactive visuals (but even in some of those cases, I was using Observable's cell takeout so it seems like that shouldn't be too onerous). The original plan for this project was to also include marker filters via a select input based on the building type (which is currently used to color code markers). However, I couldn't figure out how to pass an event handler to the [Observable Input](https://observablehq.com/framework/inputs/select) that would update the map. I know how to do this in React using `useRef` hooks for libraries that require manipulating the canvas (ThreeJs, Maplibre-GL-JS), but outside of working directly with [html inputs](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/Input), I didn't know how to accomplish this with Observable components (wrestling with styling raw HTML inputs is not how I want to spend my free time). I created a div in the markdown file and then used that as the container for the map, but I couldn't access that reference in a different `js` cell. I also had to host my map font files elsewhere, rather than in the same location as the `pmtiles` file. ```js const features = display(document.createElement("div")); features.id = "features"; features.style = "z-index: 100"; const mapContainer = display(document.getElementById("mapContainer")); mapContainer.appendChild(features); const div = display(document.getElementById("mapContainer")); const windowHeight = window.innerHeight; const windowWidth = window.innerWidth; div.style = `position: relative; height: ${windowHeight - 50}px; width: 100%`; const map = new maplibregl.Map({ container: div, zoom: 12, maxZoom: 14, minZoom: 10, maxBounds: [ [-71.191247, 42.227911], [-70.648072, 42.450118], ], center: [-71.08936258403622, 42.3181973483706], style: { version: 8, sources: { openmaptiles: { type: "vector", tiles: ["pmtiles://" + mapFile.source.getKey() + "/{z}/{x}/{y}"], }, }, layers: mapStyle.layers, glyphs: "https://m-clare.github.io/map-glyphs/fonts/{fontstack}/{range}.pbf", }, }); ``` Outside of these issues, this was a pretty fun way to test some of the edges of Observable Framework. Next steps will be testing out self-hosting with Digital Ocean, trying to refactor the map into its own reuseable component, and (maybe) mucking around with inputs again. ## Acknowledgements Thanks to Brandon Liu, the creator of [Protomaps](https://protomaps.com/) for quickly troubleshooting the initial issues with loading `pmtiles`, and creating a [sample map example](https://bdon.github.io/observable-framework-maps/example-map#interaction%3A-maplibre-gl-js) which I used to develop this project.
-1:-- Building OSS Map Apps With Observable Framework (Post)--L0--C0--March 21, 2024 12:00 AM

Irreal: Prompting With The Interactive Command

Most Emacs users who are at all familiar with Elisp know that you have to specify the interactive command for functions if you want them to be interactive—that is, if you want them to be callable with Meta+x. Most of those people also know that you can use the interactive command to prompt a user for function parameters.

What is less well known is that you can also pass a list of parameters for the function in the interactive command. By itself, that’s not too useful, of course, but rather than specifying a list, you can specify a function that returns a list. The function can do anything as long as it returns a list. Arialdo Martini has a post that illustrates this nicely.

He looks at the problem of surrounding a region with a pair of delimiters. The question is how to get the two delimiters. The most obvious—but also most difficult—solution is to simply add code to the body of the function to prompt for the delimiters. It’s far easier to use the interactive command to get them and this also has the advantage of allowing the function to be called from Elisp without any prompting.

But what happens if you want to limit the choices to a predetermined set? That’s exactly what completing-read is for. Sadly, there’s no interactive parameter for this but you can write a function that uses completing-read and returns the delimiters as a list.

Martini’s post uses this idea to implement a set of functions that surrounds a region with any of a predetermined set of delimiters. As Martini says, there’s already plenty of functions and packages that do this. The point of his post is to explore how you can use the interactive command to pass parameters to a function. It’s a good post and worth a few minutes of your time to read.

-1:-- Prompting With The Interactive Command (Post jcs)--L0--C0--March 20, 2024 03:19 PM

Zachary Kanfer: til: Ruby's hashmap syntax

I was writing some Ruby for the first time, and I made a hashmap with some data in it.

    map1 = {"foo": 1, "bar": 2}      

I was then surprised to find out that I couldn't get any items out of it.

    > map1 = {"foo": 1, "bar": 2}> map1["foo"]=> nil      

Heck, it didn't even seem to have the key stored in it!

    > map1 = {"foo": 1, "bar": 2}> map1.has_key?("foo")=> false      

So I dug a bit into the documentation. The problem is there are two syntaxes for creating hashmap literals, a json-like syntax, and what Ruby calls the "rocket" syntax.

    map1 = {"foo": 1, "bar": 2}     # the json-like syntaxmap2 = {"foo" => 1, "bar" => 2} # "rocket" syntax      

The json-like syntax is newer, and is the cause of my confusion. It silently converts each string key to a symbol.

    > map1 = {"foo": 1, "bar": 2}> map1.keys=> [:foo, :bar]      

The rocket syntax doesn't!

    > map2 = {"foo" => 1, "bar" => 2}> map2.keys=> ["foo", "bar"]      

So, for any maps created by the json-like syntax, they must be accessed with symbols.

    > map1 = {"foo": 1, "bar": 2}> map1[:foo]=> 1      

Or, just create them with the rocket syntax, and you can access them with strings. Like you expect.

I don't understand why this is the case. This conversion is surprising, especially since the json-like syntax is a newer addition to Ruby.

-1:-- til: Ruby's hashmap syntax (Post)--L0--C0--March 20, 2024 05:00 AM

Arialdo Martini: Emacs: Playing Hansel and Gretel with Marks, Registers and Bookmarks

A cool trick not everybody knows with Visual Studio and JetBrains’ IDEs is that it is possible to navigate back to previous positions with Ctrl -.

A similar trick also works with Bash:

cd -

and with Git:

git checkout -

Neat, right?
Of course, there are way more powerful tools built around this idea (such as z, autojump and zoxide). And — did you doubt? — similar powerful features in Emacs too. Let’s explore them.

Leaving a trail of breadcrumbs

So you want to leave a track while you move around buffers and you want to retrace your steps, right?
As far as I know, Emacs offers 3 tools:

Tool Main characteristics When it is the best fit
mark ring - buffer local and global
- unnamed
- both manual and automated
- volatile
- It’s related to the region
- Quick, impromptu navigation
registers - global
- short-named
- manual
- not persistent by default
- Within a working session
- Can store more than positions
bookmarks - named
- manual
- persisted
- The longterm cornerstones for organizing your work environment
- Can store more than positions

In this series, we will explorer them all!

Table of Contents

References

-1:-- Emacs: Playing Hansel and Gretel with Marks, Registers and Bookmarks (Post Arialdo Martini)--L0--C0--March 20, 2024 12:00 AM

Arialdo Martini: Emacs: Mark Ring

Rings — fixed sized variables acting as circular buffers — are a beautiful idea: one day I will eventually write something about how undoing changes is handled in Emacs with the undo-ring. I find outrageous that other editors have not followed the same idea.

The mark ring itself is an amazingly simple idea: it is just a variable storing buffer positions. Around it, there are of course functions for pushing and pulling data, functions for browsing the stored positions, packages for making all of this even more convenient and the like. But the foundation is just that: a variable.
While I’m writing this post, the content of my mark-ring is:

M-x describe-variable <RET> mark-ring <RET>

(#<marker at 2080 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 2080 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 2074 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 2074 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 2074 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1696 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1195 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1516 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1447 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1392 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1379 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1056 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1042 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1206 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1237 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1224 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 1056 in 2024-03-17-emacs-navigate-back.md>
 #<marker at 904 in 2024-03-17-emacs-navigate-back.md>)

As you see, it is just a list of objects referencing a buffer and a position.
The mark ring is used as a storage for the historical values of marks. Marks, on their side, are positions that you intentionally asked Emacs to keep a memory of — or that Emacs, before some operations, decided to mark on its initiative.
There is a notion of the current mark, which you can inspect evaluating (mark). Whenever the mark changes, its old value is pushed in the mark ring.
If your head already spins, don’t despair. It’s easier than it seems. Think of Git.

Like Git

I find the relation between the cursor, the mark and the mark ring similar to what happens with Git with the worktree, the index and the repository. This is wonderfully represented in this interactive Git Cheatsheet. For Emacs we have something similar (and way simpler):

Change me

Just Going Back

So you have variable to store a history of marked position. The basic idea is:

  • You store a position for future use.
  • You move around.
  • When you want to get back on your tracks, you pull the position out of the history.

Here is the basic usage:

  • Whenever you want to leave a breadcrumb, use C-SPC (set-mark-command). Hit it twice: the first time it will start selecting text (in Emacs lingo: it will activate the mark). Here we don’t want to select anything: we just want to write a position down in the history. C-SPC (set-mark-command) updates the current mark and pushes the previous a value in the mark ring, in fact creating a history of positions.
  • If C-SPC saves a position, it should not come as a surprise that prefixing it with C-u reverses the behavior. That’s a common pattern in Emacs: in general, C-u (universal-argument) is a way for starting inputing a numeric argument to a function; for many commands, as for C-SPC, it serves as a flag to deviate from the default behaviour.
  • Indeed: whenever you want to move back to the previous position, hit C-u C-SPC. Your cursor will be moved where the mark is, and the previous position will be popped out from the mark ring. In other words, with C-u C-SPC you will be consuming the position history.

Follow the diagram. It’s really easier done than said.

C-u C-SPC acts like VisualStudio’s and IntelliJ’s Ctrl - with a big difference: with VS and Idea, it is not that clear when new values are pushed in the position history; in Emacs you are in full control.
But this is, anyway, just the tip of the icebearg.

Oh, My Consult!

The position history is just a variable. You could expect that someone would eventually write packages to manipulate that variable in fancy ways. Amongst others, Daniel Mendler did that, with the amazing consult.el library. To me, the most convenient way to operate with the mark ring is, hands-down, consult.el. Install it with:

(use-package consult
  :ensure t)

It will provide you with the command consult-mark which you can use to browse all the positions in the mark rink — not only the last one in a LIFO fashion — and to have a real-time preview of the content you would jump to.

Do this experiment:

  • Open a large file.
  • Move around, and from time to time hit C-SPC C-SPC wherever you want to leave a breadcrumb.
  • When you have enough of this, run M-x consult-mark.
  • Try to move up and down, back and forth the history.
  • Search for a position in the mark ring by content.

Never. Again. Without.

Multi Cursors

Once you have some positions stored in the mark ring, there are other interesting things you can do.

  • Navigate around in a buffer and push some marks here and there (C-SPC C-SPC).
  • Run M-x mc/mark pop. This will pop the last mark from the mark ring and add a second cursor there.
  • Run M-x mc/mark pop again. You will get a 3rd cursor, in the second to last position.
  • Stop when you wish.
  • Happy editing with multi-cursors!

End your multi-cursors session with C-g.

Setting the mark elsewhere

So, set-mark-command (C-SPC) is the way to manually set a mark exactly where the cursor is. Is that all?
Of course not. There are many other functions that help setting the mark to other specific positions, for example after a word, at the end of the page or at the end of a function.

In most of the applications, though, the mark is set for other reasons than just keeping a history of positions. In fact, in Emacs the mark is the foundation for defining a region, that is, for selecting text. In a sense, this ability to move back to previous positions is a byproduct of handling the region.

The Region

In the majority of editors, the region, or the selection, is the part of text that is temporarily highlighted, usually by the means of the mouse or using the arrows plus the Shift key.
In Emacs the region is way more powerful. As it often happens, this greater powers come from simpler foundations. Indeed, in Emacs the region is a trivial notion: it is just the part of buffer between the mark and the cursor.

Why do I say this is more powerful? Think how to do the following tasks with an editor other than Emacs or Vim. Start selecting some text, then:

  • Use any of the editor’s search capabilities to find where to end the selection.
  • Consult another file while you are selecting.
  • Move the focus to another application, get back to your editor and expect to find the selection where you left it.
  • Complete the selection. Then, change idea where the selection should start.

Those are all trivial tasks with Emacs. Possibly, just no possible with other editrs. If the other editors had to copy one functionality from Emacs, it should be this, hands down.

Getting back to those commands that somehow save a position in the mark: given what we said about the region, it should come with no surprises that the majority of them also activate the region. For example, mark-defun (C-M-h) sets the mark at the end of a function and moves the point at its beginning. The final effect is, the whole function will be selected.
You can find a more complete list in the chapter Marking Objects.

Knowing that selecting is about setting the the mark — so also about pushing the current value in the history — gives you the opportunity to perform some smart moves. For example:

  • Select the current function / paragraph with mark-defun (C-M-h).
  • Notice that the cursor ended up being at the beginning of the function. The selection ends where the function ends.
  • Use C-u C-SPC to move to the function end.
  • Press C-x C-x multiple times: you will keep jumping between the beginning and the end of the function.

If you’re not lacking creativity, you’ll eventually come up with a thousand clever uses for this feature.

Invisible Region

It might not be immediately clear, but when you pasted the text, it was already selected, although the selection was not highlighted. Emacs has got a very peculiar way of managing the selected text (the region): no matter if highlighted and visible, the region is always there. As we said, it is the part of text between the current cursor position and the most recent mark.

This allows you to do tricks such as the following. Imagine you have the point here:

{
    "parser-directories": [
        "/home/arialdo/prg/tree-sitter/"
    ],
*POINT*
}

You paste some code:

{
    "parser-directories": [
        "/home/arialdo/prg/tree-sitter/"
    ],
"theme": {
    "attribute": {
      "color": 124,
      "italic": true
    },
    "constant.builtin": {
      "bold": true,
      "color": 94
    }
}
}

and you notice that the result is horribly indented. Since you know that the code you just pasted is already selected — the mark and the cursor do surround it — you can run commands that expect a region. Type M-x indent-region RET or just simply C-M-\ and format it.

We saw this trick in Emacs: Let’s Surround! when talking about the region. You might be interested in finding a bit more details there.

Mark what?

Here are some questions you might legimately ask yourself:

  • Can I mark a position in a dired buffer?
  • Can I mark a commit in the Magit’s Git log buffer?
  • What about marking a position in vterm while in vterm-copy-mode?
  • Emacs can open PDF and PNG files: can I mark them?

Guess what? It’s a full house of “yes! yes! yes!”. Of course you can! Everything in Emacs is a text, everything lives a buffer. Why wouldn’t you be able to?

This is, to me, the beauty of Emacs. It’s not the amount of plugins (VS Code has 57202 extensions, so what?). It’s not its alleged Operating System nature. To me, the reason why Emacs stands out as an engineering product is that it is built on top of few core building blocks, just a handful of simple notions that marvelously build upon each other consistently, seemlessly, elegantly, creating a cohesive structure.

Note to myself: good things in life are temporary

Here’s a last question.

  • What if I mark a position and then I kill the buffer? Will Emacs resurrect it when jumping back?

And here, for once, the answer is “no”. Marks are really meant to be volatile.
But don’t despair: there are other means to do this.

Other Cases

There are other commands that automatically push the current cursor position in the mark ring before operating. A notable case is switch-to-buffer (C-x b), the command you use to move to a different buffer. You will be happy to know that every times you jump to another file and you mark a new position, Emacs scrupulously saves that breadcrumb also in the global mark ring. This will allow you to jump back with pop-global-mark (C-x C-SPC). And of course, consult.el has a command for browsing this: consult-global-mark.

This brings me to the last topic: other than the local mark rings (one per buffer), there is a global mark ring. What is it for?

Buffer Local and Global Marks

Each buffer keeps its own mark ring. This means that mark-ring is defined as a buffer local variable. You can verify that invoking M-x describe-variable RET mark-ring RET and then navigating to the source code:

(defvar-local mark-ring nil
  "The list of former marks of the current buffer, most recent first.")

By default, mark-ring keeps the last 16 positions. But you can customize this setting mark-ring-max.

As we just mentioned, there is also a global mark ring version, not surprisingly called global-mark-ring. Its maximum size is defined, guess what?, with global-mark-ring-max.

A couple of legit questions are:

  • Is it possible to feed this global mark ring?
  • When a mark is pushed locally, is this also reflected in the global mark ring?

The answer is jein! As usual, the manual is exhaustive:

Each time you set a mark, this is recorded in the global mark ring in addition to the current buffer’s own mark ring, if you have switched buffers since the previous mark setting.

Hence, the global mark ring records a sequence of buffers that you have been in, and, for each buffer, a place where you set the mark

To make it simple: as a modern Hansel & Gretel, Emacs keeps dropping off a breadcrumb every time you ask it so; and every time you happen to be on a different path, if you mark the new trail, it will lay done a white stone, a special mark you can follow to track down your journey from a bird’s-eye perspective:

global marks              
local marks
buffer A     B       C    

A bit of Lisp

If you are curious about how things work more than about how to use them (and it’s likely, if you spend time tinkering with Emacs) you might wonder: what does happen to the mark-ring variable when a value is pulled out of it? Is the mark-ring like a stack, that is progressively consumed as values are popped-out of it?

You could display the point, the mark and the mark ring value with something like:

(defun display-mark ()
  (interactive)
  (message "%s -> %s -> %s" (point) (mark) (mark-ring-positions)))

(defun mark-ring-positions ()
  (mapcar
   (lambda (item)
     (marker-position item))
   mark-ring))
   
(keymap-set global-map "C-c c" 'display-mark)

Just hit C-c c. You will find that the mark ring is really a circular structure. When you pop a value out of it, you are not really consuming it: the ring will rotate so, as long as you don’t exceed mark-ring-max values, you will never loose information.

What’s next?

Fantasizing how to improve the mark and its rings one could dream of some extra-functionalities:

  • Items in the history could be given a name so that they can be easily called back in any order.
  • Visiting back something that was killed should resurrect it.
  • Why to store positions only? Why not to store snippets of text too?
  • Now I come to think of it: everything is a text in Emacs? Why not to store keyboard macros in a history?

Wouldn’t it be cool to have all of these features?
Enter Registers!

(Thanks to Protesilaos for the kind review).

References

Comments

GitHub Discussions

-1:-- Emacs: Mark Ring (Post Arialdo Martini)--L0--C0--March 20, 2024 12:00 AM

Irreal: A Simple Macro

I’ve just had another of my occasional workflow epiphanies where I suddenly realize that some tedious routine task I regularly perform could easily be automated. In this case, only partially automated but one step at a time.

Everyday, I take a measurement and record the result in a notebook1. Every other Sunday I copy the measurements from my notebook into a file. I’ve been maintaining that file since before I started using Emacs so it doesn’t follow the conventions that I use now. In particular, each entry has a date with the format mm/dd/yy.

It’s really a pain to enter those dates. I realized that I could easily use a macro to enter them. My idea was to put the dd value in a macro counter and let the macro increment it for me for each new entry. Here’s my new procedure:

  • I use Ctrl+x Ctrl+k Ctrl+f to set the counter format to %02d so that each dd entry will be exactly two digits.
  • I use Ctrl+x Ctrl+k Ctrl+c to set the counter to the day number of the first entry’s date.
  • I define the macro by pressing F3 mm/ Ctrl+x Ctrl+k Ctrl+i to enter the day number, and then finish the macro with /yy F4.

It seems like a small thing but it really makes entering the data easier. Of course, what I actually thought was that I should write a function to do the date automatically but dealing with dates is a pain and what’s really needed here is a closure so that subsequent dates can be returned. That’s possible and I may do it eventually, but in the meantime, the macro is a big improvement.

Afternote

After I wrote this, Marcin Borkowski wrote a nice post on using keyboard macros that reminded me that you can use F3 instead of Ctrl+x Ctrl+k Ctrl+i to insert the counter value. I use counters in keyboard macros so seldomly that I’d forgotten that.

Footnotes:

1

Don’t ask. It’s my sole remaining use of pencil & paper for record keeping.

-1:-- A Simple Macro (Post jcs)--L0--C0--March 19, 2024 04:44 PM

T. V. Raman: Updated: Smart Media Selector For The Audio Desktop

Smart Media Selector For The Audio Desktop

1. Overview

I have over 60GB of audio content on my laptop spread across 755 subdirecories in over 9100 files. I also have many Internet stream shortcuts that I listen to on a regular basis.

This blog article outlines the media selector implementation in Emacspeak and shows how a small amount of Lisp code built atop Emacs' built-in affordances of completion provides a light-weight yet efficient interface. Notice that the implementation does not involve fancy things like SQL databases, MP3 tags that one needs to update etc.; the solution relies on the speed of today's laptops, especially given the speed of disk access.

2. User Experience

As I type this up, the set of requirements as expressed in English is far more verbose (and likely more complicated) than its expression in Lisp!

2.1. Pre-requisites for content selection and playback

  1. Launch either MPV (via package empv.el) or mplayer via Emacspeak's emacspeak-mplayer with a few keystrokes.
  2. Media selection uses ido with fuzzy matching.
  3. Choices are filtered incrementally for efficient eyes-free interaction; see the relevant blog article on Search, Input, Filter, Target for additional background.
  4. Content can be filtered using the directory structure, where directories conceptually equate to music albums, audio books or othre logical content groups.Once selected, a directory and its contents are played as a conceptual play-list.
  5. Searching and filtering can also occur across the list of all 9,100+ media files spread across 700+ directories.
  6. Starting point of the SIFT process should be influenced by one's current context, e.g., default-directory.
  7. Each step of this process should have reasonable fallbacks.

3. Mapping Design To Implementation

  1. Directory where we start AKA context is selected by function emacspeak-media-guess-directory.
    1. If default directory matches emacspeak-media-directory-regexp,use it.
    2. If default directory contains media files, then use it.
    3. If default directory contains directory emacspeak-media — then use it.
    4. Otherwise use emacspeak-media-shortcuts as the fallback.
  2. Once we have selected the context, function emacspeak-media-read-resourceuses ido style interaction with fuzzy-matching to pick the file to play.
  3. That function uses Emacs' built-in directory-files-recursively to build the collection to hand-off to completing-read; It uses an Emacspeak provided function ems–subdirs-recursively to build up the list of 755+ sub-directories that live under $XDGMUSICDIR.

4. Resulting Experience

  1. I can pick the media to play with a few keystrokes.
  2. I use Emacs' repeat-mode to advantage whereby I can quickly change volume etc once content is playing before going back to work.
  3. There's no media-player UI to get in my way while working, but I can stop playing media with a single keystroke.
  4. Most importantly, I dont have to tag media, maintain databases or do other busy work to be able to launch the media that I want!

5. The Lisp Code

The hyperlinks to the Emacspeak code-base are the source of truth. I'll include a snapshot of the functions mentioned above for completeness.

5.1. Guess Context

  (defun emacspeak-media-guess-directory ()
  "Guess media directory.
1. If default directory matches emacspeak-media-directory-regexp,use it.
2.  If default directory contains media files, then use it.
3. If default directory contains directory emacspeak-media --- then use it.
4. Otherwise use emacspeak-media-shortcuts as the fallback."
  (cl-declare (special emacspeak-media-directory-regexp
                       emacspeak-media emacspeak-m-player-hotkey-p))
  (let ((case-fold-search t))
    (cond
     ((or (eq major-mode 'dired-mode) (eq major-mode 'locate-mode)) nil)
     (emacspeak-m-player-hotkey-p   emacspeak-media-shortcuts)
     ((or                               ;  dir  contains media:
       (string-match emacspeak-media-directory-regexp default-directory)
       (directory-files default-directory   nil emacspeak-media-extensions))
      default-directory)
     ((file-in-directory-p emacspeak-media default-directory) emacspeak-media)
     (t   emacspeak-media-shortcuts))))

5.2. Read Resource

(defun emacspeak-media-read-resource (&optional prefix)
  "Read resource from minibuffer.
If a dynamic playlist exists, just use it."
  (cl-declare (special emacspeak-media-dynamic-playlist
                       emacspeak-m-player-hotkey-p))
  (cond
   (emacspeak-media-dynamic-playlist nil) ; do nothing if dynamic playlist
   (emacspeak-m-player-hotkey-p (emacspeak-media-local-resource prefix))
   (t                               ; not hotkey, not dynamic playlist
    (let* ((completion-ignore-case t)
           (read-file-name-completion-ignore-case t)
           (filename
            (when (memq major-mode '(dired-mode locate-mode))
              (dired-get-filename 'local 'no-error)))
           (dir (emacspeak-media-guess-directory))
           (collection
            (or
             filename                   ; short-circuit expensive call
             (if prefix
                 (ems--subdirs-recursively  dir) ;list dirs
               (directory-files-recursively dir emacspeak-media-extensions)))))
      (or filename (completing-read "Media: "  collection))))))

5.3. Helper: Recursive List Of Sub-directories

  ;;; Helpers: subdirs


(defconst ems--subdirs-filter
  (eval-when-compile
    (concat (regexp-opt '("/.." "/." "/.git")) "$"))
  "Pattern to filter out dirs during traversal.")

(defsubst ems--subdirs (d)
  "Return list of subdirs in directory d"
  (cl-remove-if-not #'file-directory-p (cddr (directory-files d 'full))))

(defun ems--subdirs-recursively (d)
  "Recursive list of  subdirs"
  (cl-declare (special ems--subdirs-filter))
  (let ((result (list d))
        (subdirs (ems--subdirs d)))
    (cond
     ((string-match ems--subdirs-filter d) nil)                              ; pass
     (t
      (cl-loop
       for dir in subdirs
       if (not (string-match ems--subdirs-filter dir)) do
       (setq result  (nconc result (ems--subdirs-recursively dir))))))
    result))


-1:-- Updated: Smart Media Selector For The Audio Desktop (Post T. V. Raman (noreply@blogger.com))--L0--C0--March 19, 2024 01:58 PM

Alvaro Ramirez: sqlite-mode-extras on MELPA

19 March 2024 sqlite-mode-extras on MELPA

sqlite-previous.gif

Emacs 29 introduced the handy sqlite-mode. Soon after, I tried a couple of experiments here and there to bring additional functionality.

Folks reached out. The additions seemed useful to them and were keen on upstreaming or pushing to MELPA. While I can't commit to upstreaming at this moment, I can happily meet halfway on MELPA.

As of a couple of days, you can find sqlite-mode-extras on MELPA and GitHub. Contributions totally welcome.

While I haven't heard of issues, please continue treating the package as experimental and exercise safety with your data. Please back up.

-1:-- sqlite-mode-extras on MELPA (Post)--L0--C0--March 19, 2024 12:54 PM

Unwound Stack: Org Mode & LaTeX

How exactly does Org Mode interact with LaTeX?

-1:-- Org Mode &amp; LaTeX (Post Michael (sp1ff@pobox.com))--L0--C0--March 19, 2024 07:00 AM

T. V. Raman: Smart Media Selector For The Emacspeak Audio Desktop

Smart Media Selector For The Audio Desktop

1. Overview

I have over 60MB of audio content on my laptop spread across 755 subdirecories in over 9100 files. I also have many Internet stream shortcuts that I listen to on a regular basis.

This blog article outlines the media selector implementation in Emacspeak and shows how a small amount of Lisp code built atop Emacs' built-in affordances of completion provides a light-weight yet efficient interface. Notice that the implementation does not involve fancy things like SQL databases, MP3 tags that one needs to update etc.; the solution relies on the speed of today's laptops, especially given the speed of disk access.

2. User Experience

As I type this up, the set of requirements as expressed in English is far more verbose (and likely more complicated) than its expression in Lisp!

2.1. Pre-requisites for content selection and playback

  1. Launch either MPV (via package empv.el) or mplayer via Emacspeak's emacspeak-mplayer with a few keystrokes.
  2. Media selection uses ido with fuzzy matching.
  3. Choices are filtered incrementally for efficient eyes-free interaction; see the relevant blog article on Search, Input, Filter, Target for additional background.
  4. Content can be filtered using the directory structure, where directories conceptually equate to music albums, audio books or othre logical content groups.Once selected, a directory and its contents are played as a conceptual play-list.
  5. Searching and filtering can also occur across the list of all 9,100+ media files spread across 700+ directories.
  6. Starting point of the SIFT process should be influenced by one's current context, e.g., default-directory.
  7. Each step of this process should have reasonable fallbacks.

3. Mapping Design To Implementation

  1. Directory where we start AKA context is selected by function emacspeak-media-guess-directory.
    1. If default directory matches emacspeak-media-directory-regexp,use it.
    2. If default directory contains media files, then use it.
    3. If default directory contains directory emacspeak-media — then use it.
    4. Otherwise use emacspeak-media-shortcuts as the fallback.
  2. Once we have selected the context, function emacspeak-media-read-resourceuses ido style interaction with fuzzy-matching to pick the file to play.
  3. That function uses Emacs' built-in directory-files-recursively to build the collection to hand-off to completing-read; It uses an Emacspeak provided function ems–subdirs-recursively to build up the list of 755+ sub-directories that live under $XDGMUSICDIR.

4. Resulting Experience

  1. I can pick the media to play with a few keystrokes.
  2. I use Emacs' repeat-mode to advantage whereby I can quickly change volume etc once content is playing before going back to work.
  3. There's no media-player UI to get in my way while working, but I can stop playing media with a single keystroke.
  4. Most importantly, I dont have to tag media, maintain databases or do other busy work to be able to launch the media that I want!

5. The Lisp Code

The hyperlinks to the Emacspeak code-base are the source of truth. I'll include a snapshot of the functions mentioned above for completeness.

5.1. Guess Context

  (defun emacspeak-media-guess-directory ()
  "Guess media directory.
1. If default directory matches emacspeak-media-directory-regexp,use it.
2.  If default directory contains media files, then use it.
3. If default directory contains directory emacspeak-media --- then use it.
4. Otherwise use emacspeak-media-shortcuts as the fallback."
  (cl-declare (special emacspeak-media-directory-regexp
                       emacspeak-media emacspeak-m-player-hotkey-p))
  (let ((case-fold-search t))
    (cond
     ((or (eq major-mode 'dired-mode) (eq major-mode 'locate-mode)) nil)
     (emacspeak-m-player-hotkey-p   emacspeak-media-shortcuts)
     ((or                               ;  dir  contains media:
       (string-match emacspeak-media-directory-regexp default-directory)
       (directory-files default-directory   nil emacspeak-media-extensions))
      default-directory)
     ((file-in-directory-p emacspeak-media default-directory) emacspeak-media)
     (t   emacspeak-media-shortcuts))))

5.2. Read Resource

(defun emacspeak-media-read-resource (&optional prefix)
  "Read resource from minibuffer.
If a dynamic playlist exists, just use it."
  (cl-declare (special emacspeak-media-dynamic-playlist
                       emacspeak-m-player-hotkey-p))
  (cond
   (emacspeak-media-dynamic-playlist nil) ; do nothing if dynamic playlist
   (emacspeak-m-player-hotkey-p (emacspeak-media-local-resource prefix))
   (t                               ; not hotkey, not dynamic playlist
    (let* ((completion-ignore-case t)
           (read-file-name-completion-ignore-case t)
           (filename
            (when (memq major-mode '(dired-mode locate-mode))
              (dired-get-filename 'local 'no-error)))
           (dir (emacspeak-media-guess-directory))
           (collection
            (or
             filename                   ; short-circuit expensive call
             (if prefix
                 (ems--subdirs-recursively  dir) ;list dirs
               (directory-files-recursively dir emacspeak-media-extensions)))))
      (or filename (completing-read "Media: "  collection))))))

5.3. Helper: Recursive List Of Sub-directories

  ;;; Helpers: subdirs


(defconst ems--subdirs-filter
  (eval-when-compile
    (concat (regexp-opt '("/.." "/." "/.git")) "$"))
  "Pattern to filter out dirs during traversal.")

(defsubst ems--subdirs (d)
  "Return list of subdirs in directory d"
  (cl-remove-if-not #'file-directory-p (cddr (directory-files d 'full))))

(defun ems--subdirs-recursively (d)
  "Recursive list of  subdirs"
  (cl-declare (special ems--subdirs-filter))
  (let ((result (list d))
        (subdirs (ems--subdirs d)))
    (cond
     ((string-match ems--subdirs-filter d) nil)                              ; pass
     (t
      (cl-loop
       for dir in subdirs
       if (not (string-match ems--subdirs-filter dir)) do
       (setq result  (nconc result (ems--subdirs-recursively dir))))))
    result))


-1:-- Smart Media Selector For The Emacspeak Audio Desktop (Post T. V. Raman (noreply@blogger.com))--L0--C0--March 19, 2024 03:53 AM

Marcin Borkowski: Follow mode

It is a fairly common opinion that a function should not be larger than your screen. The reality, though, is often different. And even if your functions are shorter, you may want to see more than one at a time. The problem is that our screen are usually not that high. (My laptop gives me 66 lines of text with normal font settings.) You can have an external monitor rotated vertically (I have that at work to see as much of the logs of the application I’m working on as possible), but Emacs gives us another solution – the Follow mode.
-1:-- Follow mode (Post)--L0--C0--March 18, 2024 04:52 PM

Irreal: Narrowing To Focus

Arthur A. Gleckler has a nifty post that discusses Emacs narrowing as a way to help with focus. The TL;DR is that sometimes you want to focus on a piece of code or other text and one way to do that is to temporarily remove all the surrounding text so that only the part you’re interested in is visible.

Emacs, of course, has you covered. It’s narrowing functions are just what’s called for. You can narrow to a region, a function, and various other parts of text. To a first approximation, it’s just what you need.

Gleckler, though, has an additional requirement. Sometimes he wants to look at the surrounding context and then return to the narrowed display. It’s easy, of course, to just widen the text, look at the context, and then narrow it again but that’s “complicated” enough that you can loose focus. He wanted a way to quickly pop in and out of narrowed displays.

He solved that problem with a bit of Elisp. The basic idea is that if there is an active region, he records the position of the region as an overlay with a special property and narrows to it. If no region is active, he looks for an overlay with the special property and uses its position as the inputs to narrow. Finally he binds a key shortcut to his function so that invoking it is as simple as it’s possible to be in Emacs.

The code is only a few lines and pretty easy to understand. If you’re interested in a similar facility or just want to see how he handled remembering which text to narrow to, take a look at his post. It’s short and easy to read.

-1:-- Narrowing To Focus (Post jcs)--L0--C0--March 18, 2024 04:04 PM

Sacha Chua: 2024-03-18 Emacs news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, kbin, programming.dev, lemmy, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

-1:-- 2024-03-18 Emacs news (Post Sacha Chua)--L0--C0--March 18, 2024 01:27 PM

Jeremy Friesen: An Emacs Function for Wrapping Ruby Functions

One of my “quality of life” improvements in my text editor is related to Rubocop 📖 ; a style guide enforcer that raises issues about complexity, formatting, and other things.

When I’m coding and feel I need to ignore Rubocop ’s opinion, I will tell Rubocop to ignore the violation. I do this by adding a “disable” comment before the violation and an “enable” comment after the violation.

I wrote an Emacs 📖 function that leverages Tree Sitter 📖 to insert the comments. Note: The jf/rubocop/list-all-cops is a constant I’ve defined that lists the numerous checks

update:

I refactored the original jf/treesit/wrap-rubocop to no longer kill the region and then yank the region back in. I also extracted the method that determines what is the current context.

The extraction added a new feature; namely attempt to determine the containing class/module declaration for point.

Read the latest code here.

  (defun jf/treesit/wrap-rubocop (&optional given-cops)
    "Wrap the current ruby region by disabling/enabling the GIVEN-COPS."
    (interactive)
    (if (derived-mode-p 'ruby-ts-mode 'ruby-mode)
      (if-let ((region (jf/treesit/derive-region-for-rubocop)))
        (let ((cops
                (or given-cops
                  (completing-read-multiple "Cops to Disable: "
                    jf/rubocop/list-all-cops nil t))))
          (save-excursion
            (goto-char (cdr region))
            (call-interactively #'crux-move-beginning-of-line)
            (let ((indentation (s-repeat (current-column) " ")))
              (goto-char (cdr region))
              (insert "\n"
                (s-join "\n"
                  (mapcar
                    (lambda (cop)
                      (concat indentation "# rubocop:enable " cop))
                    cops)))
              (goto-char (car region))
              (beginning-of-line)
              (insert
                (s-join "\n"
                  (mapcar
                    (lambda (cop)
                      (concat indentation "# rubocop:disable " cop))
                    cops))
                "\n"))))
        (user-error "Not a region nor a function"))
      (user-error "%s is not derived from a ruby mode" major-mode)))

  (defun jf/treesit/derive-region-for-rubocop ()
    "Return `cons' of begin and end positions of region."
    (cond
      ;; When given, first honor the explicit region
      ((use-region-p)
        (cons (region-beginning) (region-end)))
      ;; Then honor the current function
      ((treesit-defun-at-point)
        (cons (treesit-node-start (treesit-defun-at-point))
          (treesit-node-end (treesit-defun-at-point))))
      ;; Then fallback to attempting to find the containing
      ;; class/module.
      (t
        (when-let ((node
                     (treesit-parent-until
                       (treesit-node-at (point))
                       (lambda (n) (member (treesit-node-type n)
                                     '("class" "module"))))))
          (cons (treesit-node-start node) (treesit-node-end node))))))

I’ve bound this to C-c w r; mnemonically this is “Command Wrap Rubocop.”

A few times per week, I use the jf/treesit/wrap-rubocop function. A small improvement in my work experience.

-1:-- An Emacs Function for Wrapping Ruby Functions (Post Jeremy Friesen (jeremy@takeonrules.com))--L0--C0--March 17, 2024 11:06 PM

Manuel Uberti: The irresistible tension of language

“Mein Leben ist zu Ende, denn er ist auf dem Transport im Fluß ertrunken, er war mein Leben. Ich habe ihn mehr geliebt als mein Leben.”

Malina (Ingeborg Bachmann, 1971)1

My career at Ca’ Foscari is coming to an end, at least in terms of my BA in philosophy. I will write about this long and wild journey another time, but today I want to expand upon a brief talk I gave in class last Tuesday as part of an assignment for the Philosophy of Literature course.

The professor asked us to work in groups on one of the essays we have to study for the final exam, free to develop its contents in any feasible direction. For reasons I am not entirely able to explain I was drawn to Aldo Gargani’s Il pensiero raccontato, a marvellous study about Austrian poet and author Ingeborg Bachmann. Two other students soon joined me and we pleasantly lost ourselves in her words as well as among the ones by Paul Celan, Thomas Bernhard and Ludwig Wittgenstein.

I was particularly fascinated by Gargani’s focus on the tension in Bachmann and Bernhard writings. This tension is not just the reader’s feeling as they turn the pages, albeit there are passages in Bachmann’s Malina and Bernhard’s Gargoyles2 perfectly capable of shaking one’s own psychological grounds. The tension that Gargani highlights is within the actual language used by these authors. More specifically, the tension is between what is shown by the language and what cannot be said. Where does this tension lead?

According to Gargani the tension is always aimed at the reality we live in, its purpose nothing less than being the cause of a “moral leap”, as Bachmann puts it in her Frankfurt lessons.3 Bachmann wants us to understand that in order to confront the present we are thrown into and survive, we have to look at it through lenses that are not, and cannot, be set once and for all. The past is always transforming the present just as the present never sits still. There is nothing, then, that we can seriously take for granted. Bachmann creates a language that forces us to constantly look at where we are now and, at the same time, gaze into where we really should be. This is why in her view literature must be capable of influencing any present, the one we are currently in and every present that is yet to come.4

The quote from Malina I put at the beginning of this writing is the same one I used in class. To me it is the perfect example of what Gargani so acutely finds in Bachmann. In it she is conflating the tragic experience of Nazi Germany5 and the suicide of Romanian poet Paul Celan,6 thus keeping alive the horrible connection between the past, hers and ours, with the present, hers and ours. Furthermore, she is pushing us to look for implicit and explicit connections within the text and outside of it. Bachmann is fully aware that her book should not matter just for the people alive at the time of her writing. For it to be of any value to all the possible readers that will get to it eventually, its language has to keep renewing itself for every reader, creating tension between different times and different spaces.

As Jacques Derrida and Walter Benjamin wrote, this can only be done by using live, dynamic affinities instead of channeling static similarities.7 This means that language must not act as a homogenising common denominator. It should not be a universal item that subsumes concepts, historical processes, and cultural events under one, giant umbrella. If we lose heterogeneity, we lose the possibility of seeing beyond ourselves. If we cannot see beyond ourselves, we will never be able to change, and only the dead never changes.


  1. “My life is over, for during the deportation he has drowned in the river, he was my life. I loved him more than my life.” The English translation of Malina from New Directions uses “transport” instead of “deportation”, whereas the Italian translation published by Adelphi goes with “deportazione”. I prefer the Italian version because it makes the not so subtle reference to the concentration camps clearer. ↩︎

  2. The German title, Verstörung, translates as something like confusion or derangement, whereas the Italian title is Perturbamento, which is closer to the original meaning. ↩︎

  3. I am referring to Letteratura come utopia. Lezioni di Francoforte published by Adelphi, but there is an English translation as well titled The Critical Writings of Ingeborg Bachmann↩︎

  4. See: Letteratura come utopia. Lezioni di Francoforte↩︎

  5. Her father was a member of the Austrian National Socialist Party. ↩︎

  6. The relationship between Ingeborg Bachmann and Paul Celan was so intense and incredible it deserves a much better writer than myself. To get an idea I highly recommend starting with Correspondence: Ingeborg Bachmann and Paul Celan and then watching Ruth Beckermann’s The Dreamed Ones↩︎

  7. See Derrida’s Des tours de Babel and Qu’est-ce qu’une traduction “relevante”?, and Benjamin’s The Task of the Translator and On Language as Such and on the Language of Man↩︎

-1:-- The irresistible tension of language (Post Manuel Uberti (manuel.uberti@inventati.org))--L0--C0--March 17, 2024 07:12 AM

Please note that planet.emacslife.com aggregates blogs, and blog authors might mention or link to nonfree things. To add a feed to this page, please e-mail the RSS or ATOM feed URL to sacha@sachachua.com . Thank you!