Yet Another Refinement to My Blogging Engine

Today, I made a slight modification to how I write my blog posts.

I use Org-Mode (Org-mode 📖) to write my blog posts. I use a a modified Tufte CSS theme, derived from Tufte CSS.

My theme has a concept of the side-note and the margin-note. A few years ago, I moved away from Tufte’s preferred fonts, instead letting folks’s browsers determine the font. The margin-note is for general “scribbling” in the margins. And the side-note is analogues to an inline footnote.

Up until today I leveraged Ox-Hugo’s shortcode customizations to handle both. This afternoon, I wrote the below function to replace Ox-Hugo 📖’s export function for footnotes.

(defun jf/org-hugo-sidenote (footnote-reference _contents info)
  "Transcode a FOOTNOTE-REFERENCE element from Org to Hugo sidenote shortcode.
CONTENTS is nil.  INFO is a plist holding contextual information."
  (let* ((element (car (org-export-get-footnote-definition footnote-reference info)))
         (beg (org-element-property :contents-begin element))
         (end (org-element-property :contents-end element)))
    (format "{{< sidenote >}}%s{{< /sidenote >}}"
            (s-trim (buffer-substring-no-properties beg end)))))

;; Over-write the custom blackfriday export for footnote links.
(advice-add #'org-blackfriday-footnote-reference
            :override #'jf/org-hugo-sidenote
            '((name . "wrapper")))

;; Don't render the section for export
(advice-add #'org-blackfriday-footnote-section
            :override (lambda (&rest rest) ())
            '((name . "wrapper")))

With the above function and advice all Org-mode exports, except to my blog, the footnotes retain their original export behavior. I definitely prefer to utilize as much of the native functionality as possible.

This guide gives you a starting point on writing a tree-sitter major mode. Remember, don’t panic and check your manuals!

Build Emacs with tree-sitter

You can either install tree-sitter by your package manager, or from
source:

git clone https://github.com/tree-sitter/tree-sitter.git
cd tree-sitter
make
make install

To build and run Emacs 29:

git clone https://git.savannah.gnu.org/git/emacs.git -b emacs-29
cd emacs
./autogen.sh
./configure
make
src/emacs

Require the tree-sitter package with (require 'treesit). Note that tree-sitter always appear as treesit in symbols. Now check if Emacs is successfully built with tree-sitter library by evaluating (treesit-available-p).

Tree-sitter stuff in Emacs can be categorized into two parts: the tree-sitter API itself, and integration with fontification, indentation, Imenu, etc. You can use shortdoc to glance over all the tree-sitter API functions by typing M-x shortdoc RET treesit RET. The integration are described in the rest of the post.

Install language definitions

Tree-sitter by itself doesn’t know how to parse any particular language. It needs the language grammar (a dynamic library) for a language to be able to parse it.

First, find the repository for the language grammar, eg, tree-sitter-python. Take note of the Git clone URL of it, eg, https://github.com/tree-sitter/tree-sitter-python.git. Now check where is the parser.c file in that repository, usually it’s in src.

Make sure you have Git, C and C++ compiler, and run the treesit-install-grammar command, it will prompt for the URL and the directory of parser.c, leave other prompts at default unless you know what you are doing.

You can also manually clone the repository and compile it, and put the dynamic library at a standard library location. Emacs will be able to find it. If you wish to put it somewhere else, set treesit-extra-load-path so Emacs can find it.

Tree-sitter major modes

Tree-sitter modes should be separate major modes, usually named xxx-ts-mode. I know I said tree-sitter always appear as treesit in symbols, this is the only exception.

If the tree-sitter mode and the “native” mode could share some setup code, you can create a “base mode”, which only contains the common setup. For example, there is python-base-mode (shared), and both python-mode (native), and python-ts-mode (tree-sitter) derives from it.

In the tree-sitter mode, check if we can use tree-sitter with treesit-ready-p, it will emit a warning if tree-sitter is not ready (tree-sitter not built with Emacs, can’t find the language grammar, buffer too large, etc).

Fontification

Tree-sitter works like this: It parses the buffer and produces a parse tree. You provide a query made of patterns and capture names, tree-sitter finds the nodes that match these patterns, tag the corresponding capture names onto the nodes and return them to you. The query function returns a list of (capture-name . node).

For fontification, we simply use face names as capture names. And the captured node will be fontified in their capture name (the face).

The capture name could also be a function, in which case (NODE OVERRIDE START END) is passed to the function for fontification. START and END are the start and end of the region to be fontified. The function should only fontify within that region. The function should also allow more optional arguments with &rest _, for future extensibility. For OVERRIDE check out the docstring of treesit-font-lock-rules.

Query syntax

There are two types of nodes: “named nodes”, like (identifier), (function_definition), and “anonymous nodes”, like "return", "def", "(", ";". Parent-child relationship is expressed as

(parent (child) (child) (child (grand_child)))

Eg, an argument list (1, "3", 1) would be:

(argument_list "(" (number) (string) (number) ")")

Children could have field names:

(function_definition name: (identifier) type: (identifier))

To match any one in the list:

["true" "false" "none"]

Capture names can come after any node in the pattern:

(parent (child) @child) @parent

The query above captures both the parent and the child.

The query below captures all the keywords with capture name
"keyword":

["return" "continue" "break"] @keyword

These are the common syntax, check out the full syntax in the manual: Pattern Matching.

Query references

But how do one come up with the queries? Take python for an example, open any python source file, type M-x treesit-explore-mode RET. You should see the parse tree in a separate window, automatically updated as you select text or edit the buffer. Besides this, you can consult the grammar of the language definition. For example, Python’s grammar file is at

https://github.com/tree-sitter/tree-sitter-python/blob/master/grammar.js

Neovim also has a bunch of queries to reference from.

The manual explains how to read grammar files in the bottom of Language Grammar.

Debugging queries

If your query has problems, use treesit-query-validate to debug the query. It will pop a buffer containing the query (in text format) and mark the offending part in red. Set treesit--font-lock-verbose to t if you want the font-lock function to report what it’s doing.

Set up font-lock

To enable tree-sitter font-lock, set treesit-font-lock-settings and treesit-font-lock-feature-list buffer-locally and call treesit-major-mode-setup. For example, see python--treesit-settings in python.el. Below is a snippet of it.

Note that like the current font-lock system, if the to-be-fontified region already has a face (ie, an earlier match fontified part/all of the region), the new face is discarded rather than applied. If you want later matches always override earlier matches, use the :override keyword.

Each rule should have a :feature, like function-name, string-interpolation, builtin, etc. This way users can enable/disable each feature individually.

Read the manual section Parser-based Font-Lock for more detail.

Example from python.el:

(defvar python--treesit-settings
  (treesit-font-lock-rules
   :feature 'comment
   :language 'python
   '((comment) @font-lock-comment-face)

   :feature 'string
   :language 'python
   '((string) @python--treesit-fontify-string)

   :feature 'string-interpolation
   :language 'python
   :override t
   '((interpolation (identifier) @font-lock-variable-name-face))

   ...))

In python-ts-mode:

(treesit-parser-create 'python)
(setq-local treesit-font-lock-settings python--treesit-settings)
(setq-local treesit-font-lock-feature-list
                '(( comment definition)
                  ( keyword string type)
                  ( assignment builtin constant decorator
                    escape-sequence number property string-interpolation )
                  ( bracket delimiter function operator variable)))
...
(treesit-major-mode-setup)

Concretely, something like this:

(define-derived-mode python-ts-mode python-base-mode "Python"
  "Major mode for editing Python files, using tree-sitter library.

\\{python-ts-mode-map}"
  :syntax-table python-mode-syntax-table
  (when (treesit-ready-p 'python)
    (treesit-parser-create 'python)
    (setq-local treesit-font-lock-feature-list
                '(( comment definition)
                  ( keyword string type)
                  ( assignment builtin constant decorator
                    escape-sequence number property string-interpolation )
                  ( bracket delimiter function operator variable)))
    (setq-local treesit-font-lock-settings python--treesit-settings)
    (setq-local imenu-create-index-function
                #'python-imenu-treesit-create-index)
    (setq-local treesit-defun-type-regexp (rx (or "function" "class")
                                              "_definition"))
    (setq-local treesit-defun-name-function
                #'python--treesit-defun-name)
    (treesit-major-mode-setup)

    (when python-indent-guess-indent-offset
      (python-indent-guess-indent-offset))))

Indentation

Indentation works like this: We have a bunch of rules that look like

(MATCHER ANCHOR OFFSET)

When the indenting a line, let NODE be the node at the beginning of the current line, we pass this node to the MATCHER of each rule, one of them will match the node (eg, “this node is a closing bracket!”). Then we pass the node to the ANCHOR, which returns a point (eg, the beginning of NODE’s parent). We find the column number of that point (eg, 4), add OFFSET to it (eg, 0), and that is the column we want to indent the current line to (4 + 0 = 4).

Matchers and anchors are functions that takes (NODE PARENT BOL &rest _). Matches return nil/non-nil for no match/match, and anchors return the anchor point. An Offset is usually a number or a variable, but it can also be a function. Below are some convenient builtin matchers and anchors.

For MATHCER we have

(parent-is TYPE) => matches if PARENT’s type matches TYPE as regexp
(node-is TYPE) => matches NODE’s type
(query QUERY) => matches if querying PARENT with QUERY
                 captures NODE.

(match NODE-TYPE PARENT-TYPE NODE-FIELD
       NODE-INDEX-MIN NODE-INDEX-MAX)

=> checks everything. If an argument is nil, don’t match that. Eg,
(match nil TYPE) is the same as (parent-is TYPE)

For ANCHOR we have

first-sibling => start of the first sibling
parent => start of parent
parent-bol => BOL of the line parent is on.
prev-sibling => start of previous sibling
no-indent => current position (don’t indent)
prev-line => start of previous line

There is also a manual section for indent: Parser-based Indentation.

When writing indent rules, you can use treesit-check-indent to
check if your indentation is correct. To debug what went wrong, set
treesit--indent-verbose to t. Then when you indent, Emacs
tells you which rule is applied in the echo area.

Here is an example:

(defvar typescript-mode-indent-rules
  (let ((offset 'typescript-indent-offset))
    `((typescript
       ;; This rule matches if node at point is ")", ANCHOR is the
       ;; parent node’s BOL, and offset is 0.
       ((node-is ")") parent-bol 0)
       ((node-is "]") parent-bol 0)
       ((node-is ">") parent-bol 0)
       ((node-is "\\.") parent-bol ,offset)
       ((parent-is "ternary_expression") parent-bol ,offset)
       ((parent-is "named_imports") parent-bol ,offset)
       ((parent-is "statement_block") parent-bol ,offset)
       ((parent-is "type_arguments") parent-bol ,offset)
       ((parent-is "variable_declarator") parent-bol ,offset)
       ((parent-is "arguments") parent-bol ,offset)
       ((parent-is "array") parent-bol ,offset)
       ((parent-is "formal_parameters") parent-bol ,offset)
       ((parent-is "template_substitution") parent-bol ,offset)
       ((parent-is "object_pattern") parent-bol ,offset)
       ((parent-is "object") parent-bol ,offset)
       ((parent-is "object_type") parent-bol ,offset)
       ((parent-is "enum_body") parent-bol ,offset)
       ((parent-is "arrow_function") parent-bol ,offset)
       ((parent-is "parenthesized_expression") parent-bol ,offset)
       ...))))

Then you set treesit-simple-indent-rules to your rules, and call treesit-major-mode-setup.

Imenu

Set treesit-simple-imenu-settings and call treesit-major-mode-setup.

Navigation

Set treesit-defun-type-regexp, treesit-defun-name-function, and call treesit-major-mode-setup.

C-like languages

c-ts-mode.el has some goodies for handling indenting and filling block comments.

These two rules should take care of indenting block comments.

((and (parent-is "comment") c-ts-mode--looking-at-star)
            c-ts-mode--comment-start-after-first-star -1)
((parent-is "comment") prev-adaptive-prefix 0)

Set c-ts-mode-indent-block-type-regexp and these two rules should take care of indenting statements in “{}” blocks and closing bracket “}”.

((node-is "

") point-min c-ts-mode--close-bracket-offset)
((parent-is "compound_statement")
point-min c-ts-mode--statement-offset)
}

c-ts-mode-comment-setup will set up comment and filling for you.

Multi-language modes

Refer to the manual: Multiple Languages.

Common Tasks

M-x shortdoc RET treesit RET will give you a complete list.

How to...

Get the buffer text corresponding to a node?

(treesit-node-text node)

Don’t confuse this with treesit-node-string.

Scan the whole tree for stuff?

(treesit-search-subtree)
(treesit-search-forward)
(treesit-induce-sparse-tree)

Find/move to to next node that...?

(treesit-search-forward node ...)
(treesit-search-forward-goto node ...)

Get the root node?

(treesit-buffer-root-node)

Get the node at point?

(treesit-node-at (point))

Emacs’ release branch is now on complete feature freeze, meaning absolutely only bug fixes can happen on it. Now is a good time to talk about the state of tree-sitter in Emacs: what do you get in Emacs 29, what you don’t, and what would happen going forward.

What’s in Emacs 29

From a pure user’s perspective, Emacs 29 just adds some new built-in major modes which look more-or-less identical to the old ones. There aren’t any flashy cool features either. That sounds disappointing, but there are a lot of new stuff under the hood, a solid base upon which exciting things can emerge.

If Emacs 29 is built with the tree-sitter library, you have access to most of the functions in its C API, including creating parsers, parsing text, retrieving nodes from the parse tree, finding the parent/child/sibling node, pattern matching nodes with a DSL, etc. You also get a bunch of convenient functions built upon the primitive functions, like searching for a particular node in the parse tree, cherry picking nodes and building a sparse tree out of the parse tree, getting the node at point, etc. You can type M-x shortdoc RET treesit RET to view a list of tree-sitter functions. And because it’s Emacs, there is comprehensive manual coverage for everything you need to know. It’s in “Section 37, Parsing Program Source” of Emacs Lisp Reference Manual.

Emacs 29 has built-in tree-sitter major modes for C, C++, C#, Java, Rust, Go, Python, Javascript, Typescript, JSON, YAML, TOML, CSS, Bash, Dockerfile, CMake file. We tried to extend existing modes with tree-sitter at first but it didn’t work out too well, so now tree-sitter lives in separate major modes. The tree-sitter modes are usually called xxx-ts-mode, like c-ts-mode and python-ts-mode. The simplest way to enable them is to use major-mode-remap-alist. For example,

(add-to-list 'major-mode-remap-alist
             '(c-mode . c-ts-mode))

The built-in tree-sitter major modes have support for font-lock (syntax highlight), indentation, Imenu, which-func, and defun navigation.

For major mode developers, Emacs 29 includes integration for these features for tree-sitter, so major modes only need to supply language-specific information, and Emacs takes care of plugging tree-sitter into font-lock, indent, Imenu, etc.

Fontification

In tree-sitter major modes, fontification is categorized into “features”, like “builtin”, “function”, “variable”, “operator”, etc. You can choose what “features” to enable for a mode. If you are feeling adventurous, it is also possible to add your own fontification rules.

To add/remove features for a major mode, use treesit-font-lock-recompute-features in its mode hook. For example,

(defun c-ts-mode-setup ()
  (treesit-font-lock-recompute-features
   '(function variable) '(definition)))

(add-hook 'c-ts-mode-hook #'c-ts-mode-setup)

Features are grouped into decoration levels, right now there are 4 levels and the default level is 3. If you want to program in skittles, set treesit-font-lock-level to 4 ;-)

Language grammars

Tree-sitter major modes need corresponding langauge grammar to work. These grammars come in the form of dynamic libraries. Ideally the package manager will build them when building Emacs, like with any other dynamic libraries. But they can’t cover every language grammar out there, so you probably need to build them yourself from time to time. Emacs has a command for it: treesit-install-language-grammar. It asks you for the Git repository and other stuff and builds the dynamic library. Third-party major modes can instruct their users to add the recipe for building a language grammar like this:

(add-to-list
 'treesit-language-source-alist
 '(python "https://github.com/tree-sitter/tree-sitter-python.git"))

Then typing M-x treesit-install-language-grammar RET python builds the language grammar without user-input.

Other stuff

Things like indentation, Imenu, navigation, etc, should just work.

There is no code-folding, selection expansion, and structural navigation (except for defun) in Emacs 29. Folding and expansion should be trivial to implement in existing third-party packages. Structural navigation needs careful design and nontrivial changes to existing commands (ie, more work). So not in 29, unfortunately.

Future plans

Navigation

The tree-sitter integration is far from complete. As mentioned earlier, structural navigation is still in the works. Right now Emacs allows you to define a “thing” by a regexp that matches node types, plus optionally a filter function that filters out nodes that matches the regexp but isn’t really the “thing”. Given the definition of a “thing”, Emacs has functions for finding the “things” around point (treesit--things-around), finding the “thing” at point (treesit--thing-at-point), and navigating around “things” (treesit--navigate-thing). Besides moving around, these functions should be also useful for other things like folding blocks. Beware that, as the double dash suggests, these functions are experimental and could change.

I also have an idea for “abstract list elements”. Basically an abstract list element is anything repeatable in a grammar: defun, statement, arguments in argument list, etc. These things appear at every level of the grammar and seems like a very good unit for navigation.

Context extraction

There is also potential for language-agnostic “context extraction” (for the lack of a better term) with tree-sitter. Right now we can get the name and span of the defun at point, but it doesn’t have to stop there, we can also get the parameter list, the type of the return value, the class/trait of the function, etc. Because it’s language agnostic, any tool using this feature will work on many languages all at once.

In fact, you can already extract useful things, to some degree, with the fontification queries written by major modes: using the query intended for the variable query, I can get all the variable nodes in a given range.

There are some unanswered questions though: (1) What would be the best function interface and data structure for such a feature? Should it use a plist like (:name ... :params ...), or a cl-struct? (2) If a language is different enough from the “common pattern”, how useful does this feature remains? For example, there isn’t a clear parameter list in Haskell, and there could be several defun bodies that defines the same function. (3) Is this feature genuinely useful, or is it just something that looks cool? Only time and experiments can tell, I’m looking forward to see what people will do with tree-sitter in the wild :-)

Major mode fallback

Right now there is no automatic falling back from tree-sitter major modes to “native” major modes when the tree-sitter library or language grammar is missing. Doing it right requires some change to the auto-mode facility. Hopefully we’ll see a good solution for it in Emacs 30. Right now, if you need automatic fallback, try something like this:

(define-derived-mode python-auto-mode prog-mode "Python Auto"
  "Automatically decide which Python mode to use."
  (if (treesit-ready-p 'python t)
      (python-ts-mode)
    (python-mode)))

Other stuff

Existing tree-sitter major modes are pretty basic and doesn’t have many bells and whistles, and I’m sure there are rough corners here and there. Of course, these things will improve over time.

Tree-sitter is very different and very new, and touches many parts of Emacs, so no one has experience with it and no one knows exactly how should it look like. Emacs 29 will give us valuable experience and feedback, and we can make it better and better in the future.

If you are interested, get involved! Read Contributing to Emacs for some tips in getting involved with the Emacs development. Read Tree-sitter Starter Guide if you want to write a major mode using tree-sitter. And of course, docstrings and the manual is always your friend. If you have questions, you can ask on Reddit, or comment in this post’s public inbox (see the footer).

Revisiting Past Writings, Further Describing Personal Workflow, and Issuing and Admonition

Long ago, I started writing an ebook for my “Take on Development”. It sits fallow.

One chapter that I continue to think about is the Modes of Development; in particular the section “Modes that I can think of”.

As I was preparing for the day’s work, I recognized a new mode of thinking. It’s the mode of work that comes before Writing a Commit Message. But before I dive into that, let’s first look at Writing a Commit Message.

Writing a Commit Message

Goal

To create a helpful message to future developers

Method

Follow a well established pattern for commit messages; 50 character first line, references to issue tracker incidences, meaningful message body if the concept is complicated. Unless self evident, the message should describe the state prior to the commit, then after the commit, and provide insight into why you made the change.

Caveat

If I am working on a feature that will require multiple commits, I will make "disposable commits" along the way. These commits are mental placeholders. I will revisit them as I prepare a branch for merging.

At this point in my workflow, I’m explaining what I’m sending forward into the future. Yet there’s an assumption and a mode I haven’t identified. And it maps to an extent Git 📖 commands: git add. The git add command is how you stage your changes, the changes that you’ll describe with your commit message.

In the years since writing the chapter, I’ve adopted a new approach. One that’s powered by Magit. But first let’s talk about the new Modes of Development:

Staging the Commit

Goal

To ensure that I’m sending forward the “right” stuff.

Method

Using the equivalent of git add --interactive (see Interactive Staging), I review the chunks of change (e.g. each diff section).

During this mode, I’m reviewing what has changed and explicitly accepting, reverting, or skipping the possible changes. I’m switching from writing’s author to it’s editor.

While writing the commit message, I’m moving from to publisher; writing the press release and headline.

As eager as I am to just publish this change, slowing down and following these steps provides opportunities for improved understanding.

I’ve caught bugs in the editing and publishing stages, because I’ve stopped coding. While staging the commit, I’m reflecting on the specific code I’ve written. And while writing the commit message, I’ve reflected not on the specific code, but the changing state of the code.

Put another way, this is about the process of synthesis; of reflecting on the written code and writing about what’s changed and why.

Using Tools for Interactive Staging

When I first started using Git, when I was preparing to write a commit, I’d run git status. This command would show me the files I was changing. I’d then run git add my/filename to add one file.

In interactive staging, you’re instead reviewing and operating on the chunks of changes grouped by file. As I mentioned before, I use Magit for this work.

First and foremost, I devote my full attention to this; it’s the only window I have open. And the only thing in that window is the list of untracked and unstaged changes (e.g. the default Magit dispatch buffer).

Why is this important? Focus. I’m no longer writing. I’m reviewing changes. I’m in a different frame of mind; one that deserves focus.

I don’t want the rest of my editing ecosystem—such as minimaps, terminal sessions, project navigation trees, and other tabs with code in them—cluttering my current task.

Next, I begin reviewing the diff of the changed files. Within Magit I can stage, skip, or even revert each chunk. I can also highlight lines within a diff to just stage or revert those lines. Surgical precision! As I complete my actions on one region, Magit advances me to the next.

Sometimes, I’ll do all of this rather quickly; if I haven’t had any disruptions from when I first started writing the code. Responding to Slack, jumping to another project, stepping away for the evening all can count as distractions. When that is done, I begin shifting modes. Now I’m ready to begin Writing a Commit Message. For this I also use Emacs 📖; if I’m going to write something, I want to do it in my text editor!

Commit Hygiene Admonition

Use a proper editor for writing your commit message. The git commit -m "Fixing broken template" is thumbing your nose at future maintainers. If this is an important change, take time to describe and summarize it. The summary is the commit’s title (e.g. the first line). The body is the lines that come after.

I’ve wrote more about this in Commit Messages Do Matter and Leveraging Commit Messages as Pull Request Text. Too Long; Didn't Read (TL;DR 📖) All of the major remote Git services will use your commit’s title as the title of the Pull Request (PR 📖) and the body of the PR as the text of the pull request.

And to follow the principles of Publish (on your) Own Site, Syndicate Elsewhere (POSSE 📖) and Don't Repeat Yourself (DRY 📖), write those messages to the Git repository. Let them travel with you and future code maintainers.

I cannot emphasize the above enough.

But to add some additional context consider the following: Have you ever wanted to get better at using your text editor or Integrated Development Environment (IDE 📖)?

If the answer is yes, then one way to do that is to use it. And if you’re not using it to write commit messages, this is a golden opportunity to add another place of practice.

Take the time to do this. Think of your text editor as a mental muscle that you exercise and strengthen and keep flexible.

For myself, I’m always hungry to move new processes and approaches into Emacs; because I have seen the dividends in the output and I assume quality of my writing. I mention this in passing in Wrapping up Another Year for Take on Rules. Here are some posts that touch on my thoughts regarding my text editor:

• Principles of My Text Editor
• Revisiting the Principles of My Text Editor
• On Sharpening Your Tools

Conclusion

Modernity fills life with distractions; ever hungry for our attention. Consider the state of mind your work requires and establish procedures rituals to get you there.

Want to see how I do the interatcive staging? Contact me and I could do a screen share and walk through this. I’d probably consider recording that session so we could share it out further.

And while I favor Emacs, I know that this approach is most certainly available in your editor of choice. Go forth and practice.

John McCarthy, the progenitor of LISP is quoted to have said

An excessive knowledge of Marxism is a sign of a misspent youth.

I can’t find any good source for this quote, but I can think of a contemporary variation

An excessive knowledge of Lisp is a sign of a misspent youth.

My first encounter with “Lisp” was exactly 8 years ago, when a college of my father lent me his copy of The Structure and Interpretation of Computer Programes. I did not understand it immediately, and I can now reflect on the manifestation of my misunderstandings by reading through a failed implementation I wrote as a teenager (it also demonstrates my lack of understanding C). Eventually I grokked Scheme, Common Lisp and Emacs Lisp (and C). The last of these three has become the one I am the most familiar and the most comfortable using. For the fun of it, I recently wrote a 16-page article on a simple Lisp macro, just to comment on various aspects of the family of languages.

Over time I began contributing to emacs-devel, the development mailing list and besides maintaining my own packages, I tried to help with the popularisation of NonGNU ELPA by collecting packages and reviewing submissions. I contributed a few features to core Emacs such as IRCv3 support for rcirc and recently support for installing packages directly from source using package.el.

While I do think that this has been a valuable experience, I do feel a certain embarrassment when reflecting upon the time I have spent on Emacs and related technologies. Combine this with the fact that I am coming close to the end of my studies, one has to acknowledge that (Emacs) Lisp is not an active research topic. While my interests lie in theoretical computer science, a professor was right in noting to me after a sub-optimal exam: “I know you are a good hacker, but you also need to know theory”.

I look at the number of articles I have written on this site on the topic of Emacs and it saddens me that my repertoire is so one sided: My life as a “programmer” up until now has roughly been divided into two stages. The first (2012-2017) was as an autodidact, where I attempted to teach myself various topics through books, blogs and whatever I could understand. My main mistake during this phase was that I intentionally tried to avoid certain topics, as to not “taint” my understanding of theoretical topics I ultimately never engaged with, in some naive hope to perhaps be able and contribute a novel perspective. The second (2017-now) is as a student where I was taught the curriculum of a CS bachelor and master.

While the latter has been important in forcing me to engage with topics I didn’t get at first or was too lazy to acquire on my own (the formal notion of computation, proof and logic, the functioning operating systems, etc.) I am also under the impression that the quality of my learning has been a lot worse. If I had to guess, this is probably due to the structuring effect of the necessity to grade, hence to write exams, hence to teach what is “examable” – that is to say what can reasonably be inspected to see if students appear to have understood some topic. On more than one occasion I have been too stubborn to engage with this fact. I refuse to memorise what interests me, and instead try to acquire a fundamental understanding of some topic; which is not the best strategy when you only have a few days to prepare for an exam. Another time I just had no interest to study because I found myself already knowing everything on a topic that interested me. My grade suffered accordingly. It should be said that none of this is surprising from an institution that is in some sense a thousand years old, caught between the needs of modern industry and it’s own bureaucracy.

My hope is that the next five years, and beyond that I will manage to grow beyond the shortcomings of the past. I want to both get around to learning and engaging with various topics (both theoretical and the opportunity to engage with programming languages I had been wanting to properly learn for a while, among others: Ada/SPARK, Erlang, Smalltalk, Julia, APL, Forth), and make use of a clearer and improve understanding of the fundamentals. As with all new years resolutions, there is a certain naivete in this desire, but I intend to make it slightly more realistic by taking a break from my “work” on and around Emacs.

Sadly this is not as easy, as I have taken up some responsibilities, that out of respect towards other users and developers I don’t just wish to drop. The most important one of these is the development of a backwards compatibility package “Compat”, that will have to be updated for the upcoming release of Emacs 29.1 (the majority of the work has been done, what is missing is to check if any major functions have been added and to ensure that upgrading will work as expected).

If anyone is interested in temporarily maintaining, or perhaps even adopting any of the following packages, I would be most grateful:

• setup, a configuration macro and a alternative to use-package
• shell-command+, a extended version of shell-command with Eshell-like abilities.
• vc-backup, a VC backend that uses backup files.
• compat, an Elisp compatibility library (back until 24.3) used by a number of prominent packages such as Magit, ERC and Consult.
• autocrypt, a implementation of the autocrypt protocol.
• clang-capf, a completion at point backend that uses Clang.
• face-shift, a package that allows shifting the hue of buffers depending on their major mode
• insert-kaomoji, a categorised collection of eastern emoticons
• package-vc, a VC based backend for package.el (in developed in core)

Most of these are “finalised” and don’t require much work, let alone have many users to worry about. Note that due to being part of Emacs, most of these require a copyright assignment to the FSF. If necessary, I am willing to relocate development to any code forge, including GitHub.

I should note that this decision has not only been made out of boredom but also because of other personal issues I’d rather not go into here. While I know that my role in the world around Emacs is nothing close to significant people like Eli Zaretskii, Stefan Monnier or Jonas Bernoulli, I nevertheless apologise in case this turns out to be an inconvenience.

I thank anyone who has read this far, and in case anyone is interested in helping out, please contact me via email.

Moving Quickly to Project Work Spaces

On Saturday in the middle of November I wrote Project Dispatch Menu with Org Mode Metadata Denote and Transient; since then I’ve further extended and refined and generalized the functionality. You can find the underlying code in my dotemacs jf-project.el package.

In this post, I want to walk through the functionality and generalizations.

Explaining the Terms

There are two major terms to discuss: project and work space.

Project

In my work flows, I create a single note for a project. I use the Denote 📖 package for managing that note and all other notes. The format I’ve chosen for my notes is Org-Mode 📖.

I indicate that a note is a project in two ways:

• Adding the :projects: to the note’s #+FILETAGS: keyword’s list.
• Adding the top level #+PROJECT_NAME: to the note.

The purpose of the #+FILETAGS: is for searching via tags. The purpose of the #+PROJECT_NAME: is to be the short-name of how I reference the project. For example “Newman Numesmatic Portal” is the title of note, but NNP is the #PROJECT_NAME:.

This #+PROJECT_NAME: is also what I use when I’m tracking my work time.

Project Work Spaces

In this context a work space is a place where I read and write material related to a project.

Examples are:

• A project board for managing the team activities of work.
• A local code repository, for code, tests, and inline documentation.
• A local project notes file, for indexing high level project information.
• A remote repository, for working on issues and pull requests.
• A timesheet, for tracking time and writing notes regarding task resolution.

This list is arbirtrary, but a common property is that each work space has two pieces of data:

Title

The name of the work space.

Path

The path to that work space; either a local file name or one on a remote server (e.g. http://takeonrules.com/path/to/project).

While working on a project I often move between those work spaces. I also bounce between projects, helping unstick folks or looking for previous reference implementations.

Implementation Example

Here is the project metadata for the The Travels of Duinhir Tailwind series:

#+TITLE:      The Travels of Duinhir Tailwind
#+DATE:       [2022-11-29 Tue 09:18]
#+FILETAGS:   :rpgs:the-one-ring:projects:
#+IDENTIFIER: 20221129T091857

#+PROJECT_NAME: The Travels of Duinhir Tailwind
#+PROJECT_PATHS: ("Notes" . "~/git/org/denote/indices/20221129T091857--the-travels-of-duinhir-tailwind__rpgs_the-one-ring.org")
#+PROJECT_PATHS: ("WWW Series" . "https://takeonrules.com/series/the-travels-of-duinhir-tailwind/")

TITLE

The name of the project.

DATE

The date I created this note.

FILETAGS

The tags associated with this note.

IDENTIFIER

The unique identifier assigned by Denote.

PROJECT_NAME

The name I say to myself when “working on” this project.

PROJECT_PATHS

Each line is a Cons cell with the `car` (e.g. first element) as the work space name and the `cdr` (e.g. last element) as the path.

Org-Mode takes these multiple PROJECT_PATHS and creates a list. Something that is super helpful for Emacs 📖’s completing-read function.

Selecting a Project and Work Space

I have bound Cmd + 2 (or s-2 in Emacs syntax) to jf/project/jump-to/project-work-space. The function does the following:

1. Prompts for the project
2. Then for the given project, prompts for the work space
3. And then opens that work space.

The prompt for project applies some “Do What I Mean” function (DWIM 📖) logic; currently the rules are as follows:

• If I am tracking time to a project use that instead of prompting.
• If the current buffer is an Org-Mode agenda file and the cursor (e.g. point) is a descendant of headline is a project, use that instead of prompting.
• If the current buffer is a file that is part of a version control project that is registered as one of the PROJECT_PATHS, use that instead of prompting.

To force the prompt and not auto-select, I can provide a prefix argument when I call jf/project/jump-to/project-work-space.

Conclusion

My hope in sharing this is to reinforce that one of the pillars of Emacs is quick navigation. In this case, I built up functionality to navigate between related concepts; leveraging metadata and my existing plain text note taking ecosystem to power the functionality.

And in sharing these details, my hope is that others can come along, glean approaches from my code, and continue to extend their tools to do what they want and need.

A month ago, I was talking with a team mate. They were sharing their note taking process. I listened and made one suggestion: to find a tool that doesn’t lock your notes within the note taking application.

I pointed them to Logseq 📖; in part because they articulated a strong need for note taking across multiple devices and I wasn’t trying to sell them on Emacs.

I hope this post demonstrates the utility of having notes that can be accessed by a “scripting” tool; thus extending the utility of those notes. Which in turn increases the “value” of the notes; creating what I hope to be a virtuous cycle.

Over a year ago, I wrote a text titled “What I’d like to see done in Emacs”. There I mentioned a few ideas and projects related Emacs that I think are worth perusing.

Some time later, one has materialised (Compat), one has been worked on in a variation (package-vc) and one more has been worked on, but I lost the patch…

Nevertheless, I’d like to propose a few more ideas along the same lines. An early “five-year-plan” so to speak. Of course I don’t (and can’t reserve) any exclusivity on tackling these projects – anyone interested pick these up gladly, with our without my cooperation.

Eww Unmangler

It is no secret that the web is a mess. The last 30 years have demonstrated that HTML hasn’t been expressive enough to satisfy the needs of a world-wide-web. CSS and Javascript didn’t suffice as a thin layer, but became the material on which the desired path was built that led us to where we are. Combine this with the economic interests of platforms centred around advertisement and the trajectory, especially the deviation from the initial intentions doesn’t appear that surprising – in retrospect.

It doesn’t come to a surprise to many that the built-in browser EWW struggles with handling anything beyond the simplest websites. For a website to work with these kinds of text-oriented web browser (links, w3m, …) a web-developer has to consciously restrict their technologies and attempt to design a site that can be used with simpler tools. This is not a given.

To make the web work with EWW, or at least some segment of the web usable, it appears that manual intervention is necessary. What I have been thinking about it a tool that hooks into EWW, and depending on the site applies transformations to make the site readable: Reformatting headers, removing unnecessary elements, making the page more readable.

This requires a database of popular websites (For programmers the focus would initially lie in pages like StackOverflow, Reddit, Quora, GitHub, GitLab, …), and ideally a DOM-manipulation language to make it more maintainable. This could make use of eww-after-render-hook, but I fear that this would be too late. Instead it might be necessary to advise eww-display-html (or in the future extend EWW to make these kinds of manipulations easier).

I have been working on an initial sketch for a package like this, but am not satisfied with what I have written up until now. As mentioned above, the main problem is not technical but in finding an elegant way to express the problem.

ELPA Submission Helper

Sharing and publishing packages should be easier than it is now. Of course you can just upload somewhere, but then any interested user would have to fetch, install check for dependencies and look out for updates it manually. There is a reason why package managers like package.el are popular.

Sadly it is not as convenient on the other end. Sure, contributing a package to GNU or NonGNU ELPA just requires sending an email to emacs-devel@gnu.org, but there remain a number of implicit conventions that new contributors may be confused by. In my own experience, these include but are not restricted to:

• Adding a package to GNU ELPA requires signing the FSF copyright assignment.
• It is best to provide a URL to a Git repository.
• Running the byte-compiler and checkdoc can detect common mistakes.
• Adding an .elpaignore file can be used to remove files that don’t have to be distributed to users
• Packages shouldn’t have hard dependencies on non-free software

I believe it should be possible to provide a little package that checks and interactively/mechanically prompts the user most of these questions, resulting in a message that can be simply sent out.

This could include preparing a Git repository and suggesting a Git forge like Codeberg or Sourcehut. After explaining the difference between NonGNU and GNU ELPA, a request to sign the CA could be prepared as well, if the maintainer chooses to distribute their package as part of GNU ELPA, which is part of Emacs. It might even make sense to clone emacs/elpa.git or emacs/nongnu.git and directly prepare a patch.

If package-lint is added to NonGNU ELPA, then that could also be integrated into the process.

Another advantage of this approach is that the message could be generated with some special header information that would allow a CI-like process to detect the message and run a few automatised tests on some recent versions of Emacs, to be shared on the mailing list.

I don’t have any code for this idea yet, but preparing a preliminary version shouldn’t be that difficult, if there is interest in a little elpa-helper.

Distributed Content-Addressable Elisp Pastebin

I have a lot of small utility functions that I wrote-up once, and never changed since. One of the most frequent commands I use is this:

(defun site/kill-region-or-word ()
  "Call `kill-region' if there is an active region.
Otherwise kill the last word, just like in Unix."
  (interactive)
  (call-interactively (if (region-active-p)
                          #'kill-region
                        #'backward-kill-word)))

The chance is slim that this will ever require changing. This is finished code, and will work for as long as all the function it uses work.

If I wanted to share this snippet with someone else, I don’t think that creating a package and submitting it to an archive would be the right approach. I am not fond the idea of packages that just collect unrelated functionality like crux or consult. Sending them my code directly works just well enough for 1:1 situations.

What I really want is some pastebin service dedicated to (Emacs-)Lisp code. Ideally content-addressable and distributed. Perhaps this could be based on Torrents, perhaps IPFS, or something else entirely. Maybe this could also use CRDT as a basis?

Perhaps an example demonstrating can clarify my idea. Assuming some name, say dcap (distributed content-addressed pastebin), I could define a little function in my own configuration as follows

(dcap-defun kill-region-or-word ()
  "Call `kill-region' if there is an active region.
Otherwise kill the last word, just like in Unix."
  (interactive)
  (call-interactively (if (region-active-p)
                          #'kill-region
                        #'backward-kill-word)))

Basically, the same as above, with the minor difference that I used a macro called dcap-defun instead of defun. This would define a function for my own use, and declare a public snippet. The snippet would then be addressed using a hashsum, say 6465c9e5c3426b66a9fa45663946884faebc80db3260c55192d1cd4322472450.

On the other end, someone might decide to use this command and include it on their end. They might write something like

(defalias 'kill-word-or-region
  (dcap-fetch-func "6465c9e5c3426b66a9fa45663946884faebc80db3260c55192d1cd4322472450"))

Note that the name used here is not the same as the one I used. What dcap-fetch-func does is retrieve a definition from the network (say using a more generic primitive like dcap-fetch) or use a cached copy, and ensure the return value is a function.

As a hashsum like this can be inconvenient, having alias lists could be useful. Each such list could be designated by a URI that contains an association of human-readable names to hashsums.

$ curl https://some-website.net/path/to/my-alias.file
((kill-region-or-word "6465c9e5c3426b66a9fa45663946884faebc80db3260c55192d1cd4322472450)
 ;; ...
)

If configured, you could then do the same as above using a more sensible name (or a more convenient macro):

(defalias 'kill-word-or-region (dcap-fetch-func 'kill-region-or-word))

Implementing this is primarily a technical issue. Points that have to be considered are:

• How to ensure a definition is shared among the network and doesn’t get lost,
• How to prevent the network from being flooded with spam or illegal content. Even if the content is restricted to s-expressions only, you could host anything using strings,
• Figure out if and how dependencies between snippets can be established. Is any other meta-data necessary (Date, Author, etc.)?

It might be that a distributed system introduces too much complexity? It is probable that the network couldn’t just rely on idle Emacs instances, and a stand-alone node implementation would have to be implemented.

The important thing is: Reducing the overhead in sharing small improvements is one of the things I admired about the Emacs Commune. Packages usually imply there is a long-term project, that might grow over time. Copying code verbatim can be a nuisance and is not always reliable. I believe there is a niche between the two that can be satisfied.

Esoterical Text Manupulation Language

Another far-fetched idea is a little programming language for text manipulation. The twist is that I’d like to combine various features from different paradigms and programming languages.

My motivation stems from an appreciation but persistent scepticism regarding modal editing. I do believe that an expressive language for manipulating text is of use, but I don’t think that vi’s approach – throw you into “normal mode” at first, and have “insert mode” be a something you request – is ideal. I know that evil-mode, an emulator mode for people who have previously been using “Vim” (vi’s little brother), has the option of inverting this by default, but I remain unsatisfied. Other modal editing systems like Objed might be interesting if developed further, but I have been thinking if an entirely different approach could be viable instead.

So how about this: A programming language that has buffer ranges as a primitive data type, and treats these as mutable strings. We borrow the stack paradigm from Forth, implicit mapping of function over lists from APL, and intuitive regular expressions from AWK. This would allow us to express an intention like

Take all empty substrings at the beginning of each line and append a constant string.

There are many ways we could write this. Say we want to be verbose, and type out every intention word for word:

/^/ match-all "foo" append

we can imagine the stack being manipulated by each command as follows:

TOS                 ;; We start with an empty stack (TOS: Top of Stack)

TOS
/^/                 ;; A regular expression matching the beginning of a line

TOS
[(0;0) (41;41) ...] ;; A list of buffer intervals that match /^/

TOS
"foo"               ;; A constant string
[(0;0) (41;41) ...] 

TOS
[(0;3) (41;44) ...] ;; The intervals have been modified

This is fairly trivial, but how about an idea like

Match all lines that include of “bar”, “baar”, “baaar”, … and reverse their order of occurrence.

This time let us assume a terse syntax,

/ba+r/ ml lr

Again, we begin with a regular expression, request all intervals of the lines that match it and then reverse the list – which has an effect on the buffer. We press enter and the program is executed. This could involve a special interpreter or it could be compiled into Emacs Lisp.

One more feature I would like to see is strong typing – specifically interactive and immediate strong typing. While we are at it why not throw dependent typing into the mix? Let us consider an example to illustrate my point. Imagine the following

Replace each instance of “foo” with a number in increasing order of their occurrence.

This time we use a single unicode character for each command and assume an appropriate input-method is provided.

foo×↑ρι%s→

This time regular expressions (foo) and strings (%s) aren’t quoted at all. They are distinguished by being regular ASCII characters, so adding quotation is optional. Next we…

• ×: Select all regions that match a regular expression, just like match-all in the first example,
• ↑: Duplicate the top-of-stack (type ∀ α . α → α),
• ρ: Return the length of the list (type ∀ α . [α] → ℕ),
• ι: Generate a list of integers in rising order (type ℕ → [ℕ]),
• →: Replace the region (or in our case list of regions) with the result of applying the format string %s using the in–between argument (type (fmt : 𝕊) → (formatted fmt) → ⅈ, where ⅈ is a region and formatted is a function that returns a type for a format string).

Ideally this should fail and ding right after typing →, because the values on the stack are a list of integers, and not strings, before anything is even executed. Replace the %s with a %d and the program types. It can now be executed. While this is going on, and since the typing is interactive, the active buffer intervals and their replacements can be visualised on the active portion of the window.

(A different question is if you actually want this degree of strictness in a convenience language…)

The main issue here will be figuring out a good vocabulary (which will probably have to be user-extensible) and a flexible syntax to accommodate its needs.

A User Compat Library

Last year I shared the idea of working on a Forwards-Compatibility library for Emacs Lisp, and it has since not only been implemented but also published. It allows versions of Emacs going back to 24.3 (released 2013) to make use of a number of newer functions and macros. I am currently working on preparing support for Emacs 29.1, and hope to release it soon.

One restriction I drew when starting the project was that it won’t include any user-facing code. Any function that is also a command would only be usable as a function. The development branch for Emacs 29 intentionally leaves out the setopt macro. This is because Compat is a package that is rarely installed manually. Instead it is added as a dependency. And as dependencies are, they might appear or disappear, depending on what packages are installed and how clean you keep your package list.

(This argument is actually not that solid, because Emacs intentionally doesn’t draw a line between developers and users. If Compat is installed on Emacs 24.3, you could be using and-let* in your personal code in init.el and suddenly be confused if the dependency is removed.)

The idea here is simple: Provide a package with these missing definitions (commands and user-facing macros), that is supposed to be explicitly installed by the user.

There is not that much more to this idea, just a nice thing I think some people would appreciate. Being a package people would consciously installed, it could risk being more invasive and opinionated, e.g. by (pseudo-)depending on other packages in ELPA such as project, xref, etc. to ensure the newest versions are installed.

I am curious to hear if anyone things if these ideas have any merit. It would be great if someone were interested in collaborating on developing or even implementing these projects. Right now, I am under the impression that I am reaching a limit as to how much time I want to invest into a hobby like Emacs development. I am a full-time student (and part-time TA) after all. This means I’ll be thinking twice before starting any new Elisp project, as I always have other ideas I would like to work on as well.

Creating a Means of Quickly Navigating Amongst a Projects Important “Pages”

At Software Services by Scientist.com I work on several different projects. Ideally, one project at a time, but within a week I might move between two or three. Note taking procedures help me re-orient to a new project.

On Wednesday I spent some time reflecting on the common jumping off points for a project:

Local Source Code

the local repository of code for the project; thus far each project has one repository.

Remote Source Code

the remote repository where I interact with issues and pull requests.

Remote Project Board

the page that has the current project tasks and their swimlanes.

Agenda/Timesheet

the place where I track my local time and write notes.

Local Project Note

the place where I track important links or information regarding the project.

When I’m working on the project, I’m often navigating between those five points. Since I work in Emacs I figured I’d write up some code.

First, I thought about the data. Where should I store this information? Looking at the above list, the best candidate was the Local Project Note; a note written in Org-Mode and I use Denote to help me manage this kind of note.

For each project document I added the following keywords (e.g. those that can be found by the org-collect-keywords function):

#+PROJECT_NAME:

By convention, this is the short-name that I use for my timesheet and task management. (See Org Mode Capture Templates and Time Tracking for more details.)

#+PROJECT_PATH_TO_CODE:

The file path to the code on my machine.

#+PROJECT_PATH_TO_REMOTE:

The URL of the remote repository.

#+PROJECT_PATH_TO_BOARD:

The URL of the remote project board.

The Helper Functions

I wanted a common mechanism for selecting the project. I wrote the following function:

(cl-defun jf/project/list-projects (&key (project ".+")
					 (directory org-directory))
  "Return a list of `cons' that match the given PROJECT.

The `car' of the `cons' is the project (e.g. \"Take on Rules\").
The `cdr' is the fully qualified path to that projects notes file.

The DIRECTORY defaults to `org-directory' but you can specify otherwise."
  (mapcar (lambda (line)
	    (let* ((slugs (s-split ":" line))
'		   (proj (s-trim (car (cdr slugs))))
		   (filename (file-truename (s-trim (car slugs)))))
	      (cons proj filename)))
	  (split-string-and-unquote
	   (shell-command-to-string
	    (concat
	     "rg \"^#\\+PROJECT_NAME: +(" project ") *$\" " directory
	     " --only-matching --no-ignore-vcs --with-filename -r '$1' "
	     "| tr '\n' '@'"))
	   "@")))

It searches through my org-directory for the given project; by default that project is a fragment of a regular expression. That regular expression is “any and all characters.” I can use the above function as a parameter for completing-read.

I also want to set my default project. For this, I used Transient’s transient-define-suffix function. Below is jf/project/transient-current-project, a function I use to manage and display the jf/project/current-project variable.

(defvar jf/project/current-project
  nil
  "The current contextual project.")

(transient-define-suffix jf/project/transient-current-project (project)
  "Select FILES to use as source for work desk."
  :description '(lambda ()
		  (concat
		   "Current Project:"
		   (propertize
		    (format "%s" jf/project/current-project)
		    'face 'transient-argument)))
  (interactive (list (completing-read "Project: "
				      (jf/project/lis't-projects))))
  (setq jf/project/current-project project))

I also recognized that I might want to auto-magically select a project. So I wrote up the basic jf/project/find-dwim:

(defun jf/project/find-dwim ()
  "Find the current project."
  (completing-read "Project: " (jf/project/list-projects)))

The above function could look at the current clock in Org Mode and determine the associated project. Or, if I’m in a repository look to see what project it is associated with. Or whatever other mechanisms. For now, it prompts for me to pick a project.

The Interactive Functions

With the above “plumbing” I wrote five functions:

• jf/project/jump-to-agenda
• jf/project/jump-to-board
• jf/project/jump-to-code
• jf/project/jump-to-notes
• jf/project/jump-to-remote

The jf/project/jump-to-agenda function is a bit different, it tries to jump to today’s agenda item for the project.

(cl-defun jf/project/jump-to-agenda (&optional project
				     &key
				     (tag "project")
				     (within_headline
				      (format-time-string "%Y-%m-%d %A")))
  "Jump to the agenda for the given PROJECT."
  (interactive)
  (let ((the-project (or project (jf/project/find-dwim))))
    (with-current-buffer (find-file jf/pri
mary-agenda-filename-for-machine)
      (let ((start (org-element-map (org-element-parse-buffer)
		       'headline
		     ;; Finds the begin position of:
		     ;; - a level 4 headline
		     ;; - that is tagged as a :project:
		     ;; - is titled as the given project
		     ;; - and is within the given headline
		     (lambda (hl)
		       (and (=(org-element-property :level hl) 4)
			    ;; I can't use the :title attribute as it
			    ;; is a more complicated structure; this
			    ;; gets me the raw string.
			    (string= the-project
				     (plist-get (cadr hl) :raw-value))
			    (member tag
				    (org-element-property :tags hl))
			    ;; The element must have an ancestor with
			    ;; a headline of today
			    (string= within_headline
				     (plist-get
				      ;; I want the raw title, no
				      ;; styling nor tags
				      (cadr
				       (car
					(org-element-lineage hl)))
				      :raw-value))
			    (org-element-property :begin hl)))
		     nil t)))
	(goto-char start)
	(pulsar-pulse-line)))))

The jf/project/jump-to-board function assumes a remote URL.

(cl-defun jf/project/jump-to-board (&optional
				    project
				    &key
				    (keyword "PROJECT_PATH_TO_BOARD"))
  "Jump to the given PROJECT's project board."
  (interactive)
  (let* ((the-project (or project (jf/project/find-dwim)))
	 (filename (cdar (jf/project/list-projects :project the-project))))
    (with-current-buffer (find-file-noselect filename)

      (let ((url (cadar (org-collect-keywords (list keyword)))))
	(eww-browse-with-external-browser url)))))

The jf/project/jump-to-board function assumes a directory on my local machine. The code is similar to the jf/project/jump-to-board.

(cl-defun jf/project/jump-to-code (&optional
				   project
				   &key
				   (keyword "PROJECT_PATH_TO_CODE"))
    "Jump to the given PROJECT's source code."
    (interactive)
    (let* ((the-project (or project (jf/project/find-dwim)))
           (filename (cdar (jf/project/list-projects :project the-project))))
      (with-current-buffer (find-file-noselect filename)
        (let ((filename (file-truename (cadar
					(org-collect-keywords
					 (list keyword))))))
          (if (f-dir-p filename)
              (dired filename)
            (find-file filename))))))

The jf/project/jump-to-notes prompts for the project and then finds the filename.

(cl-defun jf/project/jump-to-notes (&optional project)
  "Jump to the given PROJECT's notes file.

Determine the PROJECT by querying `jf/project/list-projects'."
  (interactive)
  (let* ((the-project (or project (jf/project/find-dwim)))
	 (filename (cdar (jf/project/list-projects :project the-project))))
    (find-file filename)))

Akin to the jf/project/jump-to-board, the jf/project/jump-to-remote opens a remote URL.

(cl-defun jf/project/jump-to-remote (&optional
				     project
				     &key
				     (keyword "PROJECT_PATH_TO_REMOTE"))
  "Jump to the given PROJECT's remote."
  (interactive)
  (let* ((the-project (or project (jf/project/find-dwim)))
	 (filename (cdar (jf/project/list-projects :project the-project))))
    (with-current-buffer (find-file-noselect filename)
      (let ((url (cadar (org-collect-keywords (list keyword)))))
	(eww-browse-with-external-browser url)))))

The Menu

Using Transient I define a menu for my projects. Lower case is for dispatching to the current project. Upper case prompts for the project then dispatches.

(transient-define-prefix jf/project/menu ()
  "My Project menu."
  ["Projects"
   ["Current project"
    ("a" "Agenda…" (lambda () (interactive)
		     (jf/project/jump-to-agenda jf/project/current-project)))
    ("b" "Board…" (lambda () (interactive)
		    (jf/project/jump-to-board jf/project/current-project)))
    ("c" "Code…" (lambda () (interactive)
		   (jf/project/jump-to-code jf/project/current-project)))
    ("n" "Notes…" (lambda () (interactive)
		    (jf/project/jump-to-notes jf/project/current-project)))
    ("r" "Remote…" (lambda () (interactive)
		     (jf/project/jump-to-remote jf/project/current-project)))
    ("." jf/project/transient-current-project :transient t)]
 '  ["Other projects"
    ("A" "Agenda…" jf/project/jump-to-agenda)
    ("B" "Board…" jf/project/jump-to-board)
    ("C" "Code…" jf/project/jump-to-code)
    ("N" "Notes…" jf/project/jump-to-notes)
    ("R" "Notes…" jf/project/jump-to-remote)]
   ])

Conclusion

During my day I spend a lot of time writing and reading; and for large chunks of time those are all related to a single project. Each project has predictable “places” where I will read and write.

The above functions help me both document the relationship of those predictable “places” and automate my navigation to those different tools. In all things Emacs remains my homebase; it is where I can go to re-orient.

My jf-project.el document has the above code and any further updates, bug fixes, etc to the above now static code.

Over on /r/emacs the community has been answering What are your favorite packages for improving vanilla emacs text editing?

Such a great thread; folks sharing their favorite packages.

Personally I learned about symbol-overlay and am giving it a spin.

Delving into org-mode and org-element-map

Yesterday evening I spent some time hacking on Emacs. What follows builds from Org Mode Capture Templates and Time Tracking.

My intention in sharing this post is to add to the corpus of examples for Org Element API; to share working with the constraints of org-element-at-point; namely that the element at point does load content of that element. To do so, you must parse the buffer.

Over the last 3 months I’ve been using Org-Mode’s Clocking Commands. I have settled on the following structure:

• The first three headline level’s come from the datetree target; levels for year, month and day if you will.
• The fourth level is for projects; these are for clients and we bill at the project level.
• The fifth heading is a task; I track my time on the tasks.

Headlines deeper than level 5 are for organizational purposes.

Below is an example of my structure.

* 2022
** 2022-10 October
*** 2022-10-23 Sunday
**** Samvera :project:
***** Review pull requests on samvera/hyrax :task:

The Problem Statement

My logbook entries track time to the task level. I don’t always complete a task within a single day. Which means I need to carry forward a task from one day to another.

When I carry forward that task, I want all of the prior context except the logbook. If I bring forward the logbook, this messes up my time reporting process.

Code

The following function moves point to the task level. It recursively walks up to a level 5 headline.

(defun jf/org-agenda-task-at-point ()
  "Find the `org-mode' task at point."
  (let ((element (org-element-at-point)))
    (if (eq 'headline (org-element-type element))
	(pcase (org-element-property :level element)
	  (1 (error "Selected element is a year"))
	  (2 (error "Selected element is a month"))
	  (3 (error "Selected element is a day"))
	  (4 (error "Selected element is a project"))
	  (5 (progn (message "%s" element) element))
	  (_ (progn (org-up-heading-safe) (jf/org-task-at-point))))
      (progn
	(org-back-to-heading)
	(jf/org-task-at-point)))))

Below, the jf/org-agenda-get-day-and-project-and-task-at-point function retrieves the task and it’s associated project and day.

(defun jf/org-agenda-get-day-and-project-and-task-at-point ()
  "Return a plist of :day, :project, and :task for element at point."
  (let* ((task (jf/org-agenda-task-at-point))
	 (project (progn
		    (org-up-heading-safe)
		    (org-element-at-point)))
	 (day (progn
		(org-up-heading-safe)
		(org-element-at-point))))
    (list :project project :task task :day day)))

Now we get to the interactive function jf/org-agenda-carry-forward-task; this does the in buffer adjustments to carry the text forward.

(cl-defun jf/org-agenda-carry-forward-task ()
  "Carry an `org-mode' task node forward."
  (interactive)
  (save-excursion
    (let* ((day-project-task (jf/org-agenda-get-day-and-project-and-task-at-point))
	   (from-project (plist-get day-project-task :project))
	   (from-task (plist-get day-project-task :task)))

      ;; Narrowing the region to perform quicker queries on the element
      (narrow-to-region (org-element-property :begin from-task)
			(org-element-property :end from-task))

      ;; Grab each section for the from-task and convert that into text.
      ;;
      ;; Yes we have the from-task, however, we haven't parsed that entity.
      ;; Without parsing that element, the `org-element-contents' returns nil.
      (let ((content (s-join "\n" (org-element-map (org-element-parse-buffer) 'section
				    (lambda (section)
				      (mapconcat
				       (lambda (element)
					 (pcase (org-element-type element)
					   ;; I want to skip my time entries
					   ('drawer nil)
					   (_ (buffer-substring-no-properties
					       (org-element-property :begin element)
					       (org-element-property :end element)))))
				       (org-element-contents section)
				       "\n"))))))

	;; Capture the following to the “Day with plain entry”.  Because it’s a
	;; plain node, I’m adding the headline level.
	(org-capture-string (format "%s %s :%s:\n\n%s %s %s :%s:\n%s"
				    (s-repeat (org-element-property :level from-project) "*")
				    (org-element-property :title from-project)
				    (s-join ":" (org-element-property :tags from-project))

				    (s-repeat (org-element-property :level from-task) "*")
				    (org-element-property :todo-keyword from-task)
				    (org-element-property :title from-task)
				    (s-join ":" (org-element-property :tags from-task))
				    content)
			    "d")

	;; Widen what we once narrowed
	(widen))
      ;; Now that we've added the content, let's tidy up the from-task.
      (goto-char (org-element-property :begin from-task))
      ;; Prompt for the todo state of the original task.
      (call-interactively 'org-todo))))

And below is one of the entries I added to org-capture-templates. Note, I’m using the plain type.

("d" "Day with plain entry"
 plain (file+olp+datetree jf/primary-agenda-filename-for-machine)
 "%i"
 :empty-lines 1
 :time-prompt t
 :immediate-finish t)

Conclusion

My hope is that this blog post and associated code can add to the shared examples of how to work with the Org Element API.

Going forward, I might look to amend the previous task to indicate that I carried the task forward. I might also add a line item saying, task carried forward from the date.

As with all things Emacs, once you begin you start seeing all kinds of options unfold.

Going down the Blogosphere Rabbit Hole to Find Further Inspiration

Today while waiting on others during an errand, I started reading previous posts from Irreal. I picked a month, June 2018, and scanned the posts. I chose Howard Abrams on Capturing Data to the Current Task; which pointed me to Capturing Content for Emacs.

I read the posts with a growing realization that I had been doing quite a lot of this during my recent work.

I often jump between several different code repositories, looking through chunks of code, and then synthesizing that work.

Functions Added to My Emacs Configuration

I set about to adopt Howard Abrams’s approach. The following functions help me gather information for later synthesis; they are now part of my Emacs Configuration.

The following code establishes the org-capture-template for writing to the currently clocked item. See Capture templates (The Org Manual) for more details.

(add-to-list 'org-capture-templates
	     `("c" "Contents to Current Clocked Task"
	       plain (clock)
	       "%i%?"
	       :empty-lines 1))

Mapped to s-8 (Cmd + (8 on my machine), the jf/capture-region-contents-with-metadata function described below provides two useful options for interacting with the current region: Capture to the clocking region or copy the text.

(bind-key "s-8" 'jf/capture-region-contents-with-metadata)
(defun jf/capture-region-contents-with-metadata (start end parg)
  "Write selected text between START and END to currently clocked `org-mode' entry.

With PARG kill the content instead."
  (interactive "r\nP")
  (let ((text (jf/region-contents-get-with-metadata start end)))
    (if (car parg)
	(kill-new text)
      (org-capture-string (concat "-----\n" text) "c"))))

In jf/region-contents-get-with-metadata we get to the chunky logic:

• Grab the filename.
• Determine the current mode.
• Conditionally grab a remote link.
• Create an Org-Mode link to the source.
• Copy the source inside of a block.

(defun jf/region-contents-get-with-metadata (start end)
      "Get the region contents between START and END and return an `org-mode' formatted string."
      (require 'magit)
      (require 'git-link)
      (let* ((file-name (buffer-file-name (current-buffer)))
	     (org-src-mode (replace-regexp-in-string
			    "-mode"
			    ""
			    (format "%s" major-mode)))
	     (func-name (which-function))
	     (type (if (derived-mode-p 'prog-mode) "SRC" "EXAMPLE"))
	     (code-snippet (buffer-substring-no-properties start end))
	     (file-base (file-name-nondirectory file-name))
	     (line-number (line-number-at-pos (region-beginning)))
	     (remote-link (when (magit-list-remotes)
			    (progn
			      (call-interactively 'git-link)
			      (car kill-ring))))
	     (initial-txt (if (null func-name)
			      (format "From [[file:%s::%s][%s]]:"
				      file-name
				      line-number
				      file-base)
			    (format "From ~%s~ (in [[file:%s::%s][%s]]):"
				    func-name
				    file-name
				    line-number
				    file-base))))
	(format (concat "\n- Local :: %s"
			(when remote-link (format "\n- Remote :: %s" remote-link))
			"\n\n#+BEGIN_%s %s"
			"\n%s"
			"\n#+END_%s\n")
		initial-txt
		type
		org-src-mode
		code-snippet
		type)))

Conclusion

This builds on Revisiting the Principles of My Text Editor. In both reading older blog posts and looking to other folks for inspiration, I learned a bit more about my editor:

magit-list-remotes

If the file is part of a git project with remotes, this function returns the list of those remotes. Otherwise it returns nil.

which-function

Returns the current “function” name based on point. In the current function in an Org-Mode document is the parent node in the outline hierarchy. In other words, some powerful introspection.

Capturing to a clock

I had read the documentation on capture templates but skimmed over this functionality.

The %i capture directive

The active region is the default value of %i.

Calling org-capture-string within a function

In calling org-capture-string, I can pass arbitrary text as the %i value of the template. This allows for me to inject the richer metadata.

line-number-at-pos

Given a position in the buffer, calculate the line number. I knew this had to exist, I just hadn’t gone looking until today.

All told, a nice evening activity.

My Text Editor Must Facilitate My Engagement with the Ever Evolving Digital Aspect of My Life

On Tuesday I presented on Org-Mode and Emacs to the developers at Scientist.com. Below is the introduction to that presentation:

There’s something about the 1970s; two of my favorite technologies come from that decade: Emacs and Structured Query Language (SQL 📖). And also my friend Dungeons and Dragons. The four of us have seen many technology paradigms come and overstay their welcome, ultimately creating barriers to ownership, adoption, and maintainability.

Today I want to delve into one of those technologies; my beloved Emacs. In 2020 I moved from Atom text editor (Atom 📖) to Emacs; and I have found what I hope to be my final text editor.

In preparing for this Show and Tell, I chose to focus not on projects nor the production code I write, but instead on the tools I use.

Let’s invoke Stephen Covey: “Habit 7: Sharpen the Saw is the principle of renewal. It is the habit that makes all the others possible.”

Emacs is my saw and quickly became the linchpin of my workflow.

During my work day I jump from project to project; concept to concept. I have built up a workflow and tools to help me navigate this text-based life.

During that presentation I talked and walked through my use of Org-Mode. I also mentioned both my text editor history, referencing writing up Principles of My Text Editor when I last looked to switch.

The history I mentioned regarding text editors was:

JEdit

My first editor for web development.

Textmate

TextMate roared onto the scene alongside Rails; it made sense to adopt.

Sublime Text

Sublime Text had solved the slowness of Textmate’s search.

Atom

I wanted an open-source editor that I could extend.

Vim

I saw the writing on the wall for Atom and choose to explore this along side Visual Studio Code (VS Code 📖) and Emacs.

VS Code

The Vim paradigm didn’t quite work for me.

Emacs

I abhorred the Git commit message interface of VS Code, and recalled a gentle nudge almost 15 years prior.

One of the attendees wondered the following: “I’m a long-time user of TextMate and perhaps I should be looking elsewhere. However, with any move to a new tool there’s a time investment required to get situated. Can you talk about how you can make that decision to jump?” This was paraphrased from memory. My response was along the lines of “write up the must haves of your workflow; the functional needs. Then if and when you practice moving to a new editor, practice those must haves.”

I went on to say “When I switched to Emacs, I jumped into Spacemacs. But after a bit, I decided to start from the beginning. I removed Spacemacs and worked my way through the tutorial. Then I just started writing. And when I wanted to do something I previously did, I’d stop and seek out a package, install it, and experiment with it.”

That conversation sparked my desire to revisit the principles of my text editor. And I’ve settled on the following:

My text editor must facilitate my engagement with the ever evolving digital aspect of my life.

My initial thoughts as I pondered this revistation was that it should help me organize my digital life. But organize implies a consumptive relationship; things come at me and I need to file them into buckets.

And the “ever evolving digital aspect of my life” phrase reminds me to look at what I’ve done, am doing, and might do.

The connecting phrase “my engagement” reminds me that I should actively tend my digital garden. Give attention to the tools I’m using; prune packages I’m no longer using, think about a different way, practice and explore the problem space that is my editor experience (not the problem space I solve with my editor).

The Ever Evolving Digital Aspect of My Life

Let’s look at my digital life; my resume (and this blog) provide many clues.

Prior to 2005 my work was in proprietary ecosystems; the company prescribed my text tools. I wrote code during the day but at night my life was family time and analogue games.

From 2005 forward, I began writing code using open-source 📖 languages and frameworks. This opened the door to picking my tools. And up until I started blogging in 2011, most of my digital writing was email and code.

The explosive moment of evolution was choosing to migrate from a hosted Wordpress to Jekyll in 2018. You can read about that migration in Welcome to My New Take on Rules. I have since moved from Jekyll to Hugo. In that moment I moved from writing blog posts in a textarea on a web-page to writing posts in the tool I used for writing software; which at the time was Atom.

Looking at my Posts by Year page, it’s easy to see that after this migration I was writing more than I had in most years prior. And if you dig further my switch to Emacs in 2020 was another accelerant in my writing.

For my experience, investing time in my code editor is also investing time in my note taking and general writing tool.

Paying Attention to the Journey

A constant throughout this evolution has been introspection; paying attention not just to what I was doing but how I was doing it.

Examples of that include:

• Moving from word to word; I’ve long used OPT plus arrow keys to navigate.
• Jumping from test file to “production” file; TextMate introduced that for Rails.
• Opening new files by text prompt; Again TextMate’s CMD + t started me down this path.

In other words, these text editing tools have helped me move quickly between and within files.

And these editors of ours also have the concept of snippets; a sub-system that you can select a snippet, perhaps fill out some information via a prompt and then write the snippet’s boilerplate with the prompts filled in.

In my wanderings, I’ve seen so very many tools that allow you to import TextMate’s snippets to use in their system; think on that for a moment.

Textmate introduced a syntax that other editors not only drew inspiration from but choose to repeat it’s syntax. The formalization of the computational description of snippets, paired with their portability (e.g. they were simply text files on your machine) helped create the conditions for a plugin ecosystem.

You can see that through line in Org-Mode. First starting in Emacs and now Org-Mode’s syntax as a spec with an org-parser and folks writing Org-Mode files outside of Emacs.

Intuitions Once Heeded Now Named

Earlier I mentioned my issues with VS Code and Rubymine; and sometimes to know what you want you must identify what you don’t want.

Interface of VS Code

In my brief exploration of VS Code, two things initially bothered me along with a third:

• Git commit message interface
• Marketplace for features
• Embrace, Extend, Extinguish

Git Commit Message Interface

The first time I went to write a Git commit message in VS Code, I was greeted with what looked like an text field; a 30 character input box. I canceled out and launched VIM to write my verbose commit message.

That experience sat with me for a bit and I got to thinking; source control and commit messages are one of our most useful tools for writing about the why of a change. They provide historical context for a moment in time. That context moves with the code-base.

And VS Code was encouraging terse messages. I started extrapolating what could that mean for the ecosystem? The past is irrelevant only the now matters. The code is more important that any additional clarification someone could provide.

That apparent philosophical decision was the deciding point to eschew VS Code. I’m certain there are plugins that could remediate the situation. But here we had a flag ship open source product that was encouraging what I consider bad behavior with a critical open source platform (e.g. Github) that they owned. I could conject and posit; but I’ll leave it here.

Marketplace for Features

I understand that I should invest in my tools. And that can mean time and money. Something about the User Interface (UI 📖) of VS Code felt as though I was one moment away from “We’ve got an app for that.” It felt as though my editor was steering ever closer to a Philip K Dick reality. I’m thinking about Do Androids Dream of Electric Sheep, and dropping nickles into the door and toaster. Which leads to the next concern.

Embrace, Extend, Extinguish

Microsoft had (and has?) a strategy of embracing open source, extending it with proprietary functionality, then asphyxiating the open source viability due to the reliance on proprietary functionality.

This may not be in play for VS Code, but the days of independent computing are under attack. The personal computer has given rise to a tremendous explosion of workers having the means of production.

Technology corporations are maneuvering towards all things hosted; this is a means of eroding that “means of production.” Host your text editor in the cloud; run you application in the cloud. To make your product with those tools you become beholden to the terms, conditions, fees, and fiat of the provider.

VS Code is a foot soldier in that maneuvering; Co-Pilot is also enlisted in this battle. As are the plethora of services you can leverage and consume to cede your means of production.

Rubymine Not Setting Right for Me

I forget when I was exploring Rubymine, but I recall thinking “I don’t want a tool for just one programming language.” Getting better at using Rubymine would likely only help me get better at Ruby; not HTML, nor Javascript, nor CSS, nor Shell scripts. Or writing SQL. Or any other programming language that I need to use.

But I now extend that framing as well as re-orient my focus. The purpose of my text editor is to facilitate the conveyance of thought; to my current and future self (and hopefully the Me myself Whitman so named); to the computer that I’m telling what to do, and to others who stumble upon digital representations of my thoughts.

Part of my job as a software developer is writing code; much more of it is writing up analysis, design documents, writing task breakdowns, and the myriad of other forms of documentation.

What if we could flip our expectations just a bit. What if the only thing that mattered was the documentation? The why mixed with the how. And the byproduct of all of that would be the code; the instructions we tell the computer.

Conclusion

I don’t envision moving away from Emacs; I’ve been able to extend the tool to meet my developing needs and explore my curiosities. Emacs fundamentally is a REPL tool. I can adjust my editor on the fly: adding, removing, and adjusting functionality. It has amazing introspective capabilities. And a vibrant community of contributors and enthusiasts.

A Literate Configuration

I wrote about Exploring the Denote Emacs Package and Migration Plan for Org-Roam Notes to Denote; this is now my configuration for using Denote. This sits orthogonal to my data migration of notes into Denote paradigm.

Configuration

This base configuration gets me started using Denote.

(use-package denote
  ;; I want to point to cutting edge development; there's already features I
  ;; want that have been added since v1.0.0
  :straight (denote :host nil :type git :repo "https://git.sr.ht/~protesilaos/denote")
  :commands (denote-directory denote-file-prompt denote--title-prompt)
  :bind ("H-f" . 'jf/denote-find-file)
  :hook (dired-mode . denote-dired-mode)
  :custom ((denote-directory (expand-file-name "denote" org-directory)
			     ;; These are the minimum viable prompts for notes
			     (denote-prompts '(title keywords))
			     ;; I love ‘org-mode format; reading ahead I'm setting this
			     (denote-file-type 'org)
			     ;; And `org-read-date' is an amazing bit of tech
			     (denote-date-prompt-denote-date-prompt-use-org-read-date t)))
  :config
  (cl-defun jf/denote-org-property-from-id (&key identifier property)
    "Given an IDENTIFIER and PROPERTY return it's value or nil.

  Return nil when:

  - is not a denote file
  - IDENTIFIER leads to a non `org-mode' file
  - PROPERTY does not exist on the file"
    (when-let ((filename (denote-get-path-by-id identifier)))
      (when (string= (file-name-extension filename) "org")
	(with-current-buffer (find-file-noselect filename)
	  (cadar (org-collect-keywords (list property)))))))

  (cl-defun jf/denote-org-properties-from-id (&key identifier properties)
    "Given an IDENTIFIER and PROPERTIES list return an a-list of values.

  Return nil when:

  - is not a denote file
  - IDENTIFIER leads to a non `org-mode' file
  - PROPERTY does not exist on the file"
    (when-let ((filename (denote-get-path-by-id identifier)))
      (when (string= (file-name-extension filename) "org")
	(with-current-buffer (find-file-noselect filename)
	  (org-collect-keywords properties)))))
  )
  ;;; Testing jf/denote-org-property-from-id
;; (message "%s" (jf/denote-org-property-from-id :identifier "20220930T215235"
;; 					      :property "ABBR"))
;;; Testing jf/denote-org-properties-from-id
;; (message "%s" (jf/denote-org-properties-from-id :identifier "20220930T215235"
;; 					      :properties '("TITLE" "ABBR")))

From this configuration, I’ll build out my interactions.

Foundational Functions for my Denote Interaction

In Denote’s documentation for Convenience commands for note creation, they locally bind the denote-prompts variable then call denote; in Maintain separate directories for notes they mention having silos of information by using .dir-locals.el to cordon off different domains. See the Domains section of this file for more infromation.

And while I might want to do that for my domains I have the use case of wanting to link to a Glossary item from other domains.

I envision the jf/denote-find-file as the anchor for my general file finding, and then using a local binding of denote-directory to provide an initial narrowing default.

It doesn’t take much to see a macro emerging for these local bindings; find all sub-directories of the directory defiend by the denote-directory variable. For each of those directories make an interactive function of the form jf/denote-find-file--domain. That interactive function would locally bind denote-directory to ”denote-directory/domain”.

Alternatively, as I create each domain’s create function, I also create the finder function.

(defun jf/denote-find-file ()
      "Find file in the current `denote-directory'."
      (interactive)
      (require 'consult-projectile)
      (require 'denote)
      (consult-projectile--file (denote-directory)))

(cl-defmacro jf/denote-create-functions-for (&key domain)
  "A macro to create functions for the given DOMAIN.

Creates:

- Wrapping function of `jf/denote-find-file' that narrows results
  to the given DOMAIN."
  (let ((defun-fn (intern (concat "jf/denote-find-file--" domain)))
	(docstring (concat "Find file in \""
			   domain
			   "\" subdirectory of `denote-directory'.")))
    `(defun ,defun-fn ()
       ,docstring
       (interactive)
       (let ((denote-directory (f-join (denote-directory) ,domain)))
	 (call-interactively #'jf/denote-find-file)))))

Domains

In Migration Plan for Org-Roam Notes to Denote I talked about data structures and starting articulating some domains.

As of <2022-10-02 Sun>, my denote finding implementation leverages consult-projectile--file. This populates the mini-buffer with entries of the following format: domain/identifier--multi-word-title_tag1_tag2.org. The domain is a subdirectory of my denote-directory.

I have the domains following:

Blog Post

Something I share with the world.

Dailies

An anchor for any time references.

Employer

More specifically, Scientist.com.

Epigraph

A quote that I found interesting.

Glossary

A term/concept I reference.

People

Similar to a glossary but for notes regarding people.

Blog Post

When I start writing a note, I am uncertain if it will be a Blog Post. However, once I publish something I think it makes sense to transfer the note into the Blog Post domain. By treating a Blog Post as a domain it will be visually chunked at the beginning of the line (e.g. the subdirectory).

Alternatively I could add the “blog-post” keyword/tag to the note. The primary benefit would be that something I post to my blog could be of another domain.

What might those other domains be?

I don’t think I need linger on this for too long, as I can easily migrate. The foundational element is the identifier; which is dynamically queried.

Dailies

While writing this document, I began envisioning replacing my Org-Mode 📖 date macro with a date Org-Mode link protocol. The benefits are:

1. Replacing the kludge of a macro with something that works easier with exports.
2. I would be creating a node that could provide a backlink.

None of this requires Denote but which builds on some of my musings; namely should I have a monthly timesheet in Denote. And the answer appears to be yes.

(jf/denote-create-functions-for :domain "dailies")

I want to continue using my timesheets as a single document; this makes both time reporting and personal timetracking easier.

Employer

There are certain employer specific notes that I keep; timesheets being a distinct one. I don’t envision a problem linking to other domains; a Scientist.com note could and would likely link to/reference a Glossary entry.

The primary advantage is that I can easily segement my git repositories for employer and not-employer.

I need a current timesheet function; this would help me jump to my time sheet and capture appropriate tasks, projects, merge requests and blockers.

I also want my org-agenda-files to include:

• personal agenda
• work agenda (on work machine)
• this month and last month’s time sheet

I’m okay with restarting Emacs 📖 each month.

(jf/denote-create-functions-for :domain "scientist")

Epigraph

As mentioned, I collect phrases and like to reference them as epigraphs in my posts.

Something in the Epigraph domain has the following properties:

AUTHOR_NAME (required)

The name of the author

AUTHOR_URL

Where can you “find” this author?

AUTHOR_KEY

The GLOSSARY_KEY for the given author

WORK_TITLE (required)

What’s the title of the work?

WORK_URL

Where can you “get” this work?

WORK_KEY

The GLOSSARY_KEY for the given work

POEM

Indicates if this is a poem (or not)

PAGE

The page in which this passage appears in the given work.

TRANSLATOR_NAME

The name of the translator

As part of my blog build scripts, I lookup the KEY properties in the Glossary and write the names and URL. With all of the changes I’ve made, I need to see if I’m still looking up the KEY properties when I build the script.

(bind-key "H-d c e" 'jf/denote-create-epigraph)
(cl-defun jf/denote-create-epigraph (&key
				     (body (read-from-minibuffer "Epigraph Text: "))
				     ;; Todo prompt for Author Name
				     (author_name (read-from-minibuffer "Author Name: "))
				     ;; Todo prompt for Work Title
				     (work_title (read-from-minibuffer "Work Title: "))
				     (nth-words 8))
  "Create an epigraph from the given BODY, AUTHOR_NAME, and WORK TITLE.

Default the note’s title to the first NTH-WORDS of the BODY."
  (interactive)
  (let* ((body-as-list (s-split-words body))
	 (title (s-join " " (if (> (length body-as-list) nth-words)
				(subseq body-as-list 0 nth-words)
			      body-as-list)))
	 (template (concat
		    "#+AUTHOR_NAME: " author_name "\n"
		    "#+AUTHOR_URL:\n"
		    "#+AUTHOR_KEY:\n"
		    "#+WORK_TITLE: " work_title "\n"
		    "#+WORK_URL:\n"
		    "#+WORK_KEY:\n"
		    "#+POEM:\n"
		    "#+PAGE:\n"
		    "#+TRANSLATOR_NAME:\n")))
    (denote title
	    nil
	    'org
	    (f-join (denote-directory) "epigraphs")
	    nil
	    template)))

(jf/denote-create-functions-for :domain "epigraphs")

Glossary

We’ll store glossary entries in the “glossary” subdirectory of denote-directory.

An entry in the glossary requires a KEY property. This KEY is used as the entry point for my blogging glossary.html shortcode.

All other properties, aside from TITLE, are optional. In my writing there are two ways I directly refer to a glossary entry, when I:

• Reference a Game
• Use an Abbreviation

I might create two or three glossary entries at a time; so the easiest approach is to include all of the properties with minimal prompting.

(bind-key "H-d c g" 'jf/denote-create-glossary-entry)
(cl-defun jf/denote-create-glossary-entry
    (&key
     (title (read-from-minibuffer "Name the Entry: "))
     (is-a-game (yes-or-no-p "Is this a game?"))
     (abbr (read-from-minibuffer "Abbreviation (empty to skip): ")))
  "Create a `denote' entry for the given TITLE and ABBR.

And if this IS-A-GAME then amend accordingly.

      NOTE: At present there is no consideration for uniqueness."
  (interactive)
  (let* ((key (downcase (denote-sluggify (if (s-present? abbr) abbr title))))
	 (template (concat "#+GLOSSARY_KEY: " key "\n"
			   "#+ABBR:" (when (s-present? abbr) (concat " " abbr)) "\n"
			   "#+CONTENT_DISCLAIMER:\n" ;; TODO: Include a prompt of existing disclaimers
'			   "#+DESCRIPTION:\n"
			   (when is-a-game "#+GAME: " key "\n")
			   "#+ITEMID:\n"
			   "#+ITEMTYPE:\n"
			   "#+MENTION_AS:\n"
			   "#+OFFER:\n"
			   "#+PLURAL_ABBR:\n"
			   "#+PLURAL_TITLE:\n"
			   "#+SAME_AS:\n"
			   "#+TAG:\n" ;; TODO: Assert uniqueness
			   "#+VERBOSE_TITLE:\n"))
	 (keywords (list)))
    ;; Add both "abbr" and the abbr to the keywords; both help in searching results
    (when (s-present? abbr)
      (progn (add-to-list 'keywords "abbr") (add-to-list 'keywords abbr)))
    (when is-a-game (add-to-list 'keywords "game"))
    (denote title
	    keywords
	    'org
	    (f-join (denote-directory) "glossary")
	    nil
	    template)))

(jf/denote-create-functions-for :domain "glossary")
;;; Testing jf/denote-org-property-from-id
;; (message "%s" (jf/denote-org-property-from-id :id "20220930T215235"
;; 					      :property "ABBR"))

This builds from On Storing Glossary Terms in Org Roam Nodes.

People

I do write notes about people I interact with. Technically I have glossary entries for people. But those entries are for folks I don’t interact with.

(jf/denote-create-functions-for :domain "people")

Custom Hyperlinks

I have two custom hyperlinks to consider:

• Abbrevations (and their Plural)
• Date entries

Abbreviations (and their Plural)

As part of my writing I use of abbreviations. I try to always provide the abbreviation’s title when I first introduce the abbrevation. For most of those abbreviations I reference something in my glossary.

When I export to my blog, I want those abbreviations to leverage what I have in my local glossary. I expand those abbreviatinos to use the ABBR-element. I do this via my glossary.html shortcode.

Below is the code that adds the abbr and abbr-plural link type into Org-Mode’s link handler; for more information checkout the documentation on Adding Hyperlink Types.

Building the Complete Functionality

First up is the functionality for completion. Given that I have both abbr and abbr-plural link schemes, I’m going to create a generic function.

The jf/org-link-complete-link-for function will pre-populate a search. In the case of abbr and abbr-plural all entries will be in the ./glossary subdirectory and have the keyword _abbr.

(cl-defun jf/org-link-complete-link-for (parg &key scheme keyword subdirectory))
      "Prompt for a SCHEME compatible `denote' with KEYWORD in the given SUBDIRECTORY.

      Returns a string of format: \"SCHEME:<id>\" where <id> is
      an `denote' identifier."
      (concat scheme
	      ":"
	      (let ((denote-directory (if subdirectory
					  (f-join (denote-directory)
						  (concat subdirectory "/"))
					(denote-directory))))
		;; This leverages a post v1.0.0 parameter of Denote
		;; See https://git.sr.ht/~protesilaos/denote/commit/c6c3fc95c66ba093a266c775f411c0c8615c14c7
		(denote-file-prompt (concat "_" keyword "*")))))

The above implementation assumes a post v1.0.0 implementation of Denote. As of October 7th this is not part of a released version but is part of the main branch.

I was preparing to send a suggestion for that feature when I noticed the change; it is always reassuring to see folks recommend functions that are identical to what you were going to suggest.

Building the Export Functionality

Next is the export functionality. There are many similarities between abbr and abbr-plural; what follows is the general function.

(cl-defun jf/denote-link-ol-link-with-property (link description format protocol
						     &key
						     property-name
						     additional-hugo-parameters
						     (use_hugo_shortcode jf/exporting-org-to-tor))
  "Export a LINK with DESCRIPTION for the given PROTOCOL and FORMAT.

      FORMAT is an Org export backend. We will discard the given
      DESCRIPTION.  PROTOCOL is ignored."
  (let* ((prop-list (jf/denote-org-properties-from-id
		     :identifier link
		     :properties (list "TITLE" property-name  "GLOSSARY_KEY")))
	 (title (alist-get "TITLE" prop-list nil nil #'string=))
	' (property (alist-get property-name prop-list nil nil #'string=))
	 (key (alist-get "GLOSSARY_KEY" prop-list property nil #'string=))
	 (cond
	  ((or (eq format 'html)
	       (eq format 'md))
	   (if use_hugo_shortcode
	       (format "\{\{< glossary key=\"%s\" %s >\}\}"
		       property
		       additional-hugo-parameters)
	     (format "<abbr title=\"%s\">%s</abbr>"
		     title
		     property))
	   (_ (format "%s (%s)"
		      title
		      property)))))))

Registering the Link Types

With the above preliminaries, here are the two parameter types and their configurations.

(org-link-set-parameters "abbr"
			 :complete (lambda (&optional parg) (jf/org-link-complete-link-for
							     parg
							     :scheme "abbr"
							     :keyword "abbr"
							     :subdirectory "glossary"))
			 :export (lambda (link description format protocol)
				   (jf/denote-link-ol-link-with-property link description format protocol
									 :property-name "ABBR"
									 :additional-hugo-parameters "abbr=\"t\""))
			 :face #'denote-faces-link
			 :follow #'denote-link-ol-follow
			     ;;; I'm unclear if/how I want to proceed with this
			 ;; :store (lambda (jf/org-link-store-link-for :scheme "abbr"))
			 )

    (org-link-set-parameters "abbr-plural"
			     :complete (lambda (&optional parg) (jf/org-link-complete-link-for
								 parg
								 :scheme "abbr-plural"
								 :keyword "abbr_plural"
								 :subdirectory "glossary"))
			     :export (lambda (link description format protocol)
				       (jf/denote-link-ol-link-with-property link description format protocol
					  :property-name "ABBR_PLURAL"
					  :additional-hugo-parameters "abbr=\"t\" plural=\"t\"")
			     :face #'denote-faces-link
			     :follow #'denote-link-ol-follow
			     ;;; I'm unclear if/how I want to proceed with this
			     ;; :store (lambda (jf/org-link-store-link-for :scheme "abbr-plural"))
			     )

Date Entries

I want to register the date scheme for Org-Mode links.

(org-link-set-parameters "date"
			 :complete #'jf/denote-link-complete-date
			 :export #'jf/denote-link-export-date
			 :face #'denote-faces-link
			 :follow #'jf/denote-link-follow-date
			 ;; :store (lambda (jf/org-link-store-link-for :scheme "abbr"))
			 )

(cl-defun jf/denote-link-complete-date (&optional parg)
  "Prompt for the given DATE.

While we are prompting for a year, month, and day; a reminder
that this is intended to be conformant with the TIME element.
But for my typical use I write these as either years; years and
months; and most often year, month, and days."
  (format "date:%s" (org-read-date)))

(cl-defun jf/denote-link-export-date (link description format protocol)
  "Export a date for the given LINK, DESCRIPTION, FORMAT, and PROTOCOL."
  (cond
   ((or (eq format 'html)
	(eq format 'md))
    (concat "<time datetime=\"" link "\">" description "</time>"))
   (_ (format "%s (%s)" descirption link))))


(cl-defun jf/denote-link-follow-date (date &optional parg)
  (message "TODO, implement link for %s" date))

Conclusion

I wrote this configuration with the intention of publishing to my blog. I have locally tested things, a bit, but have not incorporated it into my dotemacs. That is a future concern.

Laying Out a Game Plan

Building on Exploring the Denote Emacs Package, I want to layout a migration plan. For the migration to be successful, I have the following criteria:

Glossary Migration

When I pull glossary entries into Take on Rules I expect the ./data/glossary.yml to be functionally identical to it’s current state. That is to say no nodes dropped nor added and keys being similar. I would go so far as to say after the migration the pull event should result in no change to ./data/glossary.yml.

Blog Export

I am able to export an Org-Mode 📖 format Denote file for publication on Take on Rules.

Data Structures

I am able to organize and discover notes via different attributes.

Keywords and Controlled Vocabularies

I spent time establishing the tags I use for blogging; I’d want assistance in adhering to that exercise.

Other Stuff

As I write this document, I’m certain other things will emerge.

Feature Tests

These are my guiding “feature tests.” Revisiting Exploring the Denote Emacs Package, you’ll notice that I’ve already introduced feature creep. Such is the life of any project, but especially a software project.

Glossary Migration

I need to consider how I’m handling abbr: and abbr-plural: links; due to the newness of this functionality I don’t have many to consider. This does require navigating the export of Hugo shortcode fragments.

I will need to migrate the Rake 📖 task that I wrote for pulling data into my Hugo 📖 data/glossary.yml file. This should be relatively straight-forward and easier by introducing the convention of a glossary directory.

Yes, I Org-Roam 📖 I could have created a custom directory and template, but I was relying on the search function as my intermediary. In hindsight, having a specific directory for Glossary is something I’m now sensing I want.

Given that I’m uncertain I want to migrate to Denote, it makes sense to move my glossary items to a glossary directory.

Blog Export

I’ll need to revisit the Extending the Ox section of my Emacs configuration. With this document, I did a called jf/export-org-to-tor to see what happened. The main thing I observed was that the Ox-Hugo front-matter I inject into this document was placed at the top of the file.

That clobbers the Denote convention of it’s front-matter at the top of the file. The fix is somewhat straight-forward; I know what to do but not yet how I’ll go about it.

There likely exists a function to detect the entirety of a Denote document’s front-matter. Use that to find where to insert the Ox-Hugo front matter.

Another thing I noticed was that I was adding an :ID: to a properties drawer. This is not necessary as Denote has the :identifier: front-matter entry. This does, however, remind me that I’ll need to consider migrating nodes.

Data Structure

One thing I love thinking about, and all too often forget to think about, is the data structure.

And Denote, by it’s conventions, has me thinking about the data structure; in part because of it’s opinions regarding front-matter. In this case there are two data structure components to consider:

• File System Conventions
• Document Metadata

File System Conventions

As a refresher, Denote uses the file name to encode:

• Identifier
• Title
• Keywords (aka Tags)

There is further consideration about the directory structure. There is guidance on how to isolate directories (e.g. “work”, “personal”, “volunteer”) such that they are siloed from each other. See Section 3.3 “Maintain separate directories for notes” of Denote’s manual for further details on separate directories. However, directories need not be siloed. My aforementioned glossary is something I might not want siloed; after all these terms are useful for many contexts.

I do wonder how I might have three directories: glossary, employer, personal. I would want the glossary available to the employer and personal but disallow employer and personal “knowledge” of each other.

To put a name to metadata role of directory, I think of namespace or domain. Namespace feels too generic. Let’s go with domain, and lean on Webster’s 1913 definition:

The territory over which dominion or authority is exerted; the possessions of a sovereign or commonwealth, or the like. Also used figuratively.

I also like overlaying/amalgamating the definition with the Mathematical concept of domain:

The domain of a function is the set of its possible inputs, i.e., the set of input values where for which the function is defined.

With domain I have four pieces of metadata: title, identifier, keywords, and domain.

I should probably consider what are my implicit domains:

Glossary

I reference many terms.

Epigraphs

I collect passages and reference them in posts.

Blog Posts

I’ve written a lot of blog posts; should I move a note into this domain when I publish an internal note?

Do all of these warrant their own directory? Before I get carried away, I should do some preliminary exploration. As previously mentioned, the Glossary domain is a good experiment for adoption.

Document Metadata

I wrote about this quite a bit already, but will reiterate; because Denote allows writing in Org-Mode format, I have access to Org-Mode’s property syntax; though I’ll need to use a different method than I’ve previously used.

Keywords and Controlled Vocabularies

On June 4th I migrated the tags of my Blog; compressing 378 tags into 59. I checked Changelog // Take on Rules to see when I made this unceremonious switch. How to Use Tags spurred me to revisit my tagging. This migration inter-relates with my glossary migration; the glossary asserts what tags are allowed on Take on Rules.

To my knowledge, Org-Roam, does not have a concept of a controlled vocabulary for tags. And since I’m using Org-Roam as the mechanism for the initial composition of my blog, I’m butting up against tagging; namely I have to be disciplined about how I tag things.

In my quick read of the documentation, denote-infer-keywords provides the ability to limit tags to only those explicitly set in denote-known-keywords.

Further Considering and Pondering

One thing rattling around in my brain is how I’ve been using my agenda.org file. For my work at Scientist.com I use it as a TODO list and my monthly timesheet. When the month is done, I archive my timesheet.

What if I rethink things just a bit; each month get’s a journal.org file. Then I amend my org-agenda=files variable to point to the timesheet? This would mean I’m not archiving entries but instead removing them from my agenda workflow.

This is a future tangent but one that I’m pondering and considering. And none of this is really dependent on Denote; it is simply a byproduct of thinking about my organization system.

But I do like the idea of not archiving the entries. Why?

I use Git 📖 for version control of my Org-Mode files; and moving a subtree from one file to another gives me trepedations.

Conclusion

This document is about helping me think through a potential migration. And in writing this all down, I’m thinking about what it means to go through a migration.

Some Analsysis of Features

As of Today I use Org-Roam 📖 for writing and linking. Those notes can be glossary entries, blog posts, people, epigraphs, or just about anything text.

Org-Roam utilizes a SQLite database for storing and linking metadata. This is a feature that I consider a nice to have.

An emerging note taking utility is Denote; in this document I want to explore my usage of Org-Roam; what I’ve come to expect (e.g. consider a “must have”) and what I’ve come to appreciate (e.g. consider a “nice to have”). I then want to look at how to achieve that in Denote.

The foundational tooling that I want:

Org-Mode aware

Note taking that “understands” Org-Mode 📖; or more appropriately that Org-Mode tooling can understand.

Tags

Tag each note.

Quick filing

Insert a new note with minimal thought of where it goes.

Linking

Link to other notes.

Metadata

Add metadata that can power my epigraphs and glossary.

Search

Prompt for notes by searching title and tags and other metadata.

Export Links

When I link to a node and export, I want to export the world facing URL.

The above definitions are my “feature list”; I’ll reference those later.

A nice to have feature would be prompting for a file by more advanced searching. An example of that would be as follows:

• Filter on the programming tag
• Search the resulting filter set for the words Hello World

Another nice to have feature is collision detection; I don’t want two notes to have the same glossary key, nor alias. Right now I believe Org-Roam enforces unique aliases but not glossary keys.

Diving into Denote

With this quick establishment of my note-taking requirements I’m going to explore Denote.

Why explore Denote?

First, I found myself reading the Denote documentation for pleasure. It is one well-documented package; accessible and helpful at learning not just Denote but also Emacs Lisp as well.

More importantly (maybe?) is a statement I recall from a seminar with Neil Jeffries regarding the Oxford Common File Layout (OCFL 📖). The statement was along the lines of favoring Posix, the Unix file system. It is a consistent and underpinning technology that is very likely to continue as other technologies light the stage and fade away. In other words, it has attributes that are ideal for “preservation” of digital objects. Too Long; Didn't Read 📖 Favor it over any other system of preservation.

Reading Denote’s documentation on Portability, I see common consideration:

Notes are plain text and should remain portable. The way Denote writes file names, the front matter it includes in the note’s header, and the links it establishes must all be adequately usable with standard Unix tools. No need for a database or some specialised software.

Protesilaos Stavrou (aka Prot), the maintainer and originator of Denote, stewards many Emacs packages. A consistent attribute of those packages is fantastic documentation; both inline and of the “README” variety.

Denote’s documentation exemplifies quality documentation.

The Code I Load

When I first started, I had the minimal package declaration (use-package denote :straight t). As I explored I amended that basic declaration to the following code block.

(use-package denote
  :straight t
  :commands (denote-directory)
  :bind ("H-c a" . jf/denote-create-abbreviation)
  :custom ((denote-directory "~/git/org/denote")
	   ;; These are the minimum viable prompts for notes
	   (denote-prompts '(title keywords))
	   ;; I love org-mode format; reading ahead I'm setting this
	   (denote-file-type 'org)
	   ;; And `org-read-date' is an amazing bit of tech
	   (denote-date-prompt-denote-date-prompt-use-org-read-date t)))

(cl-defun jf/denote-create-abbreviation
    (&key
     (title (denote--title-prompt))
     (abbr (read-from-minibuffer "Abbreviation: ")))
  "Create a `denote' entry for the given TITLE and ABBR.

NOTE: At present there is no consideration for uniqueness."
  (interactive)
  (let ((template (concat "#+GLOSSARY_KEY: glossary---" abbr "\n"
			  "#+ABBR: " abbr "\n")))
    (denote title
	    '("glossary" "abbreviation")
	    'org
	    (f-join (denote-directory) "glossary")
	    nil
	    template)))

Exploration Notes

I arrived at the above code-block via the following exploration.

Customizing denote-directory

Prior to configuration my denote directory was ~/git/org/main/notes; I’m unclear why it chose this but it was a good guess as my org-roam-directory is ~/git/org and I have a capture template that writes to ~/git/org/main/.

With the configuration I’m partially sequestering my playground and will begin exploring.

Creating jf/denote-create-abbreviation

I had originally started exploring the denote-templates but chose to pursue a more explicit pathway which, based on my knowledge of Lisp, was straight forward and very quick.

There were a few turns that I took. An existing implementation for much of my tooling is that I’m assuming an Org-Mode properties drawer for the GLOSSARY_KEY and ABBR properties.

In Org-Roam the properties for the node go above the TITLE property; however by convention that is not how Denote is structured to work. I made some revisions.

I tested the above by creating a new Denote node for “Digital Humanities” with the abbreviation of “DH”.

Pause To Review Requirements and Reflect

At this point, I have verified checked off 4 of the 7 requirements. And reading ahead of the documentation I see that there are considerations for Linking and Exporting Links as well as searching.

• Org-Mode aware
• Tags
• Quick filing
• Linking
• Metadata
• Search
• Export Links

At this point, what I really like is the interface to creating a note. The denote function does the magic and allows for me to pass parameters that override the default methods.

Contrast with Org-Roam, where I provide the title/text and then say what the template shouldl be.

I’m also liking the ease at which I could create a function for creating glossary entries. To do that in Org-Roam via capture is certainly doable.

The thing I need to check is how moving from the properties drawer approach I’ve used in Org-Roam varies from using keywords. Most of my interactions with properties are via Org-Mode’s API.

This sounds like the next pathway to explore.

Investigating Moving from Property Drawer to Keywords

Before I get too much further, I need to verify that I can continue to get properties from my notes.

The following function verifies that I can retrieve a property for the Denote note I made.

(cl-defun jf/denote-org-property-from-id (&key id property)
  "Given an ID and PROPERTY return it's value or nil.

Return nil when:

- is not a denote file
- ID is not an `org-mode' file
- property does not exist on the file"
  (when-let ((filename (denote-get-path-by-id id)))
    (when (string= (file-name-extension filename) "org")
      (with-current-buffer (find-file-noselect filename)
	(cadar (org-collect-keywords (list property)))))))

(message "%s" (jf/denote-org-property-from-id :id "20220930T215235"
					       :property "ABBR"))

Let’s Look at Linking

For this, I’ll need another node. I now have two nodes: 20220930T221757 and 20220930T215235. Using denote-link I create a link in 20220930T221757 to 20220930T215235.

Then in 20220930T221757 I call denote-link-backlinks. The backlink buffer is an enumeration of links. Whereas in Org-Roam the backlink and reference buffer includes the surrounding context; a nice feature but not something I consider mandatory.

And below is my must haves:

• Org-Mode aware
• Tags
• Quick filing
• Linking
• Metadata
• Search
• Export Links

What About Search?

I need to further explain what I mean by this. I am accustomed to using my own jf/org-roam-find-node which wraps org-roam-find-node. I can filter by tags and title. With denote, I have to consider directory structure.

Let’s see about leveraging the Consult package.

(bind-key "H-f" 'jf/denote-find-file)
(defun jf/denote-find-file ()
  "Find file in `denote-directory'"
  (interactive)
  (require 'consult-projectile)
  (require 'denote)
  (consult-projectile--file (denote-directory)))

This provides me with the comparable functionality, but requires some reimaginging. However, courtesy of Section 5. The file-naming scheme of Denote documentation I can use the naming convention for tag and filename search.

Prefix the search with - for a tag and and _ for a word. This matches the functionality of what I have.

Conclusion

Reading and testing Denote, I have established feature parity in my functional needs.

What does that mean?

I am prepared to further pursue what it might mean to migrate my some 2800 Org-Roam notes to denote. I just completed a migration of my Hugo 📖 ./data/glossary.yml file to Org-Roam, so I know that it’s not an arduous process to migrate. Read about this data migration in On Storing Glossary Terms in Org Roam Nodes. I already identified the need to move from property drawers to properties that are positioned in the file after the Denote front-matter. I’d need to revisit the Ox-Hugo export process I’ve developed. As well as how I’m exporting and creating links.

I would also want to look at different directories. I like separating the different concerns (e.g. glossary, epigraph) and the ease at which I could set this up.

There is quite a bit more to consider regarding this migration. But it is an interesting (to me) exercise of consideration.

Hacking on Org-Mode

This post started as a place for me to perform analaysis on my existing blogging ecosystem. The goal is to consolidate where I’m “storing” information. The strategy is moving elements of my blogging ecosystem into my Personal Knowledge Management (PKM 📖) system; and then exporting those back to my blog.

Writing to Think Through the Problem

At present I store my glossary in a YML file. Table 240: Take on Rules's Glossary Entity Attributes describles the attributes.

Considerations for Migrating Glossary into Org Ecosystem

What would it look like to move the glossary into my Org ecosystem?

Let’s consider the following:

• What would be the benefit?
• What would be the storage strategy?
• How to map the YAML entry to the Node’s data structure?
• What are the ways in which Take on Rules references the glossary?

What Would be the Benefit?

The primary benefit that I see is in consolidation. Right now the benefits of the glossary are only available in my blogging ecosystem. And the “capture” process for those entries is outside of my normal capture process.

Further, in consolidation there is the process of thinking through the “problem” and designing a solution. The very thought exercise itself is enriching.

What Would Be the Storage Strategy?

I see two options: ./glossary or ./refs. The advantages of ./glossary is crisp demarkation. However I envision “promotion” or “drift” of refs into glossary items.

Decision

Store nodes in ./refs and add tag :glossary:

How to map the YAML entry to the Node’s data structure?

There is an analogy between my Epigraphs and my Glossary setup. An Epigraph has properties and it’s textual content (see Even if you decide never to write a single…).

In the case of a Glossary entry, the body of the node would be my additional notes. The node’s #+title: line would be it’s title property.

What Are the Ways in Which Take on Rules References the Glossary?

The following short-codes, part of the TakeOnRules Hugo Theme reference the glossary data:

themes/hugo-tufte/layouts/shortcodes/blockquote.html

when quoting a game, I add a purchase link to the citation.

themes/hugo-tufte/layouts/partials/header/contentDisclaimers.html

when a post has a glossary tag that has a content disclaimer, I look that up.

themes/hugo-tufte/layouts/shortcodes/glossary.html

this is where the major heavy liftin occurs; a glossary entry can be an abbreviation, mention, link, etc.

themes/hugo-tufte/layouts/_default/list.html :

themes/hugo-tufte/layouts/_default/single.html :

My assumption is that I would generate the ./data/glossary.yml file from my Org ecosystem.

To continue to leverage the ./data/glossary.yml as-is I would want to have means in my Org ecosystem to declare each of these.

Design

There are two major design considerations:

• Registering New Link Types in Org-Mode
• Data Migration

Registering New Link Types in Org-Mode

Org-Mode 📖 has different link handlers. One example is the roam: scheme; a scheme added by Org-Roam 📖.

The org-link-set-parameters exposes a means of registering new schemes.

I have two “links” that I want to add: abbr: and abbr-plural:. The goal is to use my glossary to “link” to an entry that has an abbreviation and then at the point of export expand that abbreviation.

Reading the org-link-parameters documentation there are functions I want to set for both abbr: and abbr-plural::

:complete

What are the possible entries I have that meet the criteria? (e.g. have an abbreviation?)

:export

When I export this link, what should logic should I use? (e.g. for HTML 📖, I should use an ABBR-tag, for other things the title (abbreviation) is perhaps best)

:follow

When I “click” on the link in my Org-Mode document, how should it resolve? Where should it go?

Data Migration

I want to export my ./data/glossary.yml to Org-Mode. There are some glossary entries that already have nodes in Org-Roam; for those I need to reconcile and adjust. For the others, I need to create a node for each glossary entry.

Once that is complete, I will stop adding and editing entries in ./data/glossary.yml; instead I will export them from my org repository to that file. An ideal test is that when I complete the round trip, my ./data/glossary.yml is unchanged.

I already do this for Epigraphs, so follow a similar “public” api process.

Conclusion

After a bit of time and exploration, I have enriched my understanding of the Org-Mode / Org-Roam ecosystem and further worked towards treating my blog as a “consumer” of my PKM system.

I’ve also solidified my love of cl-defun, &key parameters, and providing defaults. This helps me repurpose functions and think about their interactions. It’s a pattern I’ve also applied in my Ruby code.

An example perhaps? The following is a method signature: (cl-defun jf/org-roam-external-url-for (&key node (scheme "http"))

To call that method I write the following: (jf/org-roam-external-url-for :node the_node); where the_node is an Org-Roam node. By default I’m looking for URLs that start with the http scheme.

Instead of burying the default http in the method definition, I parameterize it and specify the default.

All of this is in service to sharpening my tools.

Sharing My Workflow and Tooling

In this post I’ll share my workflow and the Emacs functions I use to facilitate my workflow. One highlight is better understanding how to use org-capture\’s (file+function "filename" function-finding-location) target element.

Context

One of my administrative tasks for my role at Software Services by Scientist.com is time tracking. It’s been over a decade since I last tracked my working hours. In my role I’m both coding and helping get team members unstuck on their tasks. This means on a given day, I jump between 5 to 7 projects. I find it invigorating helping get folks unstuck; either listening to their approach or digging in and pulling out a myriad of debugging and triage heuristics I’ve developed. To help me with keeping track of all of my hours and I work, I have begun leveraging even more of Emacs’s Org-Mode; a framework and toolkit for note taking and time tracking (and so much more).

My Current Workflow

At the start of my day, I review my existing todo items. This helps me remember where to start.

As I work on a todo item, I record time and take notes; which involves links and capturing further general documentation. Those notes will sometimes turn into blog posts or playbook articles. As I start a new project:

• I start tracking my time.
• Write a bit about what I’m working on.
• And start taking notes.
• For tasks that I don’t complete, I mark as todo.

As I wrap up a project’s task I go back to my todo items. The org-agenda function provides a menu of options to view my time and todo items. See the documentation At the end of the month I then go through my projects and record that time. I do all of this in my org-mode agenda file.

Code Supporting My Workflow

Before I started down this path I spent a month exploring, noting, and adjusting my workflow. As the month closed, I started to see the pattern I could use to extend my existing toolkit to better help my emerging workflow.

This section goes into the technical implementation.

Here’s my org-capture-templates. There are two of them:

project

The client’s project that I’m working on.

task

The task within a project.

(setq org-capture-templates
      '(;; Needed for the first project of the day; to ensure the datetree is
	;; properly generated.
	("p" "Project"
	 entry (file+olp+datetree jf/primary-agenda-filename-for-machine)
	 "* %(jf/org-mode-project-prompt) :project:\n\n%?"
	 :empty-lines-before 1
	 :empty-lines-after 1)
	("t" "Task"
	 ;; I tried this as a node, but that created headaches.  Instead I'm
	 ;; making the assumption about project/task depth.
	 plain (file+function jf/primary-agenda-filename-for-machine jf/org-mode-find-project-node)
	 ;; The five ***** is due to the assumptive depth of the projects and tasks.
	 "***** TODO %? :task:\n\n"
	 :empty-lines-before 1
	 :empty-lines-after 1)
	))

Anywhere in Emacs I can call org-capture (e.g. C-c c in Emacs dialect).

Begin Capturing Notes for the Project

The capture for the project positions the content in the following headline tree:

• Year (e.g. 2022)
  • Month (e.g. 2022-09 September)
    • Day (e.g. 2022-09-03 Friday)
      • Project

The capture template for the project is (e.g. * %(jf/org-mode-project-prompt) :project:\n\n%?).

For the Project capture template, this:

• creates a headline
• prompts for the project
• tags the node as a :project:
• positions the cursor to begin note taking.

The following function prompts me to select an existing project or allows me to enter a new one.

(defun jf/org-mode-project-prompt ()
    "Prompt for project based on existing projects in agenda file.

Note: I tried this as interactive, but the capture templates
insist that it should not be interactive."
    (completing-read
     "Project: "
	 (sort
     (-distinct
      (org-map-entries
       (lambda ()
	 ;; Get the entry's title
	 (org-element-property :title (org-element-at-point)))
       ;; By convention project’s are:
       ;; - a level 4 headline
       ;; - tagged with :project:
       "+LEVEL=4+project"
       ;; Look within all agenda files
       'agenda))
     #'string<)))

When I started I thought I would need to create a local variable for projects. But I use org-map-entries to dynamically query the document for existing projects.

I also spent some time on the prompting function; in part because I thought it needed to be interactive. It does not.

Begin “Capturing” Notes for the Task

The “Task” capture template uses the file+function directive to find where in the document to insert the task.

The first parameter (e.g. jf/primary-agenda-filename-for-machine) specifies the agenda file for my machine. Those machines are work and personal; each with their own todo lists. The second parameter (e.g. jf/org-mode-find-project-node) is defined below; it finds and positions the cursor at the end of the given project within the give date.

;; Inspiration from https://gist.github.com/webbj74/0ab881ed0ce61153a82e
(cl-defun jf/org-mode-find-project-node (&key
					   (project (jf/org-mode-project-prompt))
					   ;; The `file+olp+datetree` directive creates
					   ;; a headline like “2022-09-03 Saturday”.
					   (within_headline (format-time-string "%Y-%m-%d %A")))
    "Find and position the cursor at the end of
the given PROJECT WITHIN_HEADLINE."
    ;; Ensure we’re using the right agenda file.
    (with-current-buffer (find-file-noselect jf/primary-agenda-filename-for-machine)
      (let ((existing-position (org-element-map
				   (org-element-parse-buffer)
				   'headline
				 ;; Finds the end position of:
				 ;; - a level 4 headline
				 ;; - that is tagged as a :project:
				 ;; - is titled as the given project
				 ;; - and is within the given headline
				 (lambda (hl)
				   (and (=(org-element-property :level hl) 4)
					;; I can't use the :title attribute as it is a
					;; more complicated structure; this gets me
					;; the raw string.
					(string= project (plist-get (cadr hl) :raw-value))
					(member "project" (org-element-property :tags hl))
					;; The element must have an ancestor with a headline of today
					(string= within_headline
						 (plist-get
						  ;; I want the raw title, no styling nor tags
						  (cadr (car (org-element-lineage hl))) :raw-value))
					(org-element-property :end hl)))
				 nil t)))
	(if existing-position
	    ;; Go to the existing position for this project
	    (goto-char existing-position)
	  (progn
	    ;; Go to the end of the file and append the project to the end
	    (end-of-buffer)
	    (insert (concat "\n**** " project " :project:\n\n")))))))

Current Implementation Constraint

My workflow does not need the “Project” capture. However the “Task” capture needs the headline structure that the “Project” capture creates. Future work that I could do would be for the “Task” capture to create the correct headline(s). But that’s a once a day inconvenience.

My Daily Task Sheet

Last the org-clock-report function provides a plain text tabular breakdown of my work days. Below is an anonymized example:

#+BEGIN: clocktable :scope subtree :maxlevel 5  :tcolumns 4
#+CAPTION: Clock summary at [2022-09-03 Sat 10:12]
| Headline                                           | Time    |       |      |      |
|----------------------------------------------------+---------+-------+------+------|
| *Total time*                                       | *14:30* |       |      |      |
|----------------------------------------------------+---------+-------+------+------|
| \_  2022-09 September                              |         | 14:30 |      |      |
| \_    2022-09-01 Thursday                          |         |       | 7:45 |      |
| \_      Client 1                                   |         |       |      | 0:30 |
| \_        Merge and Backport...                    |         |       |      | 0:30 |
| \_      Client 2                                   |         |       |      | 2:15 |
| \_        Get Bitnami SOLR Blocking Done           |         |       |      | 2:15 |
| \_      Learning Time                              |         |       |      | 1:30 |
| \_        Adjusting Time Tracking Automation...    |         |       |      | 0:30 |
| \_        Submit Proposal for Responsible...       |         |       |      | 0:30 |
| \_        Show and Tell                            |         |       |      | 0:30 |
| \_      Client 3                                   |         |       |      | 1:45 |
| \_        Pairing with A regarding Workflows       |         |       |      | 1:45 |
| \_      Client 4                                   |         |       |      | 1:15 |
| \_        Pairing on #138                          |         |       |      | 1:00 |
| \_        Reviewing...                             |         |       |      | 0:15 |
| \_      Client 5                                   |         |       |      | 0:30 |
| \_        Pairing with B on Collection Slugs       |         |       |      | 0:30 |
| \_    2022-09-02 Friday                            |         |       | 6:45 |      |
| \_      Client 6                                   |         |       |      | 0:15 |
| \_        Pairing with C regarding rebase...       |         |       |      | 0:15 |
| \_      Learning Time                              |         |       |      | 0:15 |
| \_        Writing About Emacs and Org-Mode Time... |         |       |      | 0:15 |
| \_      Client 1                                   |         |       |      | 2:15 |
| \_        Working on troubleshooting upstream...   |         |       |      | 0:30 |
| \_        Work on Documenting Hyrax’s IIIF...      |         |       |      | 1:45 |
| \_      Samvera                                    |         |       |      | 0:15 |
| \_        Reviewing PR for a Hyrax app without...  |         |       |      | 0:15 |
| \_      Client 2                                   |         |       |      | 1:30 |
| \_        Working on getting SOLR up and...        |         |       |      | 1:30 |
| \_      Client 7                                   |         |       |      | 1:15 |
| \_        Client 7 Upgrade Estimate                |         |       |      | 1:15 |
| \_      Client 5                                   |         |       |      | 1:00 |
| \_        Universal Viewer Overview                |         |       |      | 0:45 |
| \_        Working with D on Collections            |         |       |      | 0:15 |
#+END:

In the actual time sheet each of those lines link to the corresponding headline. The provides another way to navigate.

Conclusion

I never quite realized that I would appreciate time tracking. It helps me ensure that I’m not working more hours than I should. At other places, I’d work more hours. Here the time sheet helps set clear boundaries.

This workflow also helps me recover from context shifting. I want to help people get unstuck, but jumping in and out of that context does come with a cognitive cost. The underlying technical workflow provides the ritual/habit for re-orienting to what comes next.

As I mentioned earlier, my agenda file becomes a source for knowledge sharing; either with my future self or as a blog post. This article began as a quick note in my agenda file. And in that agenda file I’ve linked to this article.

Now to write a function to generate my daily stand-up “what did I do”; it should be rather straightforward based on my well structured time sheet and notes.

And as always, you can look to my dotemacs repository for even more regarding my Emacs configuration.

Кто-то из вас наверняка задавался вопросом: а нельзя ли программировать Arduino на чём-то более современном и удобном? Вот и я задавался. И нашёл Rust (не то, чтобы я о нём не знал). И на нём можно программировать микроконтроллеры AVR и платы Arduino, построенные на них. И здесь я расскажу о том, как настроить среду разработчика на Rust в Linux, GNU Emacs и Visual Studio Code и как запрограммировать Arduino Uno на моргание светодиодом.

#![no_std]
#![no_main]

use ruduino::Pin;
use ruduino::cores::current::{port};

#[no_mangle]
pub extern fn main() {
    port::B5::set_output();

    loop {
        port::B5::set_high();
        ruduino::delay::delay_ms(1000);
        port::B5::set_low();
        ruduino::delay::delay_ms(1000);
    }
}

In this post I’d like to publish my comments on a series of article by Ramin Honary claiming that “Emacs fulfills the UNIX Philosophy”. I consider this an interesting question from a kind of “theoretical” perspective, though probably practically irrelevant, and had previous correspondence with the Author on the matter.

The following contains a (abridged) few comments, notes, issues and suggestions I had sent Ramin before the article was published, most of which rather constitute my own than some objective view on the topic. For that reason I find it appropriate to publish them on my own site:

Emacs is an app platform

I believe we already mentioned the term “app” in our previous correspondence? I would at least add a footnote to clarify the point that had me irritated, that the term “app” had not necessarily imply a restricted, ready-made system where the user has no introspection or creative freedom on his own to mould their computing beyond the coarse, isolated, already available blocks one might find in an “app store”.

So I think it is better to define Emacs not as an editor, but as a Lisp app platform. It can be thought of as analogous to how the World Wide Web is a JavaScript app platform, or Microsoft’s .NET is a C# app platform.

I don’t know much about .NET/C#, but I hesitate to agree with the Javascript analogy. Maybe that should be elaborated on?

Seems like Functional Programming (FP)

In a UNIX Programming Environment, the fundamental unit of code that is a “program” is conceptually equivalent to a “function” in a FP language. The definition of a “program” need not be restricted to code that runs in it’s own process, and that would be a somewhat meaningless constraint to apply.

I might perhaps phrase this differently and rather say that it is an arbitrary restriction from someone considering the Unix philosophy from the specific perspective of a Unix environment (consisting of the kernel, a shell, pipes and the context of all the other traditional Unix programmes).

The point is that within a shell, Emacs appears to be alien, due to the “cultural distance” between the two. Invoicing Emacs functions from a shell is possible and can help illustrate the point, but at least when using something like Bash it is cumbersome. This can be contrasted with a tool like AWK, that in itself is comparable to Emacs as an Elisp interpreter, but is designed in such a way as to “fit in” to the existing culture.

Note that this gap also exists in the opposite direction, but Elisp has managed to bridge it slightly better (e.g. consider something like process-lines) – probably out of necessity considering that the shell remains more popular. But there remains a “translation overhead”, in that shelling-out is rarely idiomatic, seldom preferable to explicitly starting a process.

Of course, one could list several more distinguishing properties of FP than just the above three. Just to name a few: purity, referential transparency, pattern matching, polymorphism, and equational reasoning. But as long as we can agree that the three above points I mentioned are indeed useful and distinguishing properties of FP, it is easier to see the parallels between FP and the UNIX Philosophy.

I believe it would be productive to go into this point in more detail, as it is both crucial to your argument and especially prone to being misunderstood by someone who hasn’t considered this point of view before.

Perhaps one should distinguish between “lesser FP” (the core concepts you describe here) and “greater FP” (that a proper functional language like Haskell, Idris, OCaml, …) would satisfy. This is also important as Lisp is also not traditionally a “greater FP” language. After all, idiomatic Elisp is more often imperative than pure and functional – it has to be as the “side effect” of Elisp is the text editor and the user environment itself.

But yes, ultimately Lisp centres on the evaluation of expressions, rather than the execution of commands.

Lisp does FP better than UNIX shell programming

It just so happens that Emacs is one of the oldest Lisp implementations still in wide-spread use that was originally implemented for UNIX OS and it’s clones. Decades of evolution have made Emacs one of the most practical means of integrating Lisp-style FP within the UNIX Programming Environment.

This is a good point, and I might add two factors that contribute to this:

1. Elisp is not a standardised language specification (like Scheme or CL), so it was always easier to adapt the “vocabulary” and add new functionality.
2. Elisp by virtue of being the Lisp for GNU Emacs, the Emacs implementation for Unix and Unix-like systems, never found itself in the abstract position that other Lisp insisted on, in not wanting to accept the current dominance of Unix-style operating systems, and preferring to remain OS-generic (e.g. consider CL’s default file handling).

The parallel histories of UNIX and Lisp

In time, John McCarthy would come to work at CSAIL, and would bring Lisp with him to do symbolic computing and artificial intelligence, along with other AI founders like Peter Norvig and Marvin Minsky.

I might be misreading something here, but Peter Norvig was 8 years old in 1964. And from what I gather on his Wikipedia page is was never directly related to the MIT.

But by the time McCarthy started working at CSAIL, Kernighan and Ritchie were no longer working on Multics, they had moved on to invent the C programming language, which would then be used to develop UNIX.

Uh, that is not the account I am familiar with. Kernighan usually denies being involved with the creation of Unix. That is usually attributed to Ken Thompson, and it was initially written in PDP-8 assembler. Only with the PDP-11 in 1971 was there the move to rewrite it in a system-independent language Ken and Dennis created to this end, which ended up becoming C.

This is a good, recent overview by Ken Thompson himself: https://www.youtube.com/watch?v=EY6q5dv_B-o.

and MIT professor Richard M. Stallman talks about how he had used these UNIX computers while developing the first pieces of GNU software in the early 1980s.

Not sure if Stallman was ever a professor. He might have been offered the position, but to my recollection his position at the MIT was always as a kind of research assistant/systems programmer/administrator.

So Emacs was one of the earliest interactive Lisp REPLs ever programmed for the UNIX OS.

Not quite true, I think that Franz Lisp (https://en.wikipedia.org/wiki/Franz_Lisp) predates it

The UNIX people were not guided by Lambda Calculus, and without the solid mathematical foundations that McCarthy used for Lisp, UNIX shell programming turned out to not be anywhere near as elegant as Lisp.

I would be careful about that claim, as it is known that McCarthy didn’t understand the lambda calculus completely when designing Lisp, but rather just reused the name for anonymous functions. See https://www.cs.kent.ac.uk/people/staff/dat/tfp12/tfp12.pdf, page 6 (“[…] At the time he invented LISP, McCarthy was aware of (Church 1941) but had not studied it. The theoretical model behind LISP was Kleene’s theory of first order recursive functions”).

Unfortunately, without the Lambda Calculus as a guide, the UNIX Philosophy turned out to be a misguided, incomplete description of functional programming.

I would expect a phrase like this to gather some critique from unfavourable readers. You appear to be putting the cart in front of the horse, by assuming Unix was aspiring towards functional programming, instead of just realising a less general version of the same idea.

And it becomes even more difficult when you consider this post by Paul Graham, in which he claims that Doug McIlroy was present as a presentation by John McCarthy introduced Lisp.

Therefore Emacs does actually follow the tenets of the UNIX Philosophy, in some ways, even better than the UNIX Programming Environment does so itself.

This is also an inference that people might take issue with, since you are proving something for the more general (weaker) interpretation of the “UNIX Philosophy”, but then you deduce a claim regarding the specific “UNIX Philosophy” that doesn’t directly follow, since the UNIX Philosophy is embedded in the specific technical factors and limitation that were discussed above.

I believe that the safer but equally satisfying claim, that Emacs does not contradict the spirit and values of the UNIX Philosophy, would be preferable. Since unless someone is dedicated to the concrete implementations (in which case they cannot be convinced of the point regarding Emacs to begin with), they should be able to recognise that the principles of simplicity, modularity and re-usability are given when using Emacs/Elisp, as soon as they let go of the arbitrary fixation on what a program is and how it is implemented (an operating system process executing a file).

Yes, exactly. Or you could just use Emacs as your shell.

I would also remind the reader of what a shell is: A “generic” program that sits between the user and the kernel Or it wraps the kernel, just like a shell would wrap a crab. Emacs is a shell in a substantial sense, even if you don’t “chsh” it, since it can take user commands and “translate” them into system calls to the operating system so that stuff actually happens.

This also ties into the initial point, and on why the term “apps” irks me. It appears you didn’t use it to construct any argument. Considering Emacs as a shell that constitutes an interactive programming environment, you can regard Emacs as any other classical Unix shell that just has more built-in and in-shell functionality and a stronger type system (or any type system at all, albeit dynamical, as it is probably more adequate for an interactive environment).

I’ve recently continued working on an extension for package.el (Emacs default package manager) that would integrate version controlled packages next to the “tarball” installations. It has been functional for a while now, but there are still improvements (technical and UX) to be made.

There have been similar projects over the years, the main difference with my take at it is that it has been developed as part of the core of Emacs from the beginning, and with some luck might be released along with Emacs 29.

Here I’d like to invite anyone interested in this happening to try out the feature/package+vc branch emacs.git, and perhaps comment on their experience.

Next to this “main” feature, I’ve also been working on minor quality of life improvements, such as

• The ability to display package news (via describe-package)
• The ability to contact a maintainer (via M-x package-contact-maintainer)
• (TODO) The ability to send maintainers patches directly from Emacs.

A number of these features are currently non-functional, since they will require corresponding modifications to be made to the ELPA build system, which will follow in time.

If you are interested but don’t have an Emacs checkout, all you need to do is

git clone -b feature/package+vc https://git.savannah.gnu.org/git/emacs.git
cd emacs
make
./src/emacs

And then install a package using M-x package-vc-install.

If there are any issues, consult the INSTALL.REPO file.

See also on “emacs-devel”: “Request for Feedback: Adding VC support to package.el”, from February and “Re: feature/package+vc 04c4c578c7 3/4: Allow for packages to be installed directly from VCS” that ran until November.

GNU Guix (just like Nix) can easily spawn development environments with all the necessary dependencies one may need to work on a project.

For example: In a shell session, all you would have to do is to run

$ guix shell gcc

and a new shell is launched with adjusted environmental variables, so that GCC can now be used. Likewise, you can instruct Guix to fetch the developmental dependencies, i.e. exactly what you need to build a package

$ guix shell --development gcc

As a matter of convenience, one can also specify what an environment should consist of and store that in a file. An easy way to generate such a file is using the --export-manifest option:

$ guix shell --export-manifest --development gcc > manifest.scm

On my system this generates this file:

;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.
(package->development-manifest
  (specification->package "gcc"))

More details on other options can be found in the manual.

This is fine, but if you use Emacs, then the shell and the editor are “inverted”, or rather Emacs performs the function of a shell (generic user interface, that wraps the kernel). To use something like guix shell, you’d have to either

1. Start Emacs in a guix shell session so that it inherits the environmental variables set by Guix.
2. Prefix any command you might execute in M-x shell, using shell-command, compile, etc. with a guix shell ... -- prefix.`

Neither of these two options are that convenient, so I was unusually delighted when I found a simple solution in the recently updated buffer-env by Augusto Stoffel (author of many under-appreciated packages). Reading through the source for the first time was a real joy, and I kept thinking about it like one would after hearing a good, catchy song.

The package has as simple interface, and for the most part it can be configured in a single line:

(add-hook 'hack-local-variables-hook 'buffer-env-update)

The package was added to GNU ELPA earlier this year, and was initially just described as a pure-elisp direnv implementation. If this is all you need, you don’t need to bother yourself with anything else.

For those unfamiliar with hack-local-variables-hook, here is the docstring:

Normal hook run after processing a file’s local variables specs. Major modes can use this to examine user-specified local variables in order to initialize other data structure based on them.

So what buffer-env-update does, in the case of Guix, is check the buffer-env-commands variable and find an entry that says “if you find a manifest.scm command”, run

guix shell -D -f \"$0\" -- env -0

(where $0 is replaced with the absolute file path), parse the output and update variables such as process-environment and exec-path, that influence how processes are spawned.

The Guix documentation mentions something similar to this idea, but I find it much more complicated than what buffer-env allows me to do.

My configuration goes a bit further than just modifying hack-local-variables-hook: Using Setup I have the following in my init.el:

;;;;; Dynamic Environments
(setup (:if-package buffer-env)
  (:option buffer-env-script-name "manifest.scm")
  (add-hook 'comint-mode-hook #'hack-dir-local-variables-non-file-buffer)
  (add-hook 'hack-local-variables-hook #'buffer-env-update))

In other words, this changes two notable things:

1. Setting buffer-env-script-name to looks for manifest.scm files, so that buffer-env automatically picks up on manifest.scm files. Could be a list as well, if I also wanted to use guix.scm files.
2. Load directory local variables in comint buffers (REPLs, M-x shell, …) so that buffer-env also takes effect in these kinds of buffers that have no files associated with them.

What I found particularly clever is that there is no need for an ongoing guix shell session, but that by calling env -0 we extract all the environmental variables and apply them manually. This might run into issues if you use guix gc concurrently though. And after all, all there is to Guix or Nix are just a few clever symbolic links and environmental variables.

Using a Tool and Knowing Its Uses Are Not the Same

Save some time to dream
Save some time for yourself
Don't let your time slip away
Or be stolen by somebody else

Yesterday I received a lovely email that included the following question:

Is it common for programmers to spend a bit of their working hours fixing/sharpening their tools?

As I wrote up my response, I asked for permission to reference their question. The sender obliged, so here’s what I wrote (with some editing to reflect a broader audience).

The Value of Fixing/Sharpening Your Tools

This is a great question and I have two trains of thought.

First, I don’t know if you’ve heard of DEV but this kind of question is one that the community loves to interact with. I work for the Forem, the maintainers of the software that powers DEV; so I’m a bit biased on sending people there.

Second, let’s look and my specific situation and understanding.

It is my assumption that in the process of using my tools, I explore those tools for further utility or “productivity” gains. This builds on the idea that I am looking for outcomes not time in seat/at keyboard. And please know that one outcome I consider absolutely enjoyable is the puttering of time simply playing with a computer. Much like idly whittling wood with a knife is quite enjoyable; and while the tangible outcome is a pointy stick, the intangible is a sense of wonder. A case in point, I spent about 2 hours writing an emacs lisp function that helped me tidy up the the blend of our pull request template and the commit message). There are (or were?) a few outcomes of taking time to do that:

First, the solution saves me about 20 seconds of time each pull request. I’ve used that function about 150 times since writing it, so I’m at 50 minutes saved with a 2 hour spend. In addition, those 20 seconds saved also included micro-decisions.

By automating a bit of those tasks, I’ve reduced one location that might cause decision fatigue.

Second, I now have a mental pathway of how to do this or what might be possible; which helped me quickly write a script to facilitate writing end of week reports (10 minutes or so?).

I now use that new function about 3 times a week. I likely saves me 30 seconds of copy paste plus the context switching of multiple tools.

This pattern repeats itself in other tasks.

I have long mapped Cmd \+ . to toggle between spec/test and file. That alone has helped me always think about my test as well as the predictable file system structure necessary to sustain that pathway.

I have watched other developers open their project navigation tree, and click through folders to get to the related test. That consumes decision-making resources.

I recently fiddled with installing tmr.el, a timer package for Emacs (Emacs 📖). Why? Because I can now easily set a timer within Emacs. I don’t need to grab my phone (which the vary act of doing already breaks my mental thinking).

Do I anticipate using tmr.el much? No, because it’s not often that I want a timer while I’m at my computer. But it’s there. And I practiced setting it.

A final case of working with Emacs is work I did to help with my org-roam contexts. I can select a set of tags (or a named group of tags) to auto-filter my Org-Roam commands: capture, find, insert, or Completion at Point Function (CaPF 📖).

What this has meant is I can easily write work notes and play notes in the same directory. I can then both interconnect those notes but also help me not “accidentally” inter-connect them. This helps me remove a decision-making point.

It has also helped me begin moving my blog posts from stand-alone Hugo files into my Knowledge Management System (PKM 📖) system.

And I try to do all of this “exploration” of my editor when I’m working on a well understood to me problem.

When I’m trying to reason through something more complicated or vague, I avoid trying to also dive into my text editor. But I might make a quick note to tell myself “Hey you did this thing…maybe spend some time thinking about it after you’re done.”

I hope this provides some context on why you personally would want to do it, and how it’s not about asking for permission but to instead have that be part of your software development process.

Ending with a Question

Now I’m wondering, what have you learned in sharpening your tools?

It seems to me that when someone claims to be interested in Emacs, they are given two paths they are given two paths to choose from:

1. Start with nothing but an empty init.el and built up
2. Start with a “Framework”, “Starter-Kit”, “Distribution”¹ or however you want to call it, and try to understand what is going on.

Some call the former approach “Vanilla Emacs”². The latter have a number of choices.

I have never been a fan of the latter, for me or for anyone. All in all I do believe it hinders more than it helps – at least in the long term. I don’t want to go into that debate right now, but that being said, it obviously fill a niche. For a while now I have been wondering if a different approach could be taken to solve this problem, a compromise/combination of sorts.

Having some spare time on my hands, I wrote a little proof-of-concept to configure Emacs by deciding what you are interested in and being given a template of a configuration that you can use as a starting-point. It is appears to work, it is hack and it is accessible here:

In a certain sense, this is a kind of reverse-literate configuration. I have previously argued that literate configurations are mostly a waste of time and effort, especially given the self-documenting nature of Emacs. In this situation, the requirements have changed and good explanations as to what each option, mode, hook, etc. does are essential.

To emphasise my previous point, this is a proof of concept. As such it is by far not complete or in any reasonable sense coherent. The design is pretty 90ish/rudimentary³, hasn’t been tested for mobile use beyond adding a certain meta tag, the server-side is wonky⁴ the package selection is limited⁵, who knows how many typos there are and the output is not as clean as it could be.

If there is interest in making this a more serious thing, any and all input is appreciated. Best send it to my public inbox ~pkal/public-inbox@lists.sr.ht, or ping me via IRC.

A few ideas going forwards might be:

• Allow users to pick a configuration macro like use-package or Setup to use when generating the configuration.
• Optionally allow Popcon-like logging to collect a better image of what features people are interested in, which could be used as a empirical foundation (albeit not perfect) when discussing trends among Emacs users.
• A system to filter out what detail of information you are interested in. Someone just starting with Emacs won’t care too much about the differences between Fido and Vertico. Someone coming back after a while might.
• More author attribution, when referencing packages.
• Allowing for a configuration to be encoded in a URL so that it can be shared and used as a starting point. E.g. https://emacs.amodernist.com/conf/31242141 might be useful for embedded development, but https://emacs.amodernist.com/conf/8098562 has a good foundation for working with LaTeX. Note that these would link to the form, with certain options filled in and perhaps highlighted, not a finished configuration file.
• Specify the content for the generator (questions, code-generation, …) externally to the web server, so that it can be reused in other applications, such as a built-in wizard for Emacs ⁶
• Ask and make use of the operating system (or operating systems) the user intends to use.

As a brief update to my previous post on Compat, my compatibility library for Emacs Lisp, I would like to re-post my message from the compat-announce mailing list:

I am pleased to announce the first official public release of the Emacs library “compat”, the backwards compatibility library for Emacs Lisp, providing support for a substantial number of Emacs Lisp features implemented up until Emacs 28.

The current main intentions are to provide older versions of Emacs with emulated features from future releases. As of the current release, the library supports Emacs 24.3 (released 2013-03-11) and newer. This of course is not always possible, and I recommend reading the bundled Info manual to check what features Compat does and doesn’t manage provide (and with what caveats).

Compat will be distributed via GNU ELPA. Please contact compat-devel if there is interest in alternative methods of distribution.

I would like to explicitly thank Stefan Monnier for his technical input, Mattias Engdegård for his help on specific edge-cases and with writing the rather extensive test suite (currently 1593 ERT tests) and Daniel Mendler for engaging in productive discussions on the design of the library itself.

As promised, the release was to coincide with the announcement of Emacs 28. So just like Emacs, this means that the plan for Compat will now be to follow Emacs 29 and catch up to the developments on the master branch¹.

A word of caution, to anyone who has read the previous post: The usage of compat has been changed slightly, to be less controversial. Whereas previously, the library loaded itself like so²:

;;;###autoload (require 'compat)

it is now necessary for a library interested in using Compat to require it manually:

(require 'compat)

Doing so will load all missing compatibility functions that haven’t been defined yet. Notably missing will be new features that have been added to existing functions, such as the optional third test function argument added to assoc in Emacs 26.1. To access these an additional require call is necessary, e.g. in this case

(require 'compat-26)

this will additionally load “prefixed” functions, so in this case compat-assoc These were previously implemented by advising the already existing functions, such as assoc, but the decision to move to prefixing was done to avoid the performance penalty and intrusion that come with Advice.

All in all this makes the library a little less transparent, but a lot easier to reason about.

As with anything I work on, I am glad to help with any questions or issues. Just send me or the mailing list a message.

Feeding the Personal Knowledge Management System

In Using a File as a Template in Emacs, I wrote about the end of week reports that my supervisor tasked me with writing. I want to expand a bit on one of the scripts I uses to help me fill out that report.

The Script

The following script is one of the tools I use to help me write my end of week report. It only considers the Forem code base.

cd ~/git/forem ;
echo "Authored PRs Merged Since 2022-03-25\n"
git log --since=2022-03-25 \
  --format="%h — %s %an, (%cs) %d" \
  --author="(Suzanne A|Jeremy F|Arit A|Dwight S|Anna B)"\
  | rg "— ([^\(]+) \(\#(\d+)\) ([^,]+)," --only-matching -r '- $1 :: forem/forem#$2 $1' \
  | sort
echo "\nCo-authored PRs Merged Since 2022-03-25\n"
git log --since=2022-03-25 \
  --grep="Co-authored-by: (Suzanne A|Jeremy F|Arit A|Dwight S|Anna B)" \
  --format="%(trailers:key=Co-authored-by,separator=%x2C,valueonly=true) :: %s" \
  | rg "^([^<]+) <.*> :: ([^\(]+) \(\#(\d+)\)" --only-matching -r '- $1 :: forem/forem#$3 $2' \
  | sort

In the above script there are two sections:

• The first section are the commits authored by members of the Content Experience Pod.
• The second section are the commits in which pod members contributed one or more commits to the PR but were not the initiating author; Git registers these as Co-authors.

In other words, the script shows what code my team helped ship for the week.

Where to File Away that Script?

For a week or two I was running a simpler version of the above script; I would search through my shell command history, find the one that looked right, and adjust the date.

That worked but I’d prefer to not rely on that workflow. I added the script to my Content Experience Pod’s Org-Roam node; a document that is part of myKnowledge Management System (PKM 📖) system.

Here’s what it looks like in that document:

#+Begin_src sh :results output :cmdline (org-read-date)
  cd ~/git/forem ;
  echo "Authored PRs Merged Since $1\n"
  git log --since=$1 \
    --format="%h — %s %an, (%cs) %d" \
    --author="(Suzanne A|Jeremy F|Arit A|Dwight S|Anna B)"\
    | rg "— ([^\(]+) \(\#(\d+)\) ([^,]+)," --only-matching -r '- $3 :: forem/forem#$2 $1' \
    | sort
  echo "\nCo-authored PRs Merged Since $1\n"
  git log --since=$1 \
    --grep="Co-authored-by: (Suzanne A|Jeremy F|Arit A|Dwight S|Anna B)" \
    --format="%(trailers:key=Co-authored-by,separator=%x2C,valueonly=true) :: %s" \
    | rg "^([^<]+) <.*> :: ([^\(]+) \(\#(\d+)\)" --only-matching -r '- $1 :: forem/forem#$3 $2' \
    | sort
#+end_src

It’s a little different, because I’ve written it using Org-Mode’s Babel syntax.

The first line (e.g. #+Begin_src sh :results output :cmdline (org-read-date)) declares:

• A code-block with sh syntax
• To write results as raw output
• To pass the results of the (org-read-date) function as $1 to the script

The remaining lines are almost verbatim of what I previously wrote; except instead of the “hard-coded” date of 2022-03-25 I’m using the results of the org-read-date function.

To run the report, I set point (e.g. the cursor) in that code block, type C-c C-c and Emacs 📖 evaluates the Babel block.

First it calls the org-read-date function, prompting me to select a date. Then it runs the shell command. And outputs the results just below the Babel block.

From there, I can see one aspect of the work my team has done for the week.

Conclusion

Prior to adopting Emacs, and Org-Mode specifically, I would’ve floundered on where to put this. It would’ve remained in my shell history.

But now that I’ve associated this with a group of people, written it in a less ephemeral file, and written about it, I will be more likely to remember both it’s existence and what it did.

I learned about Git 📖’s commit trailers and how to find the Co-authors of a commit. Which helps make visible the work that folks do to help another person get their pull request merged.

I also learned a bit more about passing arguments from Babel into the script it’s executing.

And last, because I wrote this blog post in my PKM system, I have a reference from the published blog post, to the Content Experience Pod.

In other words, I’m doing my best to create breadcrumbs to help me find the particulars of a script I wrote.

Minor Automation to Facilitate End of Week Reporting

At Forem, one of my responsibilities is writing up an end of week status report for the projects assigned to my team. Sometimes I delegate that responsibility (if there’s someone with more information for the week’s update).

I’ve found that I enjoy writing these reports. I spend about thirty minutes per project writing up the report. During that time I gather what’ve we done, what we’re planning to do next week, and write up any risks to the project.

Earlier this week, Allison, our head of engineering, provided an adjusted template to help facilitate writing consistent reports for tracking issues.

I figured I’d go ahead and automate Emacs 📖 to help me use that template.

Forem End of Week Status Update

The following emacs-lisp creates a buffer, from an existing template, to help kick off writing my end of week status reports.

(defvar jf/forem-eow-template
  "~/git/forem-internal-eng/.github/epic-progress-update.md"
  "The location of the template to use for end of week reporting.")

(cl-defun jf/forem-prepare-end-of-week-status-update (&key (template jf/forem-eow-template))
  "Create a buffer for writing up an Engineering End of Week Status Update.

TODO: Consider pulling down the latest version of that template."
  (interactive)
  (let* ((body (with-temp-buffer
		 (insert-file-contents template)
		 (buffer-string)))
	 (eow-buffer (get-buffer-create "*Forem EoW Update*")))
    (switch-to-buffer eow-buffer)
    (erase-buffer)
    (markdown-mode)
    (hammerspoon-edit-minor-mode)
    (insert body)
    (beginning-of-buffer)
    (kill-line)
    (insert (concat "## " (format-time-string "%Y-%m-%d")))))

Details

The above code:

• copies the contents of the template
• creates a new buffer titled *Forem EoW Update*
• sets it as Markdown type content
• enables Hammerspoon (see below)
• pastes the contents into the new buffer
• sets the first line to today’s date

I use Hammerspoon and the editWithEmacs.spoon to help me use Emacs for editing Emacs text areas. I wrote about that in Send Anything in OS X to Emacs for Editing. .

Conclusion

This little bit of automation ensures that I’m using the consistent template and am writing using my favorite computer tool. It’s a quick bit of automation, but one that I need to leverage at least once a week for the foreseeable future.

The Continuing Adventures of Writing to Learn

Last Thursday I submitted the following issue for org-roam: Allow for `org-roam-buffer`’s Backlinks section to be unique per source. On Saturday I submitted a patch to address the issue: Adding unique option fro org-roam-backlinks-section.

While chatting with Jethro Org Roam’s maintainer he suggested using a pcase construct. I have read the pcase documentation and struggled to sift through it. It’s right on the boundary of my comprehension. So I proceeded with my pull request.

Later, I submitted a proposal for a customization, and Jethro explained that the pcase construct would likely be cleaner and more generalizable. He then wrote up that change and pinged me. Thank you Jethro, now I have a pcase use case that I understand what we’re doing, which will help me move pcase further into my area of comprehension.

To learn, I’m going to write about the change that Jethro put forward:

(dolist (section-fn org-roam-mode-section-functions)
  (pcase section-fn
    ((pred functionp)
     (funcall section-fn org-roam-buffer-current-node))
    (`(,fn . ,args)
     (apply fn (cons org-roam-buffer-current-node args)))
    (_
     (user-error "Invalid `org-roam-mode-section-functions specification.'")))))

Line 1: This iterates over the org-roam-mode-section-functions list. Each element of the list is section-fn. The element is “passed” to the “anonymous function” that is lines 2 through 8 (e.g. the “body” of the dolist).

Line 2: The pcase expression we’re evaluating is the section-fn. Reading the docstring for pcase, it doesn’t say mention it explicitly, but the verbose name for pcase could be patterning-matching-case-statement.

Line 3: This is the first pattern that we check. This line answers the question: Is the section-fn a function?

Line 4: When section-fn is a function, call that function passing the org-roam-buffer-current-node as the only argument.

Line 5: This is the line that breaks me. What I do know is that when section-fn is (org-roam-backlinks-section :unique t) then this is a match. But

Line 6: Call the fn (which is declared in line 5?) with the org-roam-buffer-current-node and the args. Okay, this is breaking my brain a bit.

Line 7 and 8: The fallback is to raise a user-error.

That Which Breaks Me

On a cursory read, line 5 and 6 confound me. My mind wonders what is fn and args? How do they become the “variables” of line 6?

While thrashing against this, I started building up some search terms: “pcase emacs backquote”. Which lead me to Backquote-Style Patterns. Jackpot!

Backquote-style patterns are a powerful set of pcase pattern extensions (created using pcase-defmacro) that make it easy to match expval against specifications of its structure.

Of course, it’s in the manual. And reading further, I see the following:

The first three clauses use backquote-style patterns. `(add ,x ,y) is a pattern that checks that form is a three-element list starting with the literal symbol add, then extracts the second and third elements and binds them to symbols x and y, respectively.

The commas are used to “extract” elements of the section-fn and allow them to be used later on. I’m trying to connect this to my extensive Ruby (Ruby 📖) experience, and I’m struggling to do so.

I’ve used case statements before, but I hadn’t considered how I might use comparison statement as the thing that also “declares” the variables for evaluation.

Thank You

So thank you Jethro for taking the time to refine the org-roam code. This has helped me further develop my understanding of Emacs.

Ever Improving my Personal Note Taking Process

This past week, I’ve been migrating my blogging workflow towards an Org-Mode-first workflow. By that I mean I’m writing my blog posts using Org-Mode 📖 and exporting the content to Hugo for building Take on Rules.

Why This Added Layer of Effort?

The short answer to “why this added layer of effort” is because of Org-Roam and Org-Transclusion. But those are technical implementations. First I want to talk about the functional issues I’ve encountered.

I write a lot of notes. For work, I keep a daily list of tasks I’ve worked on. I also write meeting notes

And I write notes to think through a technical approach. For all of that, I use Org-Mode. For commit messages and interactions with GitHub, I use Markdown.

For play, I write session reports, poetry, thoughts about games, reviews, and so on. Up until recently, I’ve written most of that with Markdown.

In other words, I had two different formats for my non-programming writing. That alone isn’t a reason to change; and is one reason I previously had not.

Yet this bifurcation sat as a mental irritant, even though it was not quite a problem to solve. A year ago someone told me that at some point I’d be migrating to Org-Mode for blogging. So credit to them for seeing that future. I chose to sit with this mental friction, to better understand the problem I was experiencing.

What Changed?

A slow moving confluence of moments brought about this change.

When I was facilitating the New Vistas in the Thel Sector campaign, I wrote notes using Org-Mode and Org-Roam and exported my notes to Markdown. At the time, I would then finesse the export. This involved scrubbing links to “private” notes. My Game Master (GM 📖) notes if you will. I was also new to Emacs and not yet comfortable with emacs-lisp.

As that campaign spun down, I started writing functions to help me compose Markdown blog posts. These functions lean heavily on yasnippet and later custom emacs-lisp. In all of this, I wasn’t ready to address the mixture of “public” and “private” notes paired with the idea of yet another content migration. I have migrated this blog’s page generation from Wordpress

Fast forward to , and I’m playing in two Burning Wheel Gold (BWG 📖) games. I’m facilitating the Mistimed Scroll campaign. I am about 5 session reports behind on this campaign . And I’m playing in Burning Locusts. In both campaigns, one of the fellow players is also an Emacs enthusiast.

When they shared their DOT notation graph representating the relationships in Graphviz, I followed with Using PlantUML to Model RPG Relationship Maps. They then wrote Burning Plants.

That exchange was another brain-worm that nudged me revisit my blogging process.

Why? Because Org-Roam’s Org-Roam UI can generate an interactive graph based on nodes. And more importantly, I had found and was playing games with a fellow gamer that was interested in Emacs, relationship graphs, note taking, and many other shared interestes. The kind of gamer who I’d love to meet for coffee and just talk about games and theory.

Sidebar for a Glossary

Before I go further, I want to briefly work through some graph terms.

Node

a chunk of descriptive information

Edge

a reference from one Node to another Node

In Org-Roam, when I create Node A and add a link to Node B, that creates an edge A -> B. In Org-Roam, Node A references Node B and Node B has a back link to Node A.

Org-Roam also exposes the concept of :ROAM_REFS:. Let’s say I add Node C to my notes. For Node C lets set its :ROAM_REFS: to include “https://orgmode.org”. Now, anytime I link to “https://orgmode.org” in my notes, Node C will have a back link to that reference.

In other words, :ROAM_REFS: let me create proxies for external concepts.

As I learned about that, I started exporting my blog’s posts to my Org-Mode directory. The original reason for export into Org-Mode was because I have also been writing more technical blog posts that I wanted to reference in my private note-taking; a confluence of labyrinthine moments.

But, I was treating my blog as the primary source of knowledge. And that runs contrary to the actual model. My thoughts are private, and in speaking them, they become public. But I digress a bit.

Back to taking Session Notes

With my blog now “imported” into my Org Mode ecosystem. Not migrated just imported via some Ruby scripts and Pandoc antics. Critical in this import is that I would generate a node ID and a :ROAM_REF: entry. Thus creating the connection between public URL and private node identity. I started thinking about the directional flow of information.

This is when I again revisited Ox-Hugo. I was looking to scratch the itch of resolving the directional flow of knowledge, and the Do I need to re-write my whole blog in Org? page gave me confidence to spend a bit of time exploring.

I started with a Literate Programming approach and wrote my takeonrules.org export functions. I also mentally framed this whole thing as an experiment; something I would test and observe and rollback if necessary.

The main concept being that I wanted to correct the flow of information (e.g., private to public). When exporting a node, I did not want to export links to private nodes. It is okay to export the text but I don’t want to export broken links.

And my experiment worked.

But Wait, There’s More

As I solve one problem, I become aware of more opportunities that arise with the new state. And I owe you, dear reader, information about Org-Transclusion.

Once I had the export working, I started looking at the graphical structure of my notes in Org-Roam UI. And as structured, each game session node (e.g. the node I publish to my blog) had lots of edges. After all, I was writing all of these notes in one node. My custom export process assumes that I’m exporting one file which has one Org-Roam node; an implementation detail that has thus far been adequate

Which node has the reference impacts the back links. To make it concrete; my session notes incorporate Non-Player Characters (NPCs 📖), and while back links from the NPC to their sessions is nice, I’m often more interested in the scenes in which they were present.

Around this time, I also got the wild idea of “How might I, a player in Burning Locusts share my notes with the game facilitator, such that they could overlay their notes on my notes. I spent a bit of time thinking through that in my Burning Locusts Campaign Data repository. You can also see a snapshot of the campaign data.

Which brings me to Org-Transclusion. I started reworking this in my to be published Burning Locusts: Session 7 notes. At the time of writing this I linked to my internal notes, but you dear reader, will not have such a link until I both re-export this post and publish the session notes.

What I’ve now chosen to do is to create a node for each scene. And transclude those nodes into my session report. In a way, my session report is an index of scenes. Here’s what that session report looks like:

:PROPERTIES:
:ID:       4E332C1F-57FA-47D3-B303-A4B21AF3BA3B
:SESSION_REPORT_DATE: 2022-02-16
:SESSION_REPORT_GAME: BURNING-WHEEL-GOLD
:SESSION_REPORT_LOCATION: via Discord and Roll20
:END:

#+title: Burning Locusts: Session 7
#+SUBTITLE: Arson, Ambush, and Art Sales
#+FILETAGS: :session-report:burning-locusts:rpg:burning-wheel:

* TODO More Happenings at Adriano’s Party

#+transclude: [[file:20220225---adriano-faraldo-s-party.org::*Session 7][Session 7]] :only-contents


* TODO Aftermath of the Ambush at Adriano’s

#+transclude: [[id:1226FDD8-E7D3-4AF1-9958-5DC0ABE721FF][Aftermath of the Ambush at Adriano’s]]


* DONE Background Events Resolved After Adriano’s Party

#+transclude: [[id:6AC158C5-2279-4A80-916F-E087F5B6FF2D][Background Events Resolved After Adriano’s Party]]


* DONE Frederico Meets with the City Clerk

#+transclude: [[id:8761371C-B4C2-4E06-8C13-135BB0780382][Frederico Meets with the City Clerk]]


* DONE Antonius and Maccio Have a Conversation Regarding the Arsonist

#+transclude: [[id:7DABFA0F-5200-46E0-A494-F9EBFD23CBAD][Antonius and Maccio Have a Conversation Regarding the Arsonist]]


* DONE Frederico and Antonius Seek a Fence

#+transclude:  [[id:0BEFCA44-8DA4-4167-A727-07F676F6EBD1][Frederico and Antonius Seek a Fence]]

* TODO Closing Scene

I have minimal memory of this, as it was very much a denouement.

For transclusions to “register” as back links, I removed (keyword "transclude") from the org-roam-db-extra-links-exclude-keys variable. I submitted an issue to org-transclusion describing the behavior without this adjustment

You’ll also note that I repeat the header with the label of the transclusion. I wouldn’t need to do this, if I moved the header into the file. But I prefer this method.

Conclusion

Riffing on Clarke, and as I’ve said before, “Any sufficiently advanced hobby is indistinguishable from work.”

I have a lot of tooling for helping me write Markdown blog posts, and nothing that I’ve done invalidates that. Instead I’ve extended my workflow to allow me to now better take and share session notes. If I’ve learned anything in my 2 years with Emacs 📖, all aspects of my computer life benefit when I experiment with writing and knowledge management.

For example, when I’m writing my daily work activity log and am working with someone on that activity, I add a reference to those coworkers. Yes, there’s a node for each person I interact with at work . This way, if I need to recall information, I have the back link available as a tool to assist on that recollection.

Postscript

As I finished writing this post, I realized that another catalyst in this change was adopting a Literate Programming approach. I spent time moving my Emacs configuration to use org-babel-tangle, which allows for a mixture of prose and code. Which is not to confuse the prose with code comments.

It turns out that taking the time to write through an observed problem with prose tools while also having access to coding context all in the same file helps me better think through these problems.

It seems to be that Unix and Lisp share a similar trajectory in terms of their “developmental” phase and the subsequent “stabilisation”. Both emerge from research-y circles¹, grows a culture that is interested in experimenting with these systems, later spreads, diverges and fragments. For Unix, Éric Lévénez Unix Timeline demonstrates this well.

To ameliorate the difficulties developers have to struggle with when intending to develop portable software, standardisation efforts emerged. In the one case POSIX, in the other the aforementioned Common Lisp language. Of course, an operating system and a programming system aren’t the same², and their respective approaches differed (which is a topic worth investigating in itself), but as a consequence of standardisation both systems had most of the substantial innovation was moved “outwards”, towards third-party projects or specific implementations³. Just think about when the last time was that you found out about a new CLI command that is available on any POSIX system (for me it was tsort, and I don’t remember what the one before that was)?

A difference of note between the two is that while POSIX has been revised and updated a number of times, adding new userland commands, libc functions, syscalls, etc., Common Lisp has stayed the same since 1994, when the ANSI Common Lisp standard was published. Nevertheless, Common Lisp didn’t stagnate and become deprecated. The Language was expressive enough to keep itself up to date, via libraries, macros and fundamentally a community that was interested in seeing the system flourish. This kind of dynamism was simply not possible given the structure of Unix and the intend behind POSIX.

Yet no standard is all-encompassing, in the sense that it manages to supersede all of its predecessors. Sure, MacLisp and Interlisp attract little attention nowadays, but dialects like Scheme continue to fascinate many, and Mock-, Ersatz-, Hover fly Lisps (or however you want to call them) fool many to this day.

One of the most interesting Lisps that I would strongly position within the traditional Lisp “family tree”, would be Emacs Lisp (Lisp) – and this despite the deficiencies it demonstrates when compared to the family patriarch, Common Lisp. One of the main reasons is probably that it was never standardised, and through all the years continued existing as a “Living Lisp”, growing new features, developing and deprecating best practices and changing along with the needs of its users.

Yet an issue comparable to that caused by standardisation has weighed on the shoulders of Emacs Lisp developers, namely support cross different versions of Emacs (the interpreter). A decision has to be made to either continue supporting older versions of Emacs, or to make use of newer features. This question is even more pressing, as Emacs prides itself in the explicit appeal to use by programming. Neither the decision to restrict functionality and/or increase complexity to maintain backwards compatibility, nor the option to exclude anyone stuck with an older version for the sake of convenience.

In the following I would like to present my attempt at improving this situation, taking advantage of Elisp being a Lisp.

Compatibility library for Emacs

For the past few months I have been working on a general compatibility library for Emacs, called “compat”. As of writing, it hasn’t been release yet⁴ (the plan is to coincide with the release of Emacs 28), but I would like to pre-announce it in this text anyway.

Note: I do not recommend installing compat as a user. Of course, there is nothing stopping you from doing so, but it is meant to be a end-user package.

Usage

To use compat, all you need to do is add a dependency to your Package-Requires header, like so(where XY.Z should of course be replaced by an actual version number):

;; Package-Requires: ((emacs "XY.Z") (compat "28.1.0.0"))

The rest takes care of itself, and you will now be able to backported functionality from newer versions of Emacs, and made available for all versions back until 24.3 (released in March of 2013)⁵.

For the most part this means you can invoke functions like alist-get, if-let, thread-first all from 25.1, the new JSON functions from 27.1⁶, and a slew of helpful functions from 28.1 (string-search, string-replace, length= & co., file-name-concat, named-let (with TCO!), and more).

I say “for the most part”, because some functionality had been added to existing functions. E.g. before 25.1, sort couldn’t handle vectors. There are two ways to solve this issue:

1. Advise the relevant functions, wrapping them in the new functionality. E.g. for sort, this means checking if the input is actually a vector, and handling it appropriately, otherwise calling sort as one would before.
2. Create separate, “prefixed” functions that don’t interfere with the existing functionality. So if you want to sort a vector, you have to call compat-sort.

For now, compat is taking the latter option, but this might change for some functions in the future, if it is considered safe and popular (the shift from transparent to prefixed compatibility is more difficult). See the Request for comments section below for more details on this point.

If you do decide to use compat, I would recommend subscribing to the compat-announce mailing list.

Background

While I mentioned the idea last year, the necessity first became apparent when contributing to core ELPA packages that had to make sure they didn’t rely on functionality newer than the stated minimal version. This can be awkward, considering that packages like project, xref, flymake, … are developed as part of Emacs itself, in the emacs.git repository, but are also published as packages for ELPA.

Another motivation for compat is to provide a (IMO) stylistically more idiomatic alternative to using packages like dash, f, s and others. I see these being frequently used to make up for the lack of helpful utility functions in older releases. I have never been fond of this approach, for reasons discussed elsewhere. There is also the practical issue that s, f, ht, and other package are (consciously) neither available on GNU ELPA or NonGNU ELPA, yet packages frequently depend on these for trivial functionality. My hope is that compat might be able to help with this issue, and make it easier to add packages to the default repositories, leading to more packages being available OOTB.

Request for comments and the future of compat

As of writing (23Feb22) there are still a few open questions and discussions that other developers might be interested in participating. The reason I write this text before publishing the package, it to invite those who might have something to say on points such as these:

• The need of a special manual for compat?
• What functions should be transparently advised and what functions should be prefixed?
• Is it right for compat to update the default values of built-in packages (see this thread)?
• Should compat require itself⁷ or should it be explicitly required?
• Are there performance issues that should be improved?
• Is compat missing any functions that should be added?
• Should compat attempt to follow the upstream development more closely, and add functionality that hasn’t been added to a formal Emacs release?

If you have comments or any other questions, please send a message to the development mailing list!

Always Be Refining Your Config

I want to write about my third iteration on an org-roam. It’s goal is to address use-cases that I’ve encountered while moving more of my note-taking with org-roam.

One use-case is when I’m running or playing in an Role Playing Game (RPG 📖) session. During those sessions, when I create/find/insert nodes, I almost want to leverage the same tags. That can be in my capturing of nodes or in my searching for nodes. This is something I observed while running my 13 session “Thel Sector” campaign.

A second use-case is when I’m writing notes or thoughts related to work. In a past life, I might have written notes for either my employer or Samvera (a community in which I participated). Those notes might overlap but rarely did.

While I’m writing those notes, if I’m developing out concepts, I might want to filter my captures and searches to similar tags.

Another use case is less refined, namely I’m writing but am not “in” a specific context.

However, v2 of my org-roam structure, didn’t quite get out of the way. Iterating on my v2 org-roam setup was critical in learning more about Emacs. I will certainly reference my v2 org roam configuration as I continue my Emacs usage. I never quite got to the speed of note taking that I had for the original Thel Sector campaign.

What follows builds on Jethro Kuan’s How I Take Notes with Org-roam. Reading Jethro Kuan’s post helped me see how I could do this.

Preliminaries

The jf/org-roam-capture-templates-plist variable defines the possible org-roam capture templates that I will use. I have chosen to narrow these to three types:

refs

References to other people’s thoughts.

main

My thoughts, still churning, referencing other thoughts.

pubs

My thoughts, published and ready to share. Referencing any thoughts I’ve captured (and probably more).

Note: I chose to go with 4 character types to minimize it’s impact on rendering “type” in the search results (4 characters requires less visual space than 10 characters).

(defvar jf/org-roam-capture-templates-plist
  (list
   ;; These are references to "other people's thoughts."
   :refs '("r" "refs" plain "%?"
	   :if-new (file+head "refs/%<%Y%m%d>---${slug}.org" "#+title: ${title}\n#+FILETAGS:\n")
	   :unnarrowed t)
   ;; These are "my thoughts" with references to "other people's thoughts."
   :main '("m" "main" plain "%?"
	   :if-new (file+head "main/%<%Y%m%d>---${slug}.org"
			      "#+title: ${title}\n#+FILETAGS: ${auto-tags}\n")
	   :immediate-finish t
	   :unnarrowed t)
   ;; These are publications of "my thoughts" referencing "other people's thoughts".
   :pubs '("p" "pubs" plain "%?"
	   :if-new (file+head "pubs/%<%Y%m%d>---${slug}.org" "#+title: ${title}\n#+FILETAGS:\n")
	   :immediate-finish t
	   :unnarrowed t))
  "Templates to use for `org-roam' capture.")

The jf/org-context-plist defines and names some of the contexts in which I might be writing. Each named context defines the associated tags. These are the tags that all nodes will have when they are written in the defined context.

Loosely related is the jf/org-auto-tags--current-list; Contexts are a named set of tags. However, other functions don’t operate based on context. They instead operated based on the tags.

(defvar jf/org-context-plist
  (list
   :none
   (list
    :name "none"
    :tags (list))

   :burning-locusts
   (list
    :name "burning-locusts"
    :tags '("burning-locusts"
	    "rpg"
	    "burning-wheel"))

   :forem
   (list
    :name "forem"
    :tags '("forem"))

   :mistimed-scroll
   (list
    :name "mistimed-scroll"
    :tags '("eberron"
	    "mistimed-scroll"
	    "rpg"
	    "burning-wheel"))
   :thel-sector
   (list
    :name "thel-sector"
    :tags '("thel-sector"
	    "rpg" "swn")))
  "A list of contexts that I regularly write about.")

(defvar jf/org-auto-tags--current-list
  (list)
  "The list of tags to automatically apply to an `org-roam' capture.")

I can use jf/org-auto-tags--set to create an ad hoc context, or perhaps a “yet to be named” context. I can use jf/org-auto-tags--set-by-context to establish the current context (or clear it).

(defun jf/org-auto-tags--set (tags)
  "Prompt user or more TAGS."
  (interactive
   (list
    (completing-read-multiple
     "Tag(s): " (org-roam-tag-completions))))
  (setq jf/org-auto-tags--current-list tags))

(cl-defun jf/org-context-list-completing-read
    (&key
     (context-plist
      jf/org-context-plist))
  "Create a list of contexts from the CONTEXT-PLIST for completing read.

       The form should be '((\"forem\" 1) (\"burning-loscusts\" 2))."
  ;; Skipping the even entries as those are the "keys" for the plist,
  ;; the odds are the values.
  (-non-nil (seq-map-indexed
	     (lambda (context index)
	       (when (oddp index)
		 (list (plist-get context :name) index)))
	     context-plist)))

(cl-defun jf/org-auto-tags--set-by-context
    (context
     &key
     (context-plist jf/org-context-plist))
  "Set auto-tags by CONTEXT.

   Prompt for CONTEXT from CONTEXT-PLIST."
  (interactive
   (list
    (completing-read
     "Context: " (jf/org-context-list-completing-read))))
  (setq jf/org-auto-tags--current-list
	(plist-get
	 (plist-get
	  context-plist (intern (concat ":" context)))
	 :tags)))

With the jf/org-auto-tags--current-list variable set, I want a function to inject those tags onto my captures. Looking at the org-roam docs on template expansion, I want to create a function named org-roam-node-auto-tags.

(cl-defun org-roam-node-auto-tags
    (node
     &key
     (tag-list jf/org-auto-tags--current-list))
  "Inject the TAG-LIST into the {auto-tags} region of captured NODE.

     See https://www.orgroam.com/manual.html#Template-Walkthrough"
  (if (and tag-list (> (length tag-list) 0))
      (concat ":" (s-join ":" tag-list) ":")
    ""))

And finally, we have functions to use for establishing what templates are available based on the context, as well as what to setup as the default filter-fn for org-capture.

In other words, when I have set one or more tags, I want to use the templates appropriate for those tags and filter my org-roam-nodes so that only those nodes that have all of the tags are candidates.

(cl-defun jf/org-roam-templates-list
    (template
     &key
     (template-plist jf/org-roam-capture-templates-plist))
  "List of `org-roam' capture templates based on the given TEMPLATE.

     Searches the TEMPLATE-PLIST for the templates.

     Note, the :all template assumes we use the whole list."
  (if (eq template :all)
      (-non-nil
       (seq-map-indexed
	(lambda (tmp index)
	  (when (oddp index)
	    tmp))
	template-plist))
    (list (plist-get template-plist template))))

(cl-defun jf/org-roam-templates-context-fn
    (&key
     (tag-list jf/org-auto-tags--current-list))
  "Returns a set of templates based on TAG-LIST.

     A key assumption is that if there's a default tag list, use the
     :main template."
  (if (and tag-list (> (length tag-list) 0))
      (jf/org-roam-templates-list :main)
    (jf/org-roam-templates-list :all)))

(cl-defun jf/org-roam-filter-context-fn
    (node
     &key
     (tag-list jf/org-auto-tags--current-list))
  "Determine TAG-LIST is subset of NODE's tags."
  ;; gnus-subsetp is a more "permissive" version of subsetp.  It doesn't
  ;; consider order.  And looks at strings as equal if their values are the
  ;; same.
  (gnus-subsetp tag-list (org-roam-node-tags node)))

Configuration

I wrote three functions to mirror three core functions of org-mode:

• jf/org-roam-capture: find or create a node and file it away.
• jf/org-roam-node-insert: find or create a node and insert a link to that node. This is my “take notes quick” function.
• jf/org-roam-find-node: find a node and open that node in the frame.

For each of those functions, I establish the filter based on the current context and/or tags. I also limit the available capture templates based on the context.

(defun jf/org-roam-capture
    (&optional
     goto
     keys)
  "Call `org-roam-capture' based on set tags."
  (interactive "P")
  (org-roam-capture
   goto
   keys
   :filter-fn 'jf/org-roam-filter-context-fn
   :templates (jf/org-roam-templates-context-fn)))

(defun jf/org-roam-node-insert ()
  "Call `org-roam-node-insert' based on set tags."
  (interactive)
  (org-roam-node-insert
   'jf/org-roam-filter-context-fn
   :templates (jf/org-roam-templates-context-fn)))

(defun jf/org-roam-find-node
    (&optional
     other-window
     initial-input)
  "Call `org-roam-node-find' based on set tags."
  (interactive current-prefix-arg)
  (org-roam-node-find
   other-window
   initial-input
   'jf/org-roam-filter-context-fn
   :templates 'jf/org-roam-templates-context-fn))

And with all of that, let’s get into the org-roam configuration.

(use-package org-roam
  :straight t
  :config
  ;; I encountered the following message when attempting to export data:
  ;;
  ;; => "org-export-data: Unable to resolve link: EXISTING-PROPERTY-ID"
  ;;
  ;; See https://takeonrules.com/2022/01/11/resolving-an-unable-to-resolve-link-error-for-org-mode-in-emacs/ for details
  (defun jf/force-org-rebuild-cache ()
    "Call some functions to rebuild the `org-mode' and `org-roam' cache."
    (interactive)
    (org-id-update-id-locations)
    ;; Note: you may need `org-roam-db-clear-all' followed by `org-roam-db-sync'
    (org-roam-db-sync)
    (org-roam-update-org-id-locations))
  :custom
  (org-roam-directory (file-truename "~/git/org"))
  (org-roam-node-display-template
   (concat "${type:4}   ${title:*} "
	   (propertize "${tags:40}" 'face 'org-tag)))
  (org-roam-capture-templates (jf/org-roam-templates-list :all))
  :bind (("C-s-f" . jf/org-roam-find-node)
	 ("C-s-c" . jf/org-roam-capture))
  :bind (:map org-mode-map
	      (
	       ("C-s-;" . org-roam-buffer-toggle)
	       ("s-i" . jf/org-roam-node-insert)))
  :init
  ;; Help keep the `org-roam-buffer', toggled via `org-roam-buffer-toggle', sticky.
  (add-to-list 'display-buffer-alist
	       '("\\*org-roam\\#"
		 (display-buffer-in-side-window)
		 (side . right)
		 (slot . 0)
		 (window-width . 0.33)
		 (window-parameters . ((no-other-window . t)
				       (no-delete-other-windows . t)))))
  ;; When t the autocomplete in org documents will query the org roam database
  (setq org-roam-completion-everywhere t)
  (setq org-roam-v2-ack t)
  (org-roam-db-autosync-mode))

;; This needs to be after the `org-roam’ declaration as it is dependent on the
;; structures of `org-roam'.
(cl-defmethod org-roam-node-type ((node org-roam-node))
  "Return the TYPE of NODE."
  (condition-case nil
      (file-name-nondirectory
       (directory-file-name
	(file-name-directory
	 (file-relative-name
	  (org-roam-node-file node)
	  org-roam-directory))))
    (error "")))

All told, the past experience when running New Vistas in the Thel Sector // Take on Rules informed how I thought about my note taking.

Other Contexts

Try as I may, based on my configuration, I can’t get org-protocol to work. So I’ve opted to take a different path; write some Emacs functions instead.

• jf/org-roam-capture-ref: Capture a “refs” context org-roam-node for the given title and url.
• jf/menu-dwim--org-capture-elfeed-show: Capture an RSS entry.
• jf/menu-dwim--org-capture-firefox: Capture the active tab of Firefox.
• jf/menu-dwim--org-capture-safari: Capture the active tab of Safari.

These tie into my the context and auto-tags.

(cl-defun jf/org-roam-capture-ref (&key title url)
  "Capture the TITLE and URL in the `org-roam' :refs template"
  ;; If your installation of org-roam includes the fix fore
  ;; https://github.com/org-roam/org-roam/issues/2078 then you can leave the
  ;; below commented out.
  ;;
  ;; This looks a bit odd, but to capture the :ref we need the callback from org-roam.
  ;; (require 'org-roam-protocol)
  ;;
  (org-roam-capture-
   :keys "r"
   ;; TODO: I would love to get tags working but I'm missing something
   :node (org-roam-node-create :title title)
   :info (list :ref url)
   :props '(:immediate-finish nil)
   :templates (jf/org-roam-templates-list :refs)))

(cl-defun jf/menu-dwim--org-capture-elfeed-show (&key (entry elfeed-show-entry))
  "Create an `org-roam-node' from elfeed ENTRY."
  (interactive)
  (let ((url (elfeed-entry-link entry))
	(title (elfeed-entry-title entry)))
    (jf/org-roam-capture-ref :url url :title title)))

(defun jf/menu-dwim--org-capture-firefox ()
  "Create an `org-roam-node' from Firefox page.

  Depends on the `grab-mac-link' package."
  (interactive)
  (let* ((link-title-pair (grab-mac-link-firefox-1))
	 (url (car link-title-pair))
	 (title (cadr link-title-pair)))
    (jf/org-roam-capture-ref :url url :title title)))

(defun jf/menu-dwim--org-capture-safari ()
  "Create an `org-roam-node' from Safari page.

  Depends on the `grab-mac-link' package."
  (interactive)
  (let* ((link-title-pair (grab-mac-link-safari-1))
	 (url (car link-title-pair))
	 (title (cadr link-title-pair)))
    (jf/org-roam-capture-ref :url url :title title)))

(defun jf/menu-dwim--org-capture-eww ()
  "Create an `org-roam-node' from `eww' data"
  (interactive)
  (let* ((url (plist-get eww-data :url))
	 (title (plist-get eww-data :title)))
    (jf/org-roam-capture-ref :url url :title title)))

Conclusion

This is the core of my note taking engine. It builds on the idea that I want to reduce the number of decisions I make. This is extremely important when I’m writing session notes.

While I’m playing in a session, my entire context ideally collapses to the relevant tags that I’ve established at the beginning of the session. That way I’m certain that I’m filing away notes to their proper location.