2024-09-18 Emacs Wiki and China

2024-09-14. I’m somewhere in the Italian-speaking parts of Switzerland with my wife. There is a lot of running, hiking, hugging, kissing, eating and drinking involved. 🥰

2024-09-15. Still on the trip but late at night I spent more than an hour trying to figure out why my server had a load of nearly 40. 💻

All I discovered is that load went down when I shut down Emacs Wiki. See also 2024-09-16 on Emacs Wiki.

Well, I needed to sleep and I’ve got plans for the next few days so I shut it down while I slept hoping that the misconfigured spider is fixed or the inept programmer discovers their mistake. Just another day in the Butlerian Jihad. Some misguided soul probably wanted to download it all and wrote a broken web crawler and when that got blocked they bought some nice scaling infrastructure from Amazon, Hetzner, OHV or Alibaba Cloud or whatever they are called, allowing them to use a gazillion different IP numbers that will eventually lead me to implement some sort of cloud service provider block.

2024-09-16. Switched Emacs Wiki back on after a few hours of sleep and it did fine. But then it restarted again… at 18:00, 19:00, 21:00, 22:00… and so I switched Emacs Wiki off again. Time to ban some networks!

Anybody interested in my banning of IP ranges and possibly interested in me reverting any of these, take a look at ban-cidr … from a network that isn’t banned, I guess. 😏

2024-09-17. This continues to keep me busy and angry every evening. Too bad I don’t have a real fast network-lookup to firewall ban pipeline. I’m using this script instead of carefully checking IP numbers and networks. I’m also sick and tired of the same networks popping up again and again.

I added over a hundred Chinese networks to the firewall rules and I’m seriously considering blocking the whole country for a week. It seems that most of the offenders are networks run by China Telecoms and China Mobile.

2024-09-18. So far, so good. Load stays below two.

Here’s example usage for network-lookup, filtering for Emacs Wiki and a URL parameter used when requesting recent changes or a RSS feed for a single page only. That would count as suspicious misbehaving crawler behaviour in my book.

Made 16 DNS requests.
4 cache hits.

34.32.128.0/17 Google LLC / GOOGL-2
34.32.172.122 34.32.172.122 34.32.172.122
34.32.128.0/17 | 34.32.172.122 | 18/Sep/2024:18:02:12 +0200 | GET /cgi-bin/emacs?action=rss;rcidonly=RssExclude;days=1;all=1;showedit=1;full=1 HTTP/1.1 | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36
34.32.128.0/17 | 34.32.172.122 | 18/Sep/2024:18:02:12 +0200 | GET /cgi-bin/emacs?action=rss;rcidonly=RssExclude;days=1;all=1;showedit=1;full=1 HTTP/1.1 | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36
34.32.128.0/17 | 34.32.172.122 | 18/Sep/2024:18:02:12 +0200 | GET /emacs?action=rss;rcidonly=RssExclude;days=1;all=1;showedit=1;full=1 HTTP/1.1 | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36
ipset add banlist 34.32.128.0/17

113.0.0.0/13 UNICOM-HL / CNC Group CHINA169 Heilongjiang Province Network
113.2.156.201 113.2.184.52 113.1.92.78
113.0.0.0/13 | 113.2.156.201 | 18/Sep/2024:18:02:11 +0200 | GET /emacs?action=rc&all=1&days=14&rcidonly=tzhelp.el&showedit=0 HTTP/1.1 | Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 ADG/11.0.2566 AOLBUILD/11.0.2566 Safari/537.36
113.0.0.0/13 | 113.2.184.52 | 18/Sep/2024:18:02:16 +0200 | GET /emacs?action=rc&all=1&days=7&rcidonly=WriteRoom&showedit=1 HTTP/1.1 | Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Brave Chrome/88.0.4324.152 Safari/537.36
113.0.0.0/13 | 113.1.92.78 | 18/Sep/2024:18:02:21 +0200 | GET /emacs?action=rc&all=1&from=1&rcidonly=screen-term.el&showedit=1 HTTP/1.1 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.63 Safari/537.36
ipset add banlist 113.0.0.0/13

14.208.0.0/12 CHINANET-GD / China Telecom
14.216.128.162
14.208.0.0/12 | 14.216.128.162 | 18/Sep/2024:18:02:27 +0200 | GET /emacs?action=rc&all=1&from=1&rcidonly=SyncBBDB&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
ipset add banlist 14.208.0.0/12

39.64.0.0/11 China Unicom Shandong Province Network / UNICOM-SD
39.76.103.29
39.64.0.0/11 | 39.76.103.29 | 18/Sep/2024:18:02:17 +0200 | GET /emacs?action=rc&all=1&days=1&rcidonly=FinderMode&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36
ipset add banlist 39.64.0.0/11

111.36.192.0/20 China Mobile communications corporation / China Mobile / CMNET / ORG-CM1-AP
111.36.200.78
111.36.192.0/20 | 111.36.200.78 | 18/Sep/2024:18:01:50 +0200 | GET /emacs?action=rc&all=1&days=28&rcidonly=DiredPlusMarkMenu&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36
ipset add banlist 111.36.192.0/20

223.167.0.0/16 UNICOM-SH / China Unicom Shanghai Province Network
223.167.74.168
223.167.0.0/16 | 223.167.74.168 | 18/Sep/2024:18:01:39 +0200 | GET /emacs?action=rc&all=1&from=1726266329&rcidonly=etagsselect&showedit=1 HTTP/1.1 | Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
ipset add banlist 223.167.0.0/16

39.184.0.0/18 Internet Service Provider in China / CMNET / ORG-CM1-AP / China Mobile
39.184.45.222
39.184.0.0/18 | 39.184.45.222 | 18/Sep/2024:18:02:28 +0200 | GET /emacs?action=rss&all=1&days=14&rcidonly=BufferMenuPlus&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Brave Chrome/81.0.4044.122 Safari/537.36
ipset add banlist 39.184.0.0/18

111.18.128.0/20 China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
111.18.142.15
111.18.128.0/20 | 111.18.142.15 | 18/Sep/2024:18:02:00 +0200 | GET /emacs?action=rc&all=1&from=1&rcidonly=orgfold-separate-file.el&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36
ipset add banlist 111.18.128.0/20

27.38.200.0/24 UNICOM-GDSZ / China Unicom
27.38.200.210
27.38.200.0/24 | 27.38.200.210 | 18/Sep/2024:18:01:57 +0200 | GET /emacs?action=rc&all=1&from=1723764629&rcidonly=AnselmHelbig&showedit=1&upto=1724974229 HTTP/1.1 | Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36
ipset add banlist 27.38.200.0/24

112.47.128.0/18 China Mobile communications corporation / China Mobile Communications Corporation / ORG-CMCC1-AP / CMNET
112.47.136.104
112.47.128.0/18 | 112.47.136.104 | 18/Sep/2024:18:01:28 +0200 | GET /emacs?action=rc&all=0&days=14&rcidonly=RubyEvalRegion&showedit=1 HTTP/1.1 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.89 Safari/537.36
ipset add banlist 112.47.128.0/18

218.71.0.0/16 CHINANET-ZJ-WZ / Zhejiang Telecom
218.71.59.116
218.71.0.0/16 | 218.71.59.116 | 18/Sep/2024:18:01:55 +0200 | GET /emacs?action=rc&all=1&from=1726412272&rcidonly=AnilTappetla&showedit=1 HTTP/1.1 | Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36
ipset add banlist 218.71.0.0/16

39.130.48.0/20 CMNET / ORG-CM1-AP / China Mobile / Internet Service Provider in China
39.130.50.65
39.130.48.0/20 | 39.130.50.65 | 18/Sep/2024:18:02:28 +0200 | GET /emacs?action=rc&all=1&days=7&rcidonly=CarlMikkelsen&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36
ipset add banlist 39.130.48.0/20

222.90.0.0/16 CHINANET-SN / Beijing 100088
222.90.194.237
222.90.0.0/16 | 222.90.194.237 | 18/Sep/2024:18:01:57 +0200 | GET /emacs?action=rc&all=1&from=1&rcidonly=AnIntroductionToTheEmacsEditor&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36
ipset add banlist 222.90.0.0/16

223.81.240.0/20 ORG-CM1-AP / CMNET / China Mobile / China Mobile communications corporation
223.81.249.112
223.81.240.0/20 | 223.81.249.112 | 18/Sep/2024:18:01:43 +0200 | GET /emacs?action=rc&all=1&from=1726333566&rcidonly=iclects_-_search_search_commands,_overview&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.9999.0 Safari/537.36
ipset add banlist 223.81.240.0/20

218.107.192.0/19 CNCGROUP-FJ-XIAMEN-MAN / CNCGroup CHINA169 FuJian province network
218.107.204.27
218.107.192.0/19 | 218.107.204.27 | 18/Sep/2024:18:01:57 +0200 | GET /emacs?action=rc&all=1&from=1&rcidonly=DiredPlusPopupRegionRemoveRectMenu&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Brave Chrome/88.0.4324.152 Safari/537.36
ipset add banlist 218.107.192.0/19

219.128.0.0/13 CHINANET-GD / China Telecom
219.133.249.75
219.128.0.0/13 | 219.133.249.75 | 18/Sep/2024:18:02:11 +0200 | GET /emacs?action=rc&all=1&days=1&rcidonly=Categor%C3%ADaHerramientasOnline&showedit=1 HTTP/1.1 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36
ipset add banlist 219.128.0.0/13

As far as I am concerned, all deserve to be banned. Over-banning? Maybe. What do you think?

For demonstration purposes, this is what I ran:

ssh sibirocobombus.root grep '"^www.emacswiki.org.*rcidonly"' /var/log
/apache2/access.log \
| tail -n 20 \
| ssh sibirocobombus.root bin/admin/network-lookup \
| wl-copy

So now I’m ready to ban them all:

wl-paste | grep ipset

#Emacs #Butlerian Jihad #Administration

2024-10-01. Today I came back to the server with load at 40. Again! Here’s me filtering the log for requests that got a server error status code (500–509).

root@sibirocobombus ~# tail -n 20000 /var/log/apache2/access.log \
| grep -v ^social \
| grep " 50[0-9] " \
| tail -n 50 \
| bin/admin/network-lookup
Made 44 DNS requests.
6 cache hits.

I think the only one I didn’t ban was the unknown one, Gwene and Bing (Microsoft).

Load is now at around 31.

2024-10-01. Getting lazier… Pick a suspicious pattern and check for China…

root@sibirocobombus ~# tail -n 500 /var/log/apache2/access.log \
 | grep -v ^social \
 | grep "action=browse&id=" \
 | tail -n 50 \
 | bin/admin/network-lookup > result.log
root@sibirocobombus ~# grep ipset result.log 
ipset add banlist 27.16.0.0/12 # CHINANET-HB / China Telecom
ipset add banlist 219.145.0.0/16 # CHINANET-SN / Beijing 100032
ipset add banlist 116.179.0.0/16 # UNICOM / China Unicom CHINA169 Network
ipset add banlist 221.219.0.0/18 # China Unicom Beijing Province Network / UNICOM-BJ
ipset add banlist 119.128.0.0/12 # China Telecom / CHINANET-GD
ipset add banlist 124.234.0.0/15 # CHINANET-JL / China Telecom
ipset add banlist 101.80.0.0/13 # Beijing 100032 / CHINANET-SH
ipset add banlist 111.41.128.0/17 # China Mobile communications corporation / CMNET / China Mobile / ORG-CM1-AP
ipset add banlist 111.14.0.0/20 # ORG-CM1-AP / China Mobile / CMNET / China Mobile communications corporation
ipset add banlist 110.176.0.0/13 # shanxi telecom linfen branch ip node links to customer ip address / sxlfbas
ipset add banlist 221.232.0.0/14 # Beijing 100088 / CHINANET-HB
ipset add banlist 116.4.0.0/14 # Beijing 100032 / CHINANET-GD
ipset add banlist 111.41.16.0/22 # ORG-CM1-AP / China Mobile / China Mobile communications corporation / CMNET
ipset add banlist 36.44.0.0/15 # CHINANET-SN / Beijing 100032
ipset add banlist 112.10.192.0/18 # China Mobile communications corporation / CMNET / ORG-CMCC1-AP / China Mobile Communications Corporation
ipset add banlist 117.154.240.0/20 # China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
ipset add banlist 111.43.58.0/23 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 111.19.0.0/17 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 183.229.0.0/16 # ORG-CMCC1-AP / China Mobile Communications Corporation / CMNET / China Mobile communications corporation
ipset add banlist 223.89.128.0/18 # CMNET / China Mobile communications corporation / ORG-CM1-AP / China Mobile
ipset add banlist 42.184.0.0/16 # NO.178 Zhongshan Road,Haerbin,Heilongjiang 150040 / CHINANET-HL
ipset add banlist 60.162.0.0/15 # CHINANET-ZJ-TZ / Zhejiang Telecom
ipset add banlist 112.8.32.0/19 # China Mobile communications corporation / CMNET / ORG-CMCC1-AP / China Mobile Communications Corporation
ipset add banlist 39.129.248.0/21 # Internet Service Provider in China / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 36.46.0.0/16 # CHINANET-SN / Beijing 100032
ipset add banlist 125.76.128.0/17 # From Shanxi(CHINANET-SN) Network of ChinaTelecom / CHINANET-SN
ipset add banlist 123.121.128.0/18 # China Unicom Beijing Province Network / UNICOM-BJ
ipset add banlist 140.75.0.0/16 # CHINANET-SD / Beijing 100032
ipset add banlist 112.88.0.0/13 # UNICOM-GD / China Unicom CHINA169 Guangdong Province Network
ipset add banlist 112.49.192.0/18 # ORG-CMCC1-AP / China Mobile Communications Corporation / CMNET / China Mobile communications corporation
ipset add banlist 113.204.0.0/14 # UNICOM-CQ / CNC Group CHINA169 Chongqing Province Network
ipset add banlist 123.149.0.0/16 # CHINANET-HA / Beijing 100032
ipset add banlist 60.14.0.0/15 # CNC Group CHINA169 Heilongjiang Province Network / UNICOM-HL
ipset add banlist 114.92.0.0/17 # Beijing 100032 / CHINANET-SH
ipset add banlist 1.56.0.0/13 # China Unicom Heilongjiang Province Network / UNICOM-HL
ipset add banlist 111.14.64.0/21 # ORG-CM1-AP / China Mobile / CMNET / China Mobile communications corporation
ipset add banlist 123.128.0.0/13 # CNC Group CHINA169 Shandong Province Network / UNICOM-SD
ipset add banlist 119.120.0.0/13 # China Telecom / CHINANET-GD
ipset add banlist 222.68.0.0/16 # CHINANET-SH / Beijing 100032
ipset add banlist 36.248.0.0/14 # UNICOM-FJ-QUANZHOU-MAN / China Unicom Fujian Province Network
ipset add banlist 221.0.0.0/15 # UNICOM-SD / CNC Group CHINA169 Shandong Province Network
ipset add banlist 113.56.0.0/15 # CNC Group CHINA169 Hubei Province Network / UNICOM-HB
ipset add banlist 223.78.192.0/18 # China Mobile / China Mobile communications corporation / CMNET / ORG-CM1-AP
ipset add banlist 112.2.128.0/19 # ORG-CMCC1-AP / CMNET / China Mobile communications corporation / China Mobile Communications Corporation
ipset add banlist 183.227.0.0/16 # CMNET / ORG-CMCC1-AP / China Mobile Communications Corporation / China Mobile communications corporation
ipset add banlist 106.46.0.0/16 # CHINANET henan province network / CHINANET-HA
ipset add banlist 123.8.0.0/13 # CNC Group CHINA169 Henan Province Network / UNICOM-HA
ipset add banlist 219.136.0.0/15 # CHINANET-GD / China Telecom
ipset add banlist 111.172.0.0/14 # CHINANET-HB / Beijing 100032
ipset add banlist 118.112.0.0/13 # Beijing 100088 / CHINANET-SC
ipset add banlist 27.148.0.0/14 # Beijing 100032 / CHINANET-FJ
ipset add banlist 153.101.0.0/16 # UNICOM-JS / China Unicom Jiangsu Province Network
ipset add banlist 221.237.0.0/16 # CHINANET-SC / Beijing 100088
ipset add banlist 121.28.0.0/15 # UNICOM-HE / CNC Group CHINA169 Hebei Province Network
ipset add banlist 61.241.192.0/19 # UNICOM / Beijing 100140 ,P.R.China
ipset add banlist 114.86.128.0/17 # CHINANET-SH / Beijing 100032
ipset add banlist 223.67.32.0/19 # CMNET / ORG-CM1-AP / China Mobile / China Mobile communications corporation
ipset add banlist 122.136.0.0/13 # UNICOM-JL / CNC Group CHINA169 Jilin Province Network
ipset add banlist 27.36.0.0/14 # China Unicom Guangdong Province Network / UNICOM-GDDG
ipset add banlist 117.147.0.0/17 # China Mobile / China Mobile communications corporation / ORG-CM1-AP / CMNET
ipset add banlist 153.3.0.0/16 # UNICOM-JS / China Unicom Jiangsu Province Network
ipset add banlist 14.112.0.0/12 # China Telecom / CHINANET-GD
ipset add banlist 183.253.0.0/19 # China Mobile Communications Corporation / China Mobile Communications Corporation / ORG-CMCC1-AP / CMNET
ipset add banlist 183.228.0.0/16 # China Mobile communications corporation / China Mobile Communications Corporation / ORG-CMCC1-AP / CMNET
ipset add banlist 221.196.0.0/15 # UNICOM-TJ / CNC Group CHINA169 Tianjin Province Network
ipset add banlist 111.196.192.0/18 # UNICOM-BJ / China Unicom Beijing Province Network
ipset add banlist 14.107.172.0/22 # CHINANET-CQ / China Telecom
ipset add banlist 114.216.66.0/23 # Beijing 100032 / CHINANET-JS
ipset add banlist 112.1.32.0/19 # China Mobile communications corporation / China Mobile Communications Corporation / CMNET / ORG-CMCC1-AP
ipset add banlist 221.182.0.0/18 # China Mobile communications corporation / China Mobile / CMNET / ORG-CM1-AP
ipset add banlist 118.72.0.0/13 # CNC Group CHINA169 Shanxi Province Network / sxyc-yongji-BAS
ipset add banlist 58.59.0.0/17 # No.999,Shunhua road,Jinan,Shandong / CHINANET-SD
ipset add banlist 111.1.96.0/20 # China Mobile / China Mobile communications corporation / ORG-CM1-AP / CMNET
ipset add banlist 36.104.0.0/16 # China Telecom / CHINANET-ZJ
ipset add banlist 117.151.16.0/20 # China Mobile / China Mobile communications corporation / CMNET / ORG-CM1-AP
ipset add banlist 119.34.128.0/17 # GZPRBNET / GuangDong, China 510010
ipset add banlist 112.0.192.0/19 # China Mobile Communications Corporation / China Mobile communications corporation / ORG-CMCC1-AP / CMNET
ipset add banlist 111.1.192.0/19 # CMNET / ORG-CM1-AP / China Mobile / China Mobile communications corporation
ipset add banlist 183.225.0.0/19 # CMNET / ORG-CMCC1-AP / China Mobile communications corporation / China Mobile Communications Corporation
ipset add banlist 58.60.0.0/14 # Beijing 100032 / CHINANET-GD
ipset add banlist 117.24.0.0/13 # CHINANET-FJ / 7,East Street ,Fuzhou ,Fujian ,PRC
ipset add banlist 112.24.224.0/19 # CMNET / ORG-CMCC1-AP / China Mobile Communications Corporation / China Mobile communications corporation
ipset add banlist 39.130.64.0/19 # Internet Service Provider in China / China Mobile / CMNET / ORG-CM1-AP
ipset add banlist 119.164.0.0/14 # UNICOM-SD / CNC Group CHINA169 Shandong Province Network
ipset add banlist 39.190.96.0/21 # China Mobile / Internet Service Provider in China / CMNET / ORG-CM1-AP
ipset add banlist 111.224.0.0/14 # Beijing 100032 / CHINANET-HE
ipset add banlist 112.0.128.0/19 # China Mobile Communications Corporation / China Mobile communications corporation / CMNET / ORG-CMCC1-AP
ipset add banlist 211.103.72.0/21 # China Mobile / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 1.202.0.0/17 # CHINANET-BJ / CHINANET Beijing Province Network
ipset add banlist 39.149.80.0/20 # CMNET / Internet Service Provider in China / ORG-CM1-AP / China Mobile
ipset add banlist 218.10.0.0/16 # UNICOM-HL / CNC Group CHINA169 Heilongjiang Province Network
ipset add banlist 36.143.46.0/23 # CMNET / China Mobile Communications Corporation / ORG-CMCC1-AP / China Mobile Communications Corporation
ipset add banlist 223.97.160.0/19 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 111.14.96.0/19 # China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
ipset add banlist 122.240.0.0/13 # CHINANET-ZJ-WZ / Zhejiang Telecom
ipset add banlist 120.80.0.0/13 # UNICOM-GD / CNC Group CHINA169 Guangdong Province Network
ipset add banlist 120.85.182.0/23 # China Unicom / GuangZhou-unicom
ipset add banlist 171.43.0.0/16 # China Telecom / CHINANET-HB
ipset add banlist 117.144.0.0/16 # CMNET-shanghai / China Mobile communications corporation
ipset add banlist 117.179.107.0/24 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 117.179.122.0/23 # CMNET / China Mobile communications corporation / ORG-CM1-AP / China Mobile
ipset add banlist 112.20.224.0/19 # CMNET / China Mobile communications corporation / ORG-CMCC1-AP / China Mobile Communications Corporation
ipset add banlist 111.35.224.0/20 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 223.106.128.0/18 # CMNET / China Mobile communications corporation / ORG-CM1-AP / China Mobile
ipset add banlist 61.156.0.0/16 # UNICOM-CN / CNC Group CHINA169 Shandong Province Network
ipset add banlist 111.48.77.0/24 # China Mobile / CMNET / China Mobile communications corporation / ORG-CM1-AP
ipset add banlist 36.40.0.0/14 # CHINANET-SN / Beijing 100032

2024-10-02. The situation is under control again, but since I’m vindictive, I’ll block some more.

tail -n 500 /var/log/apache2/access.log \
 | grep -v ^social \
 | grep "rcidonly=" \
 | tail -n 50 \
 | bin/admin/network-lookup > result.log
grep ipset result.log
ipset add banlist 101.16.0.0/12 # CNCGROUP-HE / China Unicom Hebei Province Network
ipset add banlist 111.18.32.0/20 # China Mobile / CMNET / China Mobile communications corporation / ORG-CM1-AP
ipset add banlist 36.16.0.0/12 # CHINANET-ZJ-TZ / Zhejiang Telecom
ipset add banlist 112.20.96.0/19 # CMNET / China Mobile communications corporation / ORG-CMCC1-AP / China Mobile Communications Corporation
ipset add banlist 223.67.224.0/19 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 36.132.168.0/22 # China Mobile Communications Corporation / CMNET / China Mobile Communications Corporation / ORG-CMCC1-AP
ipset add banlist 123.120.0.0/18 # China Unicom Beijing Province Network / UNICOM-BJ
ipset add banlist 111.43.144.0/24 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 116.231.0.0/16 # Beijing 100032 / CHINANET-SH
ipset add banlist 39.149.0.0/20 # ORG-CM1-AP / CMNET / Internet Service Provider in China / China Mobile
ipset add banlist 223.73.84.0/22 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 218.8.0.0/15 # UNICOM-HL / CNC Group CHINA169 Heilongjiang Province Network
ipset add banlist 36.132.166.0/23 # China Mobile Communications Corporation / ORG-CMCC1-AP / China Mobile Communications Corporation / CMNET
ipset add banlist 58.22.0.0/15 # CNCGROUP-FJ-XIAMEN-MAN / CNCGroup FuJian province network
ipset add banlist 117.143.52.0/22 # China Mobile communications corporation / CMNET-shanghai
ipset add banlist 111.41.34.0/24 # ORG-CM1-AP / China Mobile communications corporation / CMNET / China Mobile
ipset add banlist 125.40.0.0/13 # UNICOM-HA / CNC Group CHINA169 Henan Province Network
ipset add banlist 171.40.0.0/15 # China Telecom / CHINANET-HB
ipset add banlist 112.64.0.0/17 # Addresses from APNIC / UNICOM-SH
ipset add banlist 117.179.181.0/24 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 223.66.192.0/19 # China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
ipset add banlist 111.33.0.0/17 # China Mobile / CMNET / China Mobile communications corporation / ORG-CM1-AP
ipset add banlist 101.44.160.0/20 # Huawei-Cloud-SG / HUAWEI INTERNATIONAL PTE. LTD.
ipset add banlist 36.106.148.0/23 # CHINANET TIANJIN PROVINCE NETWORK / CHINANET-TJ
ipset add banlist 117.179.33.0/24 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 124.114.0.0/16 # Beijing 100088 / CHINANET-SN
ipset add banlist 223.89.192.0/18 # ORG-CM1-AP / China Mobile communications corporation / CMNET / China Mobile
ipset add banlist 118.248.0.0/13 # Beijing 100032 / CHINANET-HN
ipset add banlist 223.81.128.0/19 # China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
ipset add banlist 223.74.216.0/22 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 111.41.14.0/23 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 223.74.103.0/24 # China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
ipset add banlist 223.74.72.0/21 # CMNET / China Mobile communications corporation / ORG-CM1-AP / China Mobile
ipset add banlist 39.130.96.0/19 # China Mobile / CMNET / Internet Service Provider in China / ORG-CM1-AP
ipset add banlist 221.228.192.0/20 # CHINANET-JS / Beijing 100088
ipset add banlist 180.175.0.0/16 # CHINANET-SH / Beijing 100032
ipset add banlist 112.100.0.0/14 # NO.178 Zhongshan Road,Haerbin,Heilongjiang 150040 / CHINANET-HL
ipset add banlist 223.95.192.0/19 # China Mobile / China Mobile communications corporation / CMNET / ORG-CM1-AP
ipset add banlist 111.34.192.0/20 # China Mobile communications corporation / CMNET / ORG-CM1-AP / China Mobile
ipset add banlist 111.36.160.0/20 # ORG-CM1-AP / CMNET / China Mobile communications corporation / China Mobile
ipset add banlist 111.17.64.0/19 # ORG-CM1-AP / China Mobile communications corporation / CMNET / China Mobile
ipset add banlist 101.69.0.0/16 # UNICOM-ZJ / China Unicom Zhejiang Province Network
ipset add banlist 112.116.0.0/15 # CHINANET-YN / Beijing 100032
ipset add banlist 111.14.144.0/20 # China Mobile / ORG-CM1-AP / China Mobile communications corporation / CMNET
ipset add banlist 111.18.0.0/16 # China Mobile / ORG-CM1-AP / CMNET / China Mobile communications corporation
ipset add banlist 220.192.0.0/18 # China Unicom Network / UNICOM
ipset add banlist 123.118.0.0/18 # China Unicom Beijing Province Network / UNICOM-BJ

2024-10-10. Load is at 30 again. Another round!

ipset add banlist 14.208.0.0/12 # CHINANET-GD / China Telecom
ipset add banlist 113.0.0.0/13 # CNC Group CHINA169 Heilongjiang Province Network / UNICOM-HL
ipset add banlist 223.81.240.0/20 # China Mobile communications corporation / ORG-CM1-AP / China Mobile / CMNET
ipset add banlist 112.38.0.0/16 # ORG-CMCC1-AP / China Mobile communications corporation / CMNET / China Mobile Communications Corporation
ipset add banlist 222.129.0.0/18 # UNICOM-BJ / China Unicom Beijing Province Network
ipset add banlist 117.176.208.0/21 # CMNET / China Mobile / China Mobile communications corporation / ORG-CM1-AP
ipset add banlist 153.34.0.0/15 # UNICOM-JS / China Unicom Jiangsu Province Network
ipset add banlist 116.149.128.0/17 # China Unicom CHINA169 Network / UNICOM
ipset add banlist 182.112.0.0/12 # China Unicom Henan Province Network / UNICOM-HA
ipset add banlist 112.51.16.0/20 # China Mobile communications corporation / ORG-CMCC1-AP / China Mobile Communications Corporation / CMNET
ipset add banlist 39.190.112.0/21 # China Mobile / CMNET / Internet Service Provider in China / ORG-CM1-AP
ipset add banlist 112.65.0.0/17 # UNICOM-SH / Addresses from APNIC
ipset add banlist 223.88.192.0/18 # China Mobile / CMNET / China Mobile communications corporation / ORG-CM1-AP
ipset add banlist 111.33.192.0/18 # CMNET / China Mobile / ORG-CM1-AP / China Mobile communications corporation
ipset add banlist 114.88.0.0/17 # CHINANET-SH / Beijing 100032
ipset add banlist 36.32.0.0/14 # UNICOM-AH / China Unicom Anhui Province Network
ipset add banlist 61.174.0.0/15 # YINXIAN-MINGHUI-LTD / Ningbo Minghui International Trade Co.,Ltd
ipset add banlist 112.98.0.0/15 # NO.178 Zhongshan Road,Haerbin,Heilongjiang 150040 / CHINANET-HL
ipset add banlist 39.163.64.0/20 # China Mobile / CMNET / ORG-CM1-AP / Internet Service Provider in China
ipset add banlist 117.179.8.0/21 # ORG-CM1-AP / China Mobile communications corporation / China Mobile / CMNET
ipset add banlist 117.179.249.0/24 # ORG-CM1-AP / China Mobile communications corporation / China Mobile / CMNET
ipset add banlist 42.185.0.0/16 # CHINANET-HL / NO.178 Zhongshan Road,Haerbin,Heilongjiang 150040
ipset add banlist 218.58.0.0/15 # CNC Group CHINA169 Shandong Province Network / UNICOM-SD
ipset add banlist 211.143.180.0/22 # CMNET-fujian / China Mobile communications corporation
ipset add banlist 223.167.0.0/16 # China Unicom Shanghai Province Network / UNICOM-SH
ipset add banlist 112.49.128.0/17 # China Mobile communications corporation / ORG-CMCC1-AP / China Mobile Communications Corporation / CMNET
ipset add banlist 223.96.128.0/20 # China Mobile / CMNET / ORG-CM1-AP / China Mobile communications corporation
ipset add banlist 117.152.240.0/20 # China Mobile communications corporation / ORG-CM1-AP / CMNET / China Mobile
ipset add banlist 117.179.38.0/23 # China Mobile / CMNET / ORG-CM1-AP / China Mobile communications corporation
ipset add banlist 39.148.48.0/20 # China Mobile / CMNET / ORG-CM1-AP / Internet Service Provider in China
ipset add banlist 223.80.192.0/18 # ORG-CM1-AP / China Mobile communications corporation / CMNET / China Mobile
ipset add banlist 112.20.64.0/19 # CMNET / China Mobile Communications Corporation / China Mobile communications corporation / ORG-CMCC1-AP
ipset add banlist 144.52.0.0/16 # Beijing 100032 / CHINANET-SD
ipset add banlist 36.100.96.0/22 # China Telecom / CHINANET-ZJ
ipset add banlist 60.168.0.0/13 # CHINANET-AH / Beijing 100088
ipset add banlist 61.159.192.0/18 # CHINANET Yunnan province network / CHINANET-YN
ipset add banlist 125.36.0.0/14 # CNC Group CHINA169 Tianjin Province Network / UNICOM-TJ
ipset add banlist 124.236.0.0/14 # CHINANET-HE / Beijing 100032
ipset add banlist 183.226.0.0/16 # China Mobile communications corporation / ORG-CMCC1-AP / CMNET / China Mobile Communications Corporation
ipset add banlist 223.166.0.0/16 # UNICOM-SH / China Unicom Shanghai Province Network
ipset add banlist 218.93.0.0/16 # CHANGZHOU-Xinhua-bookshop-CORP / Jiangsu Province
ipset add banlist 39.183.128.0/19 # China Mobile / CMNET / Internet Service Provider in China / ORG-CM1-AP
ipset add banlist 180.160.0.0/13 # CHINANET-SH / Beijing 100032
ipset add banlist 223.73.56.0/21 # China Mobile communications corporation / ORG-CM1-AP / CMNET / China Mobile

In my previous post about creating simple functions to create a new directory and file in dired, I received an interesting comment suggesting another way to bypass the potential messiness of completion and pass through the literal text input.

My last post :

Simple Directory and File Creation in Dired

It is a subtlety of most if not all completion systems, and that is to exit with any input, hence exiting without having to necessarily match and complete.

Given this, and given that I’m using the completion framework of vertico, I thought I would revisit the vertico github manual : https://github.com/minad/vertico

and I find :

M-RET -> vertico-exit-input

vertico-exit-input exits with the minibuffer input instead. Exiting with the current input is needed when you want to create a new buffer or a new file with find-file or switch-to-buffer. As an alternative to pressing M-RET, move the selection up to the input prompt by pressing the up arrow key and then press RET.

Well this is exactly the issue I was having and why I wrote my two little simple functions. Now if I build in M-RET into my muscle memory (which already seems pretty natural), I can refine my Emacs init, while adhering more to vanilla Emacs concepts. Also as I now am being more idiomatic, I suspect the M-RET keybinding may just come in handy at some point in the future.

So, now rebinding dired-mode-map as follows:

("_" . dired-create-empty-file)

This is all that is required; note that dired-create-directory is already bound to my preferred keybinding.

So continues the evolution of my Emacs understanding!

As a side-ish note, I have found that writing this blog provides not only an opportunity to explore a topic further and thoroughly understand it, thereby enabling me to articulate the subject fully, but also to receive comments and discover subtle nuances that I hadn’t realized existed or hadn’t thought to explore!

Four years ago I wrote about James Gill’s list of Emacs resources for writers. It was a curated list of resources that addressed writing prose of various sorts with Emacs. In April, I revisited the list. Judging from what I wrote, I had forgotten about my first post.

Now, I’m writing about it again because of Gill’s latest post about the list. In it, he says that surprisingly to him, that repository is the most popular of all his Github repositories. It’s not surprising to me because it’s a really great resource.

What was surprising to me, as I read down the his list, is how any are things that I have written about on Irreal. You may or may not think of that as an indication of quality but one thing for sure, it’s indicative of how interesting they are to me.

Regardless, if you use Emacs to write prose—for a blog, for reports, for articles, or even for books—you should take a look at this list. There’s sure to be something useful to you in it.

In a typical spreadsheet app, a common user interface practice is to avoid forcing the user to keep track of cell references as much as possible. This is usually seen in selecting a range of cells to construct a formula, where the user only needs to control the range of selection; the app keeps track of which cell references to pass into the formula.

Sadly, this feature is not available by default in Org tables. But no need to fret; Emacs and Org give you the mechanisms to build a reasonable emulation of spreadsheet-style cell range selection. This post shows you how.

Before going into details, here’s a demo of selecting a row of values and using that selection to compute its sum.

| a | b | c | Total |
|---+---+---+-------|
| 1 | 5 | 2 |       |
| 3 | 4 | 1 |     8 |
| 8 | 1 | 2 |       |
#+TBLFM: $4=vsum(@3$1..@3$3)

In the example above the following steps are taken:

1. Place the point in the cell to enter a Calc formula, in this case vsum().
   • Note that vsum here stands for vector sum.
2. Select a range of cells, typically with a mouse by pointing and dragging.
   • The selection can be either be a region or a region-rectangle.
     • Did you know that C-M-mouse-1 lets you mouse select a rectangle? I didn't until recently 🤯.
3. Raise a context menu pressing the right mouse button over the selected region.
4. In the context menu, a menu item showing the selected cell region is displayed (in this case @3$1..@3$3).
   1. Select this menu item to copy the cell references into the kill-ring.
5. Move the point back to the initial cell in step 1 and yank the cell reference as an argument for vsum().
6. Press TAB or RET to compute the formula.

Some observations:

• Throughout this workflow, the user did not need to keep track of cell references.
• The resultant table formula #+TBLFM: left-hand-side (lhs) is populated with only the column after step 6.
  • The user must amend the lhs to be @3$4 to specify that the formula should only apply to that cell.
  • AMENDED 2024-10-10: A workaround for this leveraging org-table-edit-formulas has been made. Details at end of this post.
• The table formula can be modified to use relative references at the cost of requiring the user to truly understand Org table reference syntax.
• Keyboard enthusiasts can use the command cc/copy-org-table-reference-dwim to convert a text region into an Org table reference.

Intrigued? Okay, let’s do this.

Org Table Reference Background

First off, let’s understand how Org tables address cells. A table cell is addressed using the following representation:

@ROW$COLUMN

A row is prefixed with @ whereas a column is prefixed with $. The row specification is placed before the column specification which is opposite of what's seen in conventional spreadsheets and Cartesian coordinates that specify the column first, then the row. Anyways, water under the bridge. With Org tables, it is always row first, column second.

A reference to an individual cell is called a field reference.

A reference to a set of cells is called a range reference. A range reference is constructed from two field references, where a and c are row values, b and d are column values.

@a$b..@c$d

Range references are translated into a vector of values that can be fed into Calc vector functions. Typically the two field references in a range reference are either in the same row or column. If not, then the vector of values is constructed using the two field references as a bounding box.

Most importantly, a range reference specifies values in a single vector. The orientation (row or column) of a range reference matters. To clarify, this is best illustrated by example.

#+NAME: vector-example
| a | b | c |
|---+---+---|
| 1 | 2 | 3 |
| 4 | 5 | 6 |
| 7 | 8 | 9 |

| Range      | Vector       | Notes                                                  |
|------------+--------------+--------------------------------------------------------|
| @2$1..@2$3 | [1, 2, 3]    | Cell values taken left to right.                       |
| @2$2..@4$2 | [2, 5, 8]    | Cell values "sliced" wrt column specification.         |
| @3$2..@4$3 | [5, 6, 8, 9] | Cell values taken using the extents as a bounding box. |
#+TBLFM: @2$2=remote(vector-example, @2$1..@2$3)
#+TBLFM: @3$2=remote(vector-example, @2$2..@4$2)
#+TBLFM: @4$2=remote(vector-example, @3$2..@4$3)

Design Requirements

With the background out of the way, let’s put down some requirements.

1. Given an active region within an Org table, translate it to a range reference with the following specifics:

   1. If the selected region is a single column or row, return a range reference using the mark and point of the region.
   2. If the selected region crosses has multiple rows and columns, then treat the mark and point of the region as the bounding box specification for a matrix represented as a Calc vector of vectors.
   3. If the selected region is a in a single cell, only return the field reference for that cell.
   4. The bounding box calculation is the same regardless if the mark is before or after the point.

2. Support both a mouse menu and mini-buffer command interface for obtaining an Org table region reference from a selected text region.

As a table is a convenient representation of a matrix, the above requirements makes the design decision to reflect that in mapping a text region having multiple rows and columns to a vector of vectors instead of a single vector.

Show Code

Here’s code that meets the requirements described above. Commands of note:

1. cc/copy-org-table-reference-dwim
2. cc/mouse-copy-org-table-reference-dwim

Given a point or region (text or rectangle) in an Org table, either of the two commands will create an Org table reference and push it in the kill-ring. The first command is intended to be called from the keyboard, the second from a mouse menu.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189

;; Org Table Cell Reference Functions

(defun cc/org-table-cell-at-point ()
  "At point, return the cell object from an Org table.

A cell object is defined to be a list containing the row and the
column, successively."
  (if (not (org-at-table-p))
      (error "Not in a table"))

  (let* ((row (org-table-current-dline))
         (col (org-table-current-column)))
    (list row col)))

(defun cc/format-org-table-field-reference (cell)
  "Format CELL object into @r$c format.

CELL object obtained via `cc/org-table-cell-at-point'.

See Info node `(org) References' for more on Org table field
reference format."
  (let ((row (nth 0 cell))
        (col (nth 1 cell)))
    (format "@%d$%d" row col)))

(defun cc/org-table-range ()
  "Return range object from a region defined within an Org table.

A range object is a list of two cells computed via
`cc/org-table-cell-at-point', the first being the cell at the
start of the region and the last being the cell at the end of the
region."
  (if (not (and (org-at-table-p) (use-region-p)))
      (error "Not in an Org table"))

  (save-excursion
    (let* ((end (cc/org-table-cell-at-point)))
      (exchange-point-and-mark)
      (let ((start (cc/org-table-cell-at-point)))
        (list start end)))))

(defvar cc/last-org-table-reference nil
  "Last stored Org table reference.

State variable to store an Org table reference (field or range)
to be used in an Org table formula. This variable is set via
`cc/org-table-reference-dwim'

NOTE: This state variable to work-around my lack of clarity on
region and mouse menu interaction.")

(defun cc/org-table-reference-dwim ()
  "Org table reference given point or region is defined.

Return Org table reference (field or range) depending on whether
a point or region is defined in an Org table.

If the region is defined over multiple columns, then a Calc
vector matrix is returned. See Info node `(org) Formula syntax
for Calc' for more.

Calling this function will set `cc/last-org-table-reference'.

See Info node `(org) References' for more on Org table field
reference format."
  (if (not (org-at-table-p))
      (error "Not in an Org table"))

  (cond
   ((use-region-p)

    (let* ((range (cc/org-table-range))
           (start (nth 0 range))
           (end (nth 1 range))
           (msg (format "%s..%s"
                        (cc/format-org-table-field-reference start)
                        (cc/format-org-table-field-reference end))))
      (setq cc/last-org-table-reference (cc/org-table-range-to-reference range))
      msg))

   (t
    (let ((msg (cc/format-org-table-field-reference (cc/org-table-cell-at-point))))
      (setq cc/last-org-table-reference msg)
      msg))))

(defun cc/copy-org-table-reference-dwim ()
  "Copy Org table reference (field or range) into kill ring.

Given a point or region defined in an Org table, add to the
`kill-ring' an Org table field or range reference.

If the region is defined over multiple columns, then a Calc
vector matrix is returned. See Info node `(org) Formula syntax
for Calc' for more.

If the buffer *Edit Formulas* is available (usually via
`org-table-edit-formulas'), the reference will be inserted into
it.

See Info node `(org) References' for more on Org table field
reference format."
  (interactive)
  (if (not (org-at-table-p))
      (error "Not in an Org table"))

  (let ((msg (cc/org-table-reference-dwim))
        (formulas-buffer (get-buffer "*Edit Formulas*")))
    (if formulas-buffer
        (with-current-buffer formulas-buffer
          (insert cc/last-org-table-reference)))
    (message "Range: %s, Copied %s" msg cc/last-org-table-reference)
    (kill-new cc/last-org-table-reference)))

(defun cc/mouse-copy-org-table-reference-dwim ()
  "Copy Org table reference (field or range) into kill ring via mouse.

Given a point or region defined in an Org table, add to the
`kill-ring' an Org table field or range reference.

NOTE: This function is intended to be called from a mouse menu
after `cc/copy-org-table-reference-dwim' is called which will set
`cc/last-org-table-reference'. This is to work-around my lack of
clarity on region and mouse menu interaction.

If the region is defined over multiple columns, then a Calc
vector matrix is returned. See Info node `(org) Formula syntax
for Calc' for more.

If the buffer *Edit Formulas* is available (usually via
`org-table-edit-formulas'), the reference will be inserted into
it. If the point in *Edit Formulas* is at the beginning of line,
it will treat the reference as a left hand side (lhs) assignment.

See Info node `(org) References' for more on Org table field
reference format."
  (interactive)
  (if (not (org-at-table-p))
      (error "Not in an Org table"))

  (when cc/last-org-table-reference
    (let ((msg cc/last-org-table-reference)
          (formulas-buffer (get-buffer "*Edit Formulas*")))
      (if formulas-buffer
        (with-current-buffer formulas-buffer
          (if (bolp)
              (insert (format "%s = " msg))  ; treat reference as lhs assignment
            (insert msg))))
      (message "Copied %s" msg)
      (kill-new msg))))

(defun cc/org-table-range-to-reference (range)
  "Convert RANGE object to Org table reference (field or range).

If the region is defined over multiple columns, then a Calc
vector matrix is returned. See Info node `(org) Formula syntax
for Calc' for more.

See `cc/org-table-range' for more on RANGE object."
  (let* ((start (nth 0 range))
         (end (nth 1 range))
         (a (nth 0 start))
         (b (nth 1 start))
         (c (nth 0 end))
         (d (nth 1 end))

         (r1 (apply #'min (list a c)))
         (c1 (apply #'min (list b d)))

         (r2 (apply #'max (list a c)))
         (c2 (apply #'max (list b d)))

         (rowrange (number-sequence r1 r2))
         (buflist (list)))


    (cond
     ((and (= r1 r2) (= c1 c2))
      (format "@%d$%d" r1 c1 ))

     ((or (= c1 c2) (= r1 r2))
      (format "@%d$%d..@%d$%d" r1 c1 r2 c2))

     (t
      (mapc (lambda (r)
              (push (format "@%d$%d..@%d$%d" r c1 r c2) buflist))
            rowrange)

      (format "vec(%s)"
              (string-join (reverse buflist) ", "))))))

Original Source.

AMENDED 2024-10-10: cc/copy-org-table-reference-dwim and cc/mouse-copy-org-table-reference-dwim modified to insert content into table formula buffer when org-table-edit-formulas is called.

The following abridged code illustrates how one could configure their context menu to show the current Org table reference and copy it into the kill-ring if selected. See Customizing the Emacs Context Menu for more on Emacs context menus.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

(defun cc/context-menu-addon-items (menu click)
  "Context menu hook function using MENU and CLICK event.

MENU - menu to be configured.
CLICK - event"
  (save-excursion
    (mouse-set-point click)
    (cc/context-menu-org-table-items menu (not (org-at-table-p)))
    menu))

(defun cc/context-menu-org-table-items (menu &optional inapt)
  "Menu items to populate MENU for Org table section if INAPT nil."
  (when (not inapt)
    (easy-menu-add-item menu nil
                        ["Table Cell Info"
                         cc/mouse-copy-org-table-reference-dwim
                         :label (cc/org-table-reference-dwim)
                         :help "Copy Org table reference (field or range) into kill ring via mouse"])))

(add-hook 'context-menu-functions #'cc/context-menu-addon-items)

Original Source.

All code running on Emacs 29.4.

Examples

The following table illustrates the result of selecting the text region between the cell with value 2 (@2$2) and the cell with value 12 (@4$4) and inserting its translated Org table reference into different formulas.

The first formula in @2$5 is just the matrix representation between 2 and 12.

The second formula takes the transpose (trn) of the first formula.

The third formula uses some matrix arithmetic to get the sum of each row of the matrix in @2$5.

#+NAME: matrix-example
| a |  b |  c |  d | result                               |
|---+----+----+----+--------------------------------------|
| 1 |  2 |  3 |  4 | [[2, 3, 4], [6, 7, 8], [10, 11, 12]] |
| 5 |  6 |  7 |  8 | [[2, 6, 10], [3, 7, 11], [4, 8, 12]] |
| 9 | 10 | 11 | 12 | [9, 21, 33]                          |
#+TBLFM: @2$5=vec(@2$2..@2$4, @3$2..@3$4, @4$2..@4$4)
#+TBLFM: @3$5=trn(vec(@2$2..@2$4, @3$2..@3$4, @4$2..@4$4))
#+TBLFM: @4$5=@2$5 * [1 1 1]

Closing Thoughts

Manually dealing with Org table references is both tedious and error-prone. Automating their generation using text range selection seems like a good way to make this more bearable. If you're comfortable with Elisp, I recommend you give this code a try.

I’m considering publishing this code as a package on MELPA but I think the utility of it is high enough to merit being in Org itself. For next steps, I look forward to finding out if that is true.

Amendment 2024-10-10

In sharing this post at the 2024-10-09 Org Meetup, I was made aware by Ihor Radchenko that the point position in the special buffer (named *Edit Formulas*) created by the org-table-edit-formulas command (binding C-c ') will highlight any field or range reference. This special buffer provides context from which we can enhance the commands cc/copy-org-table-reference-dwim and cc/mouse-copy-org-table-reference-dwim to insert their results directly into it, removing the step to yank. cc/mouse-copy-org-table-reference-dwim is further enhanced to check if the point in *Edit Formulas* is also at the beginning of a line. If so, then the inserted field reference will be treated as the lhs argument of a formula. This approach can be viewed as a workaround the direct entry of a formula in a cell with the "=" character, which will only insert in the lhs a column reference.

A frequent complaint we hear from n00bs is that Emacs out of the box is unusable, that it’s really hard to configure, and so you need something like Doom to get going. Generations of Emacs users have put the lie to that. Lots of us have been starting with vanilla Emacs and configuring it to our needs for 40 years.

Over at the Emacs subreddit, permetz suggests that it’s fine to have a simple configuration that builds on default Emacs. Permetz has been using Emacs since 1983 and considers himself a power user but he has a simple config file that mostly consists of setting a few variables to adjust things to his liking.

That’s how I started. Back then I was still writing in C so I spent a bit of time getting Emacs to indent C the way I liked. As I went along I added some packages and wrote some custom Elisp for bespoke operations that no one else would need. Even so, after 16 or 17 years my init.el is still pretty small and I could easily trim it down by getting rid of stuff I no longer need or use.

Permetz’s message is don’t be afraid of starting small from vanilla Emacs and having a simple configuration. You’ll be surprisingly productive and your init.el can grow organically as your needs change.

• Upcoming events:
  • OrgMeetup (virtual) https://bbb.emacsverse.org/b/iho-h7r-qg8-led Thu Oct 10 0900 America/Vancouver - 1100 America/Chicago - 1200 America/Toronto - 1600 Etc/GMT - 1800 Europe/Berlin - 2130 Asia/Kolkata – Fri Oct 11 0000 Asia/Singapore
  • Atelier Emacs Montpellier (in person) https://lebib.org/date/atelier-emacs Fri Oct 11 1800 Europe/Paris
  • M-x Research: TBA https://m-x-research.github.io/ Wed Oct 16 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
  • Call for proposals - European Lisp Symposium 2025: May 19-20 Zürich, deadline March 2
• Beginner:
  • 🤪 emacs tutorial (06:20)
  • Understanding Emacs modes - YouTube (58:19)
  • Wizbango Episode 3 [Emacs Basics]​ (27:02)
  • Emacs link map (@sacha@social.sachachua.com)
  • ¿Se sigue utilizando y desarrollando GNU Emacs en 2024? Una "Breve" Intro de solo 2 horas (02:08:53)
  • Emacs 16, 17, 18, 19, 20 (03:36) (in Spanish; part of a series)
• Emacs configuration:
  • [Emacs-Kicks Project] - For all Neovim users - Seeking Feedback & Interest!
  • Request for Comment: Casual Package Consolidation (@kickingvegas@sfba.social)
  • mehrad/.emacs.d: This is my attempt to put together my own emacs config (@Mehrad@fosstodon.org)
  • patrickdelliott's Emacs Config
• Emacs Lisp:
  • Major Mode Comfort Functions, Because I'm Lazy - how to add a mode to autoloads and auto-mode-alist
  • [ANN] f.el v0.21.0 has been released! (Reddit) file times, mkdir path, shortdoc
  • Aprendendo Emacs Lisp - Corte 5: Jogo da adivinhação (01:07:41)
• Appearance:
  • pulsic.el - Temporarily highlight the current line on window state change (Reddit)
  • James Dyer: Syncing Tab Theme After Theme Load
  • Listing and setting installed coding/monospaced fonts function for your Emacs
• Navigation:
  • Simple ripgrep integration that reuses grep-mode (@jroimartin@fosstodon.org)
  • Combine orgrr-search and metasearch-search-set (@laotang@mastodon.social)
  • rtrppl/metasearch: Metasearch for Emacs (@laotang@mastodon.social)
  • project-root-initialize
  • Avyならなんでもできる | Emacs JP
• Writing:
  • [pdf] A quick guide on writing a book with Emacs Org-mode. (Reddit)
  • Writing with Emacs (Reddit)
• Org Mode:
  • SUMMARY - #11 bbb:OrgMeetup on Wed, Sep 11, 19:00 UTC+3 - Ihor Radchenko (@yantar92@fosstodon.org)
  • Beginner tip about tagging Org Mode headings and filtering them
  • Save buffer automatically with org-clock-out (@mocom@mastodon.social)
  • Literate Org Haskell
  • nix-shell-in-emacs-org-mode
  • Ross A. Baker: Creating and Styling Callouts in Hugo (@shom@fosstodon.org) - also ox-hugo
• Denote:
  • denote-capture
  • Bibliographic Notes with the citar-denote Emacs Package
• Completion:
  • Corfu section of an Emacs tutorial (@tristanC@functional.cafe)
  • From Ivy To Vert&co – Tristan's Zettelkasten
  • xvw - Expansion d'abréviations avec YASnippet
• Coding:
  • Emacs for Python and Poetry Using `basedpyright-langserver` (@webbureaucrat@floss.social)
  • Extending Emacs's Go Tree Sitter Mode’s Imenu Function to Include Test Run // Take on Rules (@takeonrules@dice.camp)
  • martianh/tp.el: Transient utilities for POSTing to an API (@mousebot@todon.nl)
  • `cargo-jump-xref.el`: Help you jump to cargo dependency's source file in `Cargo.toml` file
• Math:
  • [karthink] Emacs 💜 LaTeX (14:13, Reddit) - also Org Mode and Calc, very cool (@ctietze, @Nidish96@emacs.ch)
  • More LaTeX productivity tips
  • Emacs/math/latex twitch stream
• Shells:
  • Customizing Emacs And Eshell - DT LIVE (01:08:51)
• Web:
  • Open with EWW - The Great SummerEmacs Blog (Reddit, @summeremacs@fashionsocial.host)
• Mail, news, and chat:
  • Opening a remote mbox file from Emacs (@jroimartin@fosstodon.org)
• Fun:
  • my-3x4-mode: Pseudofonts in Emacs. (Reddit)
  • Zalgo Mode - A Minor mode for writing spooky looking text
• AI:
  • llm-apply.el: Modify buffer contents via LLMs
  • Introduce aider (AI programming in terminal), and emacs binding for it, aider.el
  • Go talk to the LLM (lobste.rs)
  • 最強ローカルLLM実行環境としてのEmacs
• Community:
  • Weekly Tips, Tricks, &c. Thread - ERT run at point
  • 5 Reasons Why I Love and Use Emacs in Linux - Make Tech Easier (@governa@fosstodon.org)
  • What are some advantages to using Emacs as a newbie?
  • On vim vs Emacs - Steve Purcell: emacs can be vim
  • Irreal: Why Emacs: The View From Vim
• Other:
  • Jiewawa: Useful Emacs commands for reading
  • Recommending side-notes and dot-mode
  • Emacs as Menu for Shell Commands, Passwords and Bluetooth Connections (01:04, @mekeor@mastodon.catgirl.cloud)
  • Recording and screenshotting windows: the lazy way - macosrec
  • How to get railwaycat's Mitsuharu 29.4 port in homebrew (Mac OS X)
  • PSA Night and day Mac Emacs builds: caldwell (night) vs. jimeh (day)
  • Emacs-plus PATH in macOS Sequoia | The Art Of Not Asking Why (@jtr@fosstodon.org)
• Emacs development:
  • emacs-devel:
    • Discussion about preconfigured Emacs
    • Preview of tty child frames in scratch/tty-child-frames
  • Add a shortcut to go up in the menu hierarchy.
  • Add file/buffer-to-register (Bug#73308)
  • ; * etc/NEWS: Document Python electric-layout-mode support.
  • Add imenu index function for DjVu files in doc-view
  • New minor mode find-function-mode replaces find-function-setup-keys
  • Add commands to interactively set/unset keyboard translations
  • New M-~ entry in save-some-buffers-action-alist
  • New function shell-command-do-open
  • New value 'permanent-only' for defcustom :local keyword
  • New command diff-delete-other-hunks
• New packages:
  • bitbake-ts-mode: A major mode to use bitbake tree-sitter (MELPA)
  • corsair: Text accumulation enhancements for GPTel (MELPA)
  • flymake-biome: A flymake plugin for SQL files using biome (MELPA)
  • genexpr-mode: Major mode for editing GenExpr files (MELPA)
  • varuga: Send ical calendar invites by email (MELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

Wait. What? Is Lisp syntax boring? My immediate response is, “Of course it is.” That’s its strength. Lisp doesn’t have much syntax so unlike other languages you can immediately start using the language without having to worry about learning a bunch of syntax rules. Everybody who’s learned a non-Lisp language knows that a significant part of learning the language is imbibing the syntax. An advantage of Lisp is that you can mostly avoid that step.

You’d think that would be obvious to anyone with a passing familiarity with Lisp but apparently not. Aggravating_Date_315 over at the Emacs subreddit asks if anyone else finds Lisp syntax boring. Not in the sense that I do but that there’s not enough structure and sugar. He wants superfluous keywords and structure because the simplicity of Emacs is boring. He loves how easy it is to edit Elisp in Emacs but finds Lisp joyless and sterile.

The commenters were not particularly supportive although a few did offer some suggestions for dealing with the matter. My favorite was from intergalactic_llama who says, “People looking to create problems for them selves always fascinate me.”

I don’t understand how anyone could think that making something more complicated would make it more natural and fun. To me, the joy of programming is writing code to solve a problem and I have no interest in making that code more convoluted than it needs to be.

Since I started using Emacs, I’ve written hundreds of functions that I only used once and then were left to rot in the depths of my init.el.

Today I would like to share some simple commands that I wrote when I was just starting off with Emacs that I had forgotten that I had written and assumed that this was the default behaviour as it now seems to natural to me. A very small amount of the code here will be specific to evil-mode, but it shouldn’t be too difficult to adapt it to work without.

Removing visual fluff

I like to have a clean, distraction free reading experiences in major-modes such as elfeed, eww, and nov.

I achieve this by:

1. Centring the text in a column and increasing padding.
2. Hiding the mode-line
3. Hiding the cursor

Centring text

I use olivetti for centring text and have no complaints. It works seamlessly for my needs. In fact, I also enable this in Org-Mode and Markdown-Mode.

(defun my/centre-visual-fill-on ()
  ;; Centre the text in one column
  (visual-line-mode 1)
  (setq olivetti-body-width 100)
  (olivetti-mode 1))

(defun my/centre-visual-fill-off ()
  (visual-line-mode 0)
  (kill-local-variable 'olivetti-body-width)
  (olivetti-mode 0))

Hiding mode-line

I’ve seen lots of people do this one and it is useful when wanting to focus. Unlike the previous commands, this is something that I sometimes toggle so I made an interactive function too.

(defun my/kill-modeline ()
  (setq-local mode-line-format nil))

(defun my/restore-modeline ()
    (kill-local-variable 'mode-line-format))

(defun my/toggle-modeline ()
  (interactive)
  (if (null mode-line-format)
      (my/restore-modeline)
    (my/kill-modeline)))

Hiding the cursor

This is one that I don’t remember seeing anybody else do, but it’s a game changer. When I’m reading, I’m following with my eyes and I don’t care where the cursor is unless I decide that I want to highlight some text. Again, I want to be able to toggle this quickly so I wrote an interactive function too.

(defun my/hide-cursor-evil ()
  (setq-local evil-default-cursor '(ignore))
  (setq-local cursor-type nil))

(defun my/show-cursor-evil ()
  (kill-local-variable 'evil-default-cursor)
  (kill-local-variable 'cursor-type))

(defun my/toggle-cursor ()
  (interactive)
  (if (null cursor-type)
      (my/show-cursor-evil)
    (my/hide-cursor-evil)))

I combine these three ideas into a simple hook that gets ran whenever I open a new buffer in one of my reading modes.

(defun my/reading-mode-hook ()
  (my/centre-visual-fill-on)
  (my/hide-cursor-evil)
  (my/kill-modeline))

Navigating

I’ve seen takes on the previous ideas many times while reading other Emacs users' blogs, but I haven’t seen many users describe how they move around buffers that are primarily meant to be read.

Initially I tried using a form of paging, but I found it tedious to have to scan for where I left off before I scrolled down.

Ultimately, I settled on the following logic, which was very simple to implement:

• If the cursor is visible, move the cursor up or down one line
• If the cursor is not visible, scroll the buffer up or down one line

(defun my-view-down ()
  (interactive)
  (if (null cursor-type)
      (scroll-up-line)
    (next-line)))

(defun my-view-up ()
  (interactive)
  (if (null cursor-type)
      (scroll-down-line)
    (previous-line)))

Tying it all together

I could have come up with a minor mode for enabling this in each major mode, but I like how this reminds me of a time when I didn’t spend so long dicking about with functions that I only use once.

Here is a simple example configuration of how I have elfeed set up.

(use-package elfeed
  :hook
  (elfeed-show-mode . my/reading-mode-hook))

(evil-define-key 'normal elfeed-show-mode-map (kbd "j") 'my-view-down)
(evil-define-key 'normal elfeed-show-mode-map (kbd "k") 'my-view-up)

As those of you who have been around for awhile know, Irreal has a fascination with fonts for Emacs. I most recently wrote about them here. But, all those posts have been about coding fonts. Today, I want to talk about proportional fonts.

You can, of course use proportional fonts for coding but it’s mostly Rob Pike and his disciples that champion that. Still, most of use proportional fonts for everything else. Recently, Charles Choi even convinced me to write my posts in a proportional font.

This post was inspired by an article on the Atkinson Hyperlegible Font. It’s a font developed by the Braille Institute for low vision readers. Those of us without vision problems should also take notice because the font really is easy to read and, most importantly, it’s easy to distinguish among those hard to discern characters that I use to decide if a font is acceptable for programming use.

It’s usually not as important to distinguish them in a prose environment but most programmers are anal about things like that and would prefer having clarity in all the text that they read. Take a look at the Atkinson Hyperlegible Font article to see it in action and to see how well it distinguishes among all those problematic characters.

For me, one of the worst problems with proportional fonts is telling the difference between capital I (eye) and lowercase l (ell). As I write this in Emacs using my proportional font, those two letters look virtually the same. That’s a problem I wouldn’t have if I were using Inconsolata, my coding font. Atkinson Hyperlegible makes the difference clear. That alone is worth adopting the font.

You can download the font for free but the Braille Institute would appreciate a contribution. This seems like a great font and is worth a contribution to help the Institute in their work.

One of the activities that Irreal encourages is the sanding down of friction points. Despite that, I’m almost always surprised when I do automate some routine task and wonder why I waited so long to do it. If you’ve been around Irreal for a while, you’ve seen me write about that several times.

Over at dyerdwelling there’s a handy post on how to sand down one set of friction points. His particular friction point is creating directories and files from within Dired. That’s possible, of course, but the various completion engines can trip you up but substituting your choice for an existing file or directory with a name that matches in the fuzzy sense.

His solution is to define two new functions that ask for the directory or file name without invoking completion. The code is simple, short, and easy to understand. If you use Dired to creat files and directories, take a look at this post

Previously I wrote about the incompatibility of some Emacs themes with the tab-bar and my temporary fix:

Syncing Tab Bar To Theme

The tab bar was introduced in Emacs 27.1, during which I created a function to apply some simple logic to make the tabs look moderately decent and fit in with the new theme. The only thing I couldn’t figure out was how to activate the function after a theme load, as there didn’t seem to be a hook.

Well, actually there does seem to be such a hook and it seems to have gone into Emacs 29. As with a lot of these little tweaks I found this on reddit in the emacs subreddit:

https://www.reddit.com/r/emacs/comments/1ft3wwm/did_the_much_needed_theme_change_hook_arrive_with/

From my previous post I wrote :

Now that the defun is available, how to activate it? My first thought was to add a hook for when a theme is loaded, but there doesn’t seem to be such a feature!? I also tried advising around `load-theme`, but that didn’t go well either, it worked for themes that were not loaded, but for those that were, a change in state was not detected, and the tab theme tweak was not applied.

The solution is to simply add the following to your init file. Now that I know about this, I might even find some other use for it.

(defun my/after-theme-loaded(theme)
  (my/sync-tab-bar-to-theme))

(add-hook 'enable-theme-functions 'my/after-theme-loaded)

with the original my/sync-tab-bar-to-theme as follows:

(defun my/sync-tab-bar-to-theme ()
  "Synchronize tab-bar faces with the current theme."
  (interactive)
  (let ((default-bg (face-background 'default))
        (default-fg (face-foreground 'default))
        (inactive-fg (face-foreground 'mode-line-inactive))) ;; Fallback to mode-line-inactive
    (custom-set-faces
     `(tab-bar ((t (:inherit default :background ,default-bg :foreground ,default-fg))))
     `(tab-bar-tab ((t (:inherit default :background ,default-fg :foreground ,default-bg))))
     `(tab-bar-tab-inactive ((t (:inherit default :background ,default-bg :foreground ,inactive-fg)))))))

My tab theme fix is now activated on every theme load, and I no longer have to rely on muscle memory to make Emacs look colour-coherent when changing themes, which I do a lot!! 😀

It just works. — Steve Jobs

If you follow this blog, you’ll know that I’m doing a series called “Emacs Config From Scratch”¹. Emacs is an editor operating system, where you can configure and customize literally everything. I like that you can truly make it yours, but it’s a lot of work to get there.

Recently, I’ve become fond of (command line) tools that just work, out of the box². This blogpost is an ode to them.

Fish

Julia Evans recently posted Reasons I still love the fish shell, and the first point she makes is “no configuration”.

Things that require plugins and lots of code in shells like ZSH, like autosuggestions, are included and configured by default in fish. At the time of writing this, my fish config has less than 31 loc, most of which are abbreviations.

I have two fish plugins configured: z for jumping to a directory, and hydro as my shell prompt. Neither need configuration.

Helix

My Neovim config had 21 external plugins. Making LSP, tree-sitter and formatting work took a while (LSP alone needs 3 plugins) and in the end there were still things that didn’t work.

I’ve switched to Helix, which can do so much out of the box, here’s a non-exhaustive list:

• LSP (including autocompletion, show signature, go to definition, show references, etc.) just works
• Tree-sitter is built in, you can even do selections on tree-sitter objects
• A file picker and global search
• Pressing a key in normal mode shows subsequent keys you can press, and what they do
• You can jump to any visible word, add/remove/replace quotes or other characters
• … and so much more

The config for the code editor I use all day is 5 loc. Here it is:

theme = "kanagawa"

[editor]
line-number = "relative"
cursorline = true
rulers = [80]

I will say that it takes some getting used to as it folows the selection -> action model, i.e. you need to run wd instead of dw to delete the next word.

Lazygit

After raving about Magit in London, my team showed me Lazygit and I’ve been using it ever since—it’s really good and it does exactly what you want it to do, without configuration³.

You can toggle different screen modes, panes adjust in size when active and pretty much everything you want to do is only a few keystrokes away.

Zellij

A batteries-included Tmux alternative, Zellij doesn’t need any configuration to work well. You can set up Layouts without additional plugins (although there is a plugin system) and I’m generally not missing anything from my Tmux configuration.

My favorite feature is the floating panes. Press Ctrl + p, w to toggle a pane floating on top of everything else—I often use this for Lazygit.

What else?

Here are some tools you have sent me that are zero/minimal config:

• Matthew shared Broot, a new way to see and navigate directory trees
• Ilija shared k9s, a CLI to maange Kubernetes clusters
• Alexander shared Orbiton, a text editor they built

Do you have a tool that requires no (or minimal) configuration? Send me an email and I’ll add it to the list!

And if you’re building something, please strive to make the default experience work really well for most people.

Paul E. Johnson is a member of the Political Science Department at the University of Kansas. As such, he’s not a canonical Emacs user but he has a provacative set of slides that suggests that, in fact, Emacs doesn’t have a steep learning curve. Or, at least, it needn’t have one.

His thesis is that one can use Emacs with more or less “standard” keybindings by using the arrow and other special purpose keys and by turning on CUA mode to make Emacs input more like “modern” apps.

That’s all true, of course, but it strikes me as riding a bicycle with training wheels. Yes, you can ride around and do bicycle-like things but you can’t really call yourself a bicycle rider. Even Johnson recognizes this and notes that you’re going to want to learn at least some of those native Emacs commands.

When I learned to ride a bike, I didn’t use training wheels and I don’t recommend them for new Emacs users either. Sometimes you just need to grasp the nettle and push through the pain. The benefit is that at the other end you can ride a bike or use Emacs well.

Still, Johnson has a point. It’s possible to start using Emacs without having to learn all those bespoke keybindings. The question is whether that’s the best path forward. I think not but obviously others disagree and I’m willing to stipulate that they may be right, at least for some people.

• Upcoming events:
  • EmacsATX: Emacs Social https://www.meetup.com/emacsatx/events/303579672/ Wed Oct 2 1600 America/Vancouver - 1800 America/Chicago - 1900 America/Toronto - 2300 Etc/GMT – Thu Oct 3 0100 Europe/Berlin - 0430 Asia/Kolkata - 0700 Asia/Singapore
  • Emacs Paris: S: Emacs workshop in Paris (online) https://emacs-doctor.com/ Thu Oct 3 0830 America/Vancouver - 1030 America/Chicago - 1130 America/Toronto - 1530 Etc/GMT - 1730 Europe/Berlin - 2100 Asia/Kolkata - 2330 Asia/Singapore
  • Emacs.si (in person): Emacs.si meetup #13 2024 (v #živo) https://dogodki.kompot.si/events/cb5ed2be-9c40-49be-a241-64dc14892ef6 Thu Oct 3 1900 CET
  • M-x Research: TBA https://m-x-research.github.io/ Fri Oct 4 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
  • EmacsSF (in person): coffee.el in SF https://www.meetup.com/emacs-sf/events/303396554/ Sat Oct 5 1100 America/Los_Angeles
  • Atelier Emacs Montpellier (in person) https://lebib.org/date/atelier-emacs Fri Oct 11 1800 Europe/Paris
• Beginner:
  • Irreal: Emacs Doc Bling
• Emacs configuration:
  • welkomscherm.el - How do you like this simple dashboard? tips for improving the code and UI very welcome - begginner! (Reddit)
  • Configuring Emacs on My New Laptop (02:00:09)
  • Una configuracion minimalista para Emacs — algo para comenzar (0x0003) - FTV (@barocio@techhub.social)
• Emacs Lisp:
  • Emacs Lisp: Inserting to a variable rather than the buffer (@robsws@hachyderm.io)
• Appearance:
  • Closing Pitch | Emacs Appearance Panel | Video 15 (04:09)
• Navigation:
  • Announcing Casual EditKit (Reddit, Irreal)
  • Cycling through most recently windows with ace-window
  • Tip for using Emacs to review logs
• Dired:
  • James Dyer: Simple Directory and File Creation in Dired (Irreal)
• Org Mode:
  • Tip about using org-tree-to-indirect-buffer
  • Emacs Org Mode Instant Preview: A VSCode-inspired setup for Org files
  • `hyperdrive-org-transclusion`: `org-transclusion` add-on (Demo, Reddit)
  • Very happy to annoced: org-zettel-ref-mode 0.3.5 Stable release!
  • Import, export, and integration:
    • Emacs Org Ros Package Take Screenshot And Insert In Open Org Buffer 2024_09_30_08:26:58 (07:36, Reddit)
    • Sacha Chua: org-attaching the latest image from my Supernote via Browse and Access
    • Sacha Chua: Include inline SVGs in Org Mode HTML and Markdown exports
    • Sacha Chua: Using Emacs Lisp to export TXT/EPUB/PDF from Org Mode to the Supernote via Browse and Access
    • Publishing on the web with Jekyll, Emacs and Org-Mode by Christian Dewein (@fabrik42@mastodon.social)
• Completion:
  • From Ivy To Vert&co – Tristan's Zettelkasten (@tristanC@functional.cafe)
• Coding:
  • Augmented nano-modeline to display which-function in orange (@root42@chaos.social)
  • Release CIDER 1.16 ("Kherson") · clojure-emacs/cider · GitHub (@bbatsov@hachyderm.io)
  • meow_king/zig-ts-mode: Emacs Zig Tree Sitter Mode - Codeberg.org (@meowking@mastodon.social)
  • masukomi/private-comments-mode (@masukomi@connectified.com)
• Mail, news, and chat:
  • rcirc-sqlite 1.0.3 new command rcirc-sqlite-stats-per-month (@mattof@functional.cafe)
• AI:
  • Adding Artificial Intelligence To Emacs (15:31)
  • Dictation with small LLMs and linguistic pragmatics (@lepisma@mathstodon.xyz)
  • Evedel: Instructed LLM Programmer/Assistant (HN)
  • Integrando AI con mi herramienta de trabajo preferida: Google Gemini, Emacs Lisp y GNU/Linux!! (10:02)
• Community:
  • Weekly Tips, Tricks, &c. Thread
  • VIM vs NANO vs Emacs: Which Linux Text Editor is Best for You? (01:14:08) (Telugu)
  • Part Five of My Battles with Emacs (@macosxguru@fosstodon.org)
  • Buff stacking Emacs (30:23)
• Other:
  • Tip about using save-interprogram-paste-before-kill (@srijan@indieweb.social)
  • Emacs: commands in popup frames with emacsclient (14:41)
  • enote: Simple note taking helper which works literally in-place (Reddit)
  • Anand Tamariya: HTML Renderer (Reddit) - render table via SVG
  • J.e.r.e.m.y B.r.y.a.n.t: Emacs display code on TTY
  • Turbocharging Emacs | Building GCCEmacs from source For Beginners (30:19)
  • Javascript and more to emacs emacs application framework (01:45)
  • localauthor: Emacs Popup Frames Anywere — MacOS Edition (Reddit, Irreal)
• Emacs development:
  • emacs-devel: Nudge to add mouse-8 and mouse-9 bindings wherever there's history navigation
  • Delete some libraries obsolete since Emacs 24.4/24.5
  • Try "python" before "python3" in python-mode
  • Cease accepting the :reverse-video face attribute (bug#73552)
  • Warn about :reverse-video in defface
  • Add buffer-local register commands to DocView
  • Move to start of current header in diff-{file,hunk}-prev
  • New command diff-revert-and-kill-hunk
  • * etc/NEWS: Describe the change to enum handling in CC Mode
  • Declare nested backquotes unsupported in Pcase patterns (bug#72328)
• New packages:
  • consult-gh-embark: Embark Actions for consult-gh (MELPA)
  • consult-gh-forge: Magit/Forge Integration for consult-gh (MELPA)
  • jwt: Interact with JSON Web Tokens (MELPA)
  • omtose-phellack-themes: Two dark themes, with cold blusish touch (MELPA)
  • torrent-mode: Display torrent files in a tabulated view (MELPA)
  • zig-ts-mode: Tree Sitter support for Zig (MELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

A couple of weeks ago, I wrote about Protesilaos Stavrou’s excellent post on triggering an Emacs popup from anywhere in your system. It’s perfect for Org capture and other things you might want to do when you’re in some other app. I have something similar specialized to Org capture and Phil in the comments said he also has a few bespoke popups. The advantage of Prot’s framework is that it’s easy to invoke any Emacs command from outside of Emacs.

Grant Rosson liked the idea but found that Prot’s Linux based code didn’t port directly to macOS and he presented the code for the minimal changes required to get things working on a Mac. Despite what some folks seem to think, Mac users are a significant portion of the Emacs community so his changes are welcome for those of us using macOS.

Rosson gives some examples of commands other than Org capture that may give you some ideas. What I really like about Prot’s and Rosson’s frameworks is that they help you move as many chores as possible into Emacs. A lot of that simply boils down to being able to enter text in a reasonable—by which I mean Emacs—environment.

If you’re a Mac user and interested in being able to popup a temporary Emacs frame to do whatever you need to do, take a look at Prot’s video and Rosson’s post. You’ll be glad you did.

I've been experimenting with the Supernote's Browse and Access feature because I want to be able to upload files quickly instead of waiting for Dropbox to synchronize. First, I want to store the IP address in a variable:

(defvar my-supernote-ip-address "192.168.1.221")

Here's how to upload:

(defun my-supernote-upload (filename &optional supernote-path)
  (interactive "FFile: ")
  (setq supernote-path (or supernote-path "/INBOX"))
  (let* ((boundary (mml-compute-boundary '()))
         (url-request-method "POST")
         (url-request-extra-headers
          `(("Content-Type" . ,(format "multipart/form-data; boundary=%s" boundary))))
         (url-request-data
          (mm-url-encode-multipart-form-data
           `(("file" . (("name" . "file")
                        ("filename" . ,(file-name-nondirectory filename))
                        ("content-type" . "application/octet-stream")
                        ("filedata" . ,(with-temp-buffer
                                         (insert-file-contents-literally filename)
                                         (buffer-substring-no-properties (point-min) (point-max)))))))
           boundary)))
    (with-current-buffer
        (url-retrieve-synchronously
         (format "http://%s:8089%s" my-supernote-ip-address supernote-path))
      (re-search-backward "^$")
      (prog1 (json-read)
        (kill-buffer)))))

HTML isn't supported. Text works, but it doesn't support annotation. PDF or EPUB could work. It would make sense to register this as an export backend so that I can call it as part of the usual export process.

(defun my-supernote-org-upload-as-text (&optional async subtree visible-only body-only ext-plist)
  "Export Org format, but save it with a .txt extension."
  (interactive (list nil current-prefix-arg))
  (let ((filename (org-export-output-file-name ".txt" subtree))
        (text (org-export-as 'org subtree visible-only body-only ext-plist)))
    ;; consider copying instead of exporting so that #+begin_export html etc. is preserved
    (with-temp-file filename
      (insert text))
    (my-supernote-upload filename)))

(defun my-supernote-org-upload-as-pdf (&optional async subtree visible-only body-only ext-plist)
  (interactive (list nil current-prefix-arg))
  (my-supernote-upload (org-latex-export-to-pdf async subtree visible-only body-only ext-plist)))

(defun my-supernote-org-upload-as-epub (&optional async subtree visible-only body-only ext-plist)
  (interactive (list nil current-prefix-arg))
  (my-supernote-upload (org-epub-export-to-epub async subtree visible-only ext-plist)))

(org-export-define-backend
    'supernote nil
    :menu-entry '(?s "Supernote"
                     ((?s "as PDF" my-supernote-org-upload-as-pdf)
                      (?e "as EPUB" my-supernote-org-upload-as-epub)
                      (?o "as Org" my-supernote-org-upload-as-text))))

Adding this line to my Org file allows me to use \spacing{1.5} for 1.5 line spacing, so I can write in more annotations..

#+LATEX_HEADER+: \usepackage{setspace}

Sometimes I use custom blocks for HTML classes. When LaTeX complains about undefined environments, I can define them like this:

#+LATEX_HEADER+: \newenvironment{whatever_my_custom_environment_is_called}

Now I can export a subtree or file to my Supernote for easy review.

I wonder if multimodal AI models can handle annotated images with editing marks…

As every Irreal surely knows, I’m a die hard Emacs user but I was, for a long time, a Vim user and still consider it one of the two best editors. It’s long been my view that Emacs and Vim serve two different constituencies and aren’t really the same type of application. The TL;DR is that Vim is for people who want a small, fast, editor that excels at editing and doesn’t bother with anything else. Emacs, on the other hand, is for people who want a unified operating environment that—sort of—recapitulates the Lisp Machines.

Nicholas Bernstein is a committed Vi/Vim user who more or less agrees with me. Like me, he thinks that Vi/Vim and Emacs are distinct types of applications that serve different needs. He has a nice nice video that discusses this and makes the case that Emacs is all about providing a consistent interface for text based applications.

I think that that’s true but doesn’t tell the entire truth about Emacs. Yes, it does provide a consistent interface for text but it offers much more. It is, as I’ve often said, a light weight Lisp Machine that provides an integrating environment for all your computing needs. If you’re happy with a text-centric browser, Emacs can be the center of all your needs. Even if, like me, you use a standard browser for the dealing with the Web, Emacs can still provide almost everything else.

As I’ve said before, I spend almost all my time in either Emacs or Safari. I have to say, it’s a pretty nice environment. Bernstein makes the same case from the point of view of a Vi/Vim user.

This post shows how to set up Emacs popup frames in macOS, allowing quick access to Emacs functions from anywhere on your system.

As much as I would like to live in Emacs full time, I’m not quite there. But a recent post by Protesilaos Stavrou (aka Prot) is helping me bridge the gap.

Prot has written a short but powerful bit of code that allows you to access any piece of Emacs goodness — like org-capture — from outside Emacs with just one keystroke. The only requirement is that you are running in server-mode or using the Emacs daemon.

Whether you’re in a web browser, a PDF reader, or just staring at an empty desktop, your favorite Emacs command can be right at your fingertips. See Prot’s video for a thorough demonstration.

Now, I can quickly open a PDF using citar or copy a password from my pass-store. See below for some other useful popups — org-agenda view, mu4e inbox, and a quick translation interface.

Setting Up Emacs Popup Frames in MacOS

Since Prot wrote the code for use in Linux, it didn’t immediately work in MacOS. It turns out, the only necessary change was to add a window-system frame parameter to the popup frame. (See code below).

But while I was at it, I made a few additions: 1) a title frame parameter, which allows me to configure the popup frame size and placement through my tiling window manager, yabai; and 2) an optional argument that runs delete-frame after the main command runs. This makes it possible to call commands that are minibuffer-centric — in my case, that’s citar-open-file and password-store-copy.

Here’s the main code:

(defun popup-frame-delete (&rest _)
  "Kill selected frame if it has parameter `popup-frame'."
  (when (frame-parameter nil 'popup-frame))
  (delete-frame))

(defmacro popup-frame-define (command title &optional delete-frame)
  "Define interactive function to call COMMAND in frame with TITLE."
  `(defun ,(intern (format "popup-frame-%s" command)) ()
     (interactive)
     (let* ((display-buffer-alist '(("")
                                    (display-buffer-full-frame))))
       (frame (make-frame
               '((title . ,title)
                 (window-system . ns)
                 (popup-frame . t)))))
     (select-frame frame)
     (switch-to-buffer " popup-frame-hidden-buffer")
     (condition-case nil
         (progn
           (call-interactively ',command)
           (delete-other-windows))
       (error (delete-frame frame)))
     (when ,delete-frame
       (sit-for 0.2)
       (delete-frame frame))))

(use-package server
  :defer 1
  :config
  (unless (server-running-p)
    (server-start)))

And here are the macro calls that define commands:

(popup-frame-define org-capture "capture-popup")
(add-hook 'org-capture-after-finalize-hook #'popup-frame-delete)

(popup-frame-define password-store-copy "minimal-popup" 'delete-frame)

(popup-frame-define citar-open-files "minimal-popup" 'delete-frame)

Triggering Popups with Keyboard Shortcuts

To quickly bring up these popup frames with keyboard shortcuts, I use skhd, a hotkey daemon for macOS. The following configuration goes in the skhd config file:

cmd + ctrl - c : emacsclient -e '(popup-frame-org-capture)' || yabai -m window --focus recent
cmd + ctrl - o : emacsclient -e '(popup-frame-citar-open-files)' || yabai -m window --focus recent
cmd + ctrl - p : emacsclient -e '(popup-frame-password-store-copy)' || yabai -m window --focus recent

Managing Windows with yabai

Finally, I have configured my tiling window manager, yabai, to give the popup frames the desired size and placement.

First, the following goes in the yabai config file:

yabai -m signal --add event=window_created app="Emacs" action="~/.yabai-emacs-window-handler.sh"

This will run a bash script (located at ~/.yabai-emacs-window-handler.sh) every time a new Emacs frame is created. The script tells yabai how to move and resize the new frame, depending on the frame’s title. Here is that script:

#!/bin/bash

data=$(yabai -m query --windows --window $YABAI_WINDOW_ID)

title=$(echo $data | jq .title)
display=$(echo $data | jq .display)

if [[ $title =~ "capture-popup" && $display == 1 ]]; then
yabai -m window $YABAI_WINDOW_ID --toggle float --move abs:430:230
yabai -m window $YABAI_WINDOW_ID --resize abs:655:300
yabai -m window $YABAI_WINDOW_ID --focus
elif
[[ $title =~ "minimal-popup" && $display == 1 ]]; then
yabai -m window $YABAI_WINDOW_ID --toggle float --move abs:430:230
yabai -m window $YABAI_WINDOW_ID --resize abs:655:200
yabai -m window $YABAI_WINDOW_ID --focus
else
false
fi

Additional Custom Commands

In some cases, an Emacs command will work nicely in a popup frame right out of the box. A good case in point is org-capture. In other cases, I have needed to write my own custom commands. Here a few:

Google Translate Interface

(defun my/translate ()
  (interactive)
  (let ((choice
         (completing-read "Select: " '("EN->LT" "LT->EN"))))
    (cond ((string= choice "EN->LT")
           (google-translate-query-translate-reverse))
          ((string= choice "LT->EN")
           (google-translate-query-translate)))))

(popup-frame-define my/translate "minimal-popup")

Unread Emails in mu4e

(defun my/mu4e-unread ()
  (interactive)
  (mu4e 'background)
  (mu4e-search-bookmark
   (mu4e-get-bookmark-query ?u)))

(popup-frame-define my/org-agenda "large-popup")

For this, I add another “elif” to the bash script, to ensure I can see the whole inbox in the popup:

elif
[[ $title =~ "large-popup" && $display == 1 ]]; then
yabai -m window $YABAI_WINDOW_ID --toggle float --move abs:350:130
yabai -m window $YABAI_WINDOW_ID --resize abs:730:600
yabai -m window $YABAI_WINDOW_ID --focus

Custom Org-Agenda View

(defun my/org-agenda (&optional p)
  (interactive "P")
  (org-agenda p "d")
  (org-agenda-redo-all))

(popup-frame-define my/org-agenda "large-popup")

Just don’t forget to add a system-wide keyboard shortcut for each new command!

An Elisp implementation of HTML table rendering using SVG.

Note: The entry point is shr-render.

Sample Table

Code

In my Org Mode HTML and Markdown exports, I usually want to include SVGs inline so that I can use links. Sometimes I also want to use Javascript and CSS to modify elements within the images. I used to use a my-include: link to do this, but I realized that I can also modify this behaviour by making my own functions that call org-html-link or org-md-link and then put those functions in org-export-backend-transcoders.

Here is an example of an SVG:

The following code overrides HTML and Markdown exports to include SVGs.

(defun my-ox-link-path (link _ info)
  (let* ((raw-path (org-element-property :path link)))
    (setq raw-path
          (org-export-file-uri
           (org-publish-file-relative-name raw-path info)))
    ;; Possibly append `:html-link-home' to relative file
    ;; name.
    (let ((home (and (plist-get info :html-link-home)
                     (org-trim (plist-get info :html-link-home)))))
      (when (and home
                 (plist-get info :html-link-use-abs-url)
                 (not (file-name-absolute-p raw-path)))
        (setq raw-path (concat (file-name-as-directory home) raw-path))))
    raw-path))

(defun my-org-html-link (link desc info)
  (if (and
       (string= (org-element-property :type link) "file")
       (not (plist-get (org-export-read-attribute :attr_html (org-element-parent-element link))
                       :data-link))
       (org-export-inline-image-p link (plist-get info :html-inline-image-rules)))
      (let ((path (my-ox-link-path link desc info)))
        (if (string= (file-name-extension path) "svg")
            (with-temp-buffer
              (insert-file-contents-literally path)
              (buffer-string))
          (org-html-link link desc info)))
    (org-html-link link desc info)))

(defun my-org-md-link (link desc info)
  (if (and (string= (org-element-property :type link) "file")
           (not (plist-get (org-export-read-attribute :attr_html (org-element-parent-element link))
                       :data-link)))
      (let ((path (my-ox-link-path link desc info)))
        (if (string= (file-name-extension path) "svg")
            (with-temp-buffer
              (insert-file-contents-literally path)
              (buffer-string))
          (org-md-link link desc info)))
    (org-md-link link desc info)))

(defun my-org-11ty-link (link desc info)
  (if (and (string= (org-element-property :type link) "file")
           (not (plist-get (org-export-read-attribute :attr_html (org-element-parent-element link))
                       :data-link)))
      (let ((path (my-ox-link-path link desc info)))
        (if (string= (file-name-extension path) "svg")
            (with-temp-buffer
              (insert-file-contents-literally path)
              (buffer-string))
          (org-11ty-link link desc info)))
    (org-11ty-link link desc info)))

(with-eval-after-load 'ox-html
  (setf
   (alist-get 'link (org-export-backend-transcoders (org-export-get-backend 'html)))
   'my-org-html-link))
(with-eval-after-load 'ox-md
  (setf
   (alist-get 'link (org-export-backend-transcoders (org-export-get-backend 'md)))
   'my-org-md-link))
(with-eval-after-load 'ox-11ty
  (setf
   (alist-get 'link (org-export-backend-transcoders (org-export-get-backend '11ty)))
   'my-org-11ty-link))

I want to quickly get drawings from my Supernote A5X into Emacs so that I can include them in blog posts. Dropbox/Google Drive sync is slow because it synchronizes all the files. The Supernote can mirror its screen as an .mjpeg stream. I couldn't figure out how to grab a frame from that, but I did find out how to use Puppeteer to take an screenshot of the Supernote's screen mirror. Still, the resulting image is a little pixelated. If I turn on Browse and Access, the Supernote can serve directories and files as webpages. This lets me grab the latest file and process it. I don't often have time to fill a full A5 page with thoughts, so autocropping the image encourages me to get stuff out there instead of holding on to things.

(defvar my-supernote-ip-address "192.168.1.221")
(defun my-supernote-get-exported-files ()
  (let ((data (plz 'get (format "http://%s:8089/EXPORT" my-supernote-ip-address)))
        (list))
    (when (string-match "const json = '\\(.*\\)'" data)
      (sort
       (alist-get 'fileList (json-parse-string (match-string 1 data) :object-type 'alist :array-type 'list))
       :key (lambda (o) (alist-get 'date o))
       :lessp 'string<
       :reverse t))))

(defun my-supernote-org-attach-latest-exported-file ()
  (interactive)
  ;; save the file to the screenshot directory
  (let ((info (car (my-supernote-get-exported-files)))
        new-file
        renamed)
    ;; delete matching files
    (setq new-file (expand-file-name
                    (replace-regexp-in-string " " "%20" (alist-get 'name info) (org-attach-dir))))
    (when (file-exists-p new-file)
      (delete-file new-file))
    (org-attach-attach
     (format "http://%s:8089%s" my-supernote-ip-address
             (alist-get 'uri info))
     nil
     'url)
    (setq new-file (my-latest-file (org-attach-dir)))
    ;; recolor
    (my-sketch-recolor-png new-file)
    ;; autocrop that image
    (my-image-autocrop new-file)
    ;; possibly rename
    (setq renamed (my-image-recognize-get-new-filename new-file))
    (when renamed
      (setq renamed (expand-file-name renamed (org-attach-dir)))
      (rename-file new-file renamed t)
      (my-image-store renamed) ; file it in my archive
      (setq new-file renamed))
    ;; use a sketch link if it has an ID
    (if (string-match "^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9] "
                      (file-name-base renamed))
        (org-insert-link nil (concat "sketchFull:" (file-name-base renamed)))
      ;; insert the link
      (org-insert-link nil (concat "attachment:" (replace-regexp-in-string "#" "%23" (file-name-nondirectory new-file)))))
    (org-redisplay-inline-images)))

As most of you know, Irreal doesn’t think much of editor-connected bling. We much prefer simple efficiency to flashy accouterments that, in the end, simply get in the way of a decent editing experience. That’s true even though they can, arguably, make it easier to get started with a new editor.

It feels a little odd, then, to be writing favorably about these reformatted Emacs documents that, while they have the same content as the official documents, have been reformatted to look a bit nicer and, more importantly, be easier to navigate. It’s probably not accurate to describes the results as “bling” especially since the point of the change is better navigation.

The two major changes are a sidebar that essentially recapitulates the table of contents so that it’s easy to get an overall view of the document and an improved search function that makes parsing the search results easier.

Irreal does think that these documents are a worthwhile addition to the Emacs document stack and wouldn’t object if, arguendo, they were adopted by GNU. Still, the Irreal bunker is probably going to continue to use the “official” documents but doesn’t object if others make a different choice.

The documents that are available are:

• The GNU Emacs Manual
• The GNU Elisp Reference Manual
• The Org Manual
• The AUCTeX Manual
• The Magit Manual

Take a look at one or more of them and see what you think.

Today I’ll simply delve into two custom functions that I’ve had in my init file for quite some time, my/dired-create-directory and my/dired-create-empty-file

I use them frequently enough that I thought they were worth mentioning and they augment my dired workflow when mapped to dired-mode-map keybindings.

For many years I would create a new file or folder in one of two ways, through the command line, or a GUI file explorer (with either a click in or bound F2 to edit the name), Emacs comes with it’s own directory manager so why not augment it with these features?

What about find-file I hear you say?, just press C-x C-f. This will prompt for a filename using the current directory of the current buffer and if you type a unique name it will create the file for you. Unfortunately though this will not always work in a simple predictable manner when the minibuffer completion mechanism kicks in.

my/dired-create-directory

This custom function simplifies creating a new directory by prompting for a name in the minibuffer, bypassing the default completion mechanism to avoid unwanted name suggestions.

(defun my/dired-create-directory ()
  "Wrapper to dired-create-directory to avoid minibuffer completion."
  (interactive)
  (let ((search-term
          (read-from-minibuffer "Dir : ")))
    (dired-create-directory search-term)))

• Purpose: This function is a wrapper around the dired-create-directory function, designed to avoid the potential confusion of minibuffer completion and directly ask for the directory name.
• Usage: When invoked, it prompts the user to input a directory name and creates it using the dired-create-directory function.

my/dired-create-empty-file

Similarly, this custom function facilitates the creation of an empty file by asking for the filename in the minibuffer without the default completion.

(defun my/dired-create-empty-file ()
  "Wrapper to dired-create-empty-file to avoid minibuffer completion."
  (interactive)
  (let ((search-term
          (read-from-minibuffer "File : ")))
    (dired-create-empty-file search-term)))

• Purpose: This function wraps around dired-create-empty-file, providing a straightforward prompt for creating a new file.
• Usage: When triggered, it asks the user to input a file name and then creates an empty file with that name.

These functions are bound to dired-mode-map as follows:

• _: Create an empty file using my/dired-create-empty-file
• +: Create a new directory using my/dired-create-directory

This gives me a good ergonomic LShift+<Right hand of number row> combination and is therefore not too taxing on the hands and each key command are just next to each-other to emphasize their similar nature.

In addition the dired mode map keybinding for creating a new directory is bound to + so there is consistency there too.

I suspect there are probably many other ways to achieve this functionality in Emacs, in this case I am specifically relying on dired function calls here but there is also the make-directory function and the shell-command for example if you want to drop in to using the operating systems tool set.

#Emacs #TIL : I learned about save-interprogram-paste-before-kill - which saves the existing system clipboard text into the kill ring before replacing it. This ensures that Emacs kill operations do not irrevocably overwrite existing clipboard text.

A common workflow for me is to copy some text from a different application and paste it inside Emacs. But, if I want to first delete a word or region to replace, the deleted word or region goes to the system clipboard and replaces my copied text. This config saves the previous entry in the system clipboard so I can do a C-p after paste to choose the previous paste.

Syndicated to:

• https://bsky.app/profile/srijan4.bsky.social/post/3l4w53avhjw2b

Charles Choi has another Casual app on offer. This time it’s a comprehensive set of menus, Casual EditKit, that capture the various editing commands. On the one hand, these are commands that you really need to know to become proficient with Emacs. On the other hand, as Choi says, the menus serve as a means of discovering those commands.

The problem is that Emacs has such a rich set of commands for editing text that it can be hard to learn them or even know that they exist. For example, I am always seeing comments from people expressing surprise about the transpose-chars and transpose-words commands. I use them everyday and wouldn’t want to live without them, yet there are lots of folks who don’t even know they exist. There are a bunch of other transpose commands that I don’t have bound but that I know exist so I can invoke them—with the help of fuzzy command completion—as named commands when I need them.

This all works because I know these commands exist and it’s not hard for me to invoke them even if I don’t remember their bindings or they have none. The advantage of Choi’s Casual EditKit is that helps you discover these commands and once you know they exist, they’re easy to invoke even if you can’t remember all the details.

Choi just announced Casual EditKit so I obviously don’t have any experience with it but my snap judgment is that its main benefit is teaching a new user what editing commands are available. But, in the long run, every user is going to have to learn those commands. Still, I suppose the menus could be useful for little used commands such as, say, the rectangle commands.

In any event, Casual EditKit is a welcome addition to Choi’s wonderful collection of Emacs menus.

An old joke of Emacs is that it’s a great operating system in need of a good text editor. In truth, Emacs is unmatched in its commands for editing, offering many different commands for editing words, sentences, paragraphs, balanced expressions (sexps) and functions (defuns). The problem is knowing beforehand that such a command that you could use exists.

Casual EditKit is an Emacs package designed to surface these editing commands in a usable way by organizing them into keyboard-driven menus. Casual EditKit is now available on MELPA.

Among the editing and editing-related menus Casual EditKit provides are those for:

• Register commands
• Rectangle commands
• Editing commands for words, sentences, paragraphs, and balanced expressions, and functions
  • Marking
  • Copying
  • Killing
  • Moving
  • Transposing
  • Transforming
  • Deleting
  • Sorting
• Window management
  • Creating
  • Moving
  • Deleting
• Search & Replace commands
• Open commands
• Project commands
• Bookmark commands
• Tool commands
• Macro commands

Shown below is a reference menu (casual-editkit-main-tmenu) designed to demonstrate all of the above menus. Use it to get an immediate feel for what Casual EditKit offers.

As editing is such a personal experience, Casual EditKit can be considered a library of menus from which a bespoke solution can be crafted. Motivated readers comfortable with Elisp are encouraged to review the Casual EditKit README file and study its source code to make their own customized menus.

Moving Text with Pinned Menus

In many ways, Casual EditKit can be seen as a continuation of my earlier work to move text elegantly. In that post, I posited that instrumenting mouse menus (both main and context) to access text movement commands was ideal for recognition. However, as mouse menus in Emacs cannot be "pinned" to stay persistent, this discourages repeated behavior. Transient menus however can be "pinned". This allows for behavior like moving text (word, sentence, paragraph, etc.) forwards or backwards in an interactive fashion.

Closing Thoughts

Knowing how to use the editing commands in Emacs is a superpower. That knowledge used to be hard won. Casual EditKit now makes this less so.

As Casual EditKit is new, feedback is welcome. Please share your thoughts on it at the discussion group.

Finally, if you have benefited from using Casual EditKit or any of the Casual packages (now counting at 12!), please consider buying me a coffee to help in their development and maintenance. It would be greatly appreciated.

• Upcoming events:
  • Emacs Berlin (virtual, in English) https://emacs-berlin.org/ Wed Sep 25 0930 America/Vancouver - 1130 America/Chicago - 1230 America/Toronto - 1630 Etc/GMT - 1830 Europe/Berlin - 2200 Asia/Kolkata – Thu Sep 26 0030 Asia/Singapore
  • EmacsSF: Emacs Hangout: Potpourri (Hybrid) https://www.meetup.com/emacs-sf/events/303385779/ Thu Sep 26 1800 America/Vancouver - 2000 America/Chicago - 2100 America/Toronto – Fri Sep 27 0100 Etc/GMT - 0300 Europe/Berlin - 0630 Asia/Kolkata - 0900 Asia/Singapore
  • Emacs APAC (virtual) https://emacs-apac.gitlab.io/ Sat Sep 28 0130 America/Vancouver - 0330 America/Chicago - 0430 America/Toronto - 0830 Etc/GMT - 1030 Europe/Berlin - 1400 Asia/Kolkata - 1630 Asia/Singapore
  • EmacsATX: Emacs Social https://www.meetup.com/emacsatx/events/303579672/ Wed Oct 2 1600 America/Vancouver - 1800 America/Chicago - 1900 America/Toronto - 2300 Etc/GMT – Thu Oct 3 0100 Europe/Berlin - 0430 Asia/Kolkata - 0700 Asia/Singapore
  • Emacs Paris: S: Emacs workshop in Paris (online) https://emacs-doctor.com/ Thu Oct 3 0830 America/Vancouver - 1030 America/Chicago - 1130 America/Toronto - 1530 Etc/GMT - 1730 Europe/Berlin - 2100 Asia/Kolkata - 2330 Asia/Singapore
  • Emacs.si (in person): Emacs.si meetup #13 2024 (v #živo) https://dogodki.kompot.si/events/cb5ed2be-9c40-49be-a241-64dc14892ef6 Thu Oct 3 1900 CET
  • M-x Research: TBA https://m-x-research.github.io/ Fri Oct 4 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
  • EmacsSF: coffee.el in SF https://www.meetup.com/emacs-sf/events/303396554/ Sat Oct 5 1100 America/Vancouver - 1300 America/Chicago - 1400 America/Toronto - 1800 Etc/GMT - 2000 Europe/Berlin - 2330 Asia/Kolkata – Sun Oct 6 0200 Asia/Singapore
• Beginner:
  • Emacs has no learning curve (2015, HN, Irreal)
  • Elementos de la interfaz de usurario de emacs y operaciones basicas con archivos (0x0002) - FTV (@barocio@techhub.social)
  • Introduccion a Emacs (0x0001) - FTV (@barocio@techhub.social)
  • ‎Starting to map beginner-oriented links (@sacha@social.sachachua.com)
• Emacs configuration:
  • A New Beginning With GNU Emacs - Configuring Emacs 01
  • Sacha Chua: Highlight the current line while still being able to easily customize/describe underlying faces
  • Using an External Git Repo for My Emacs Config with Nix (July 2024)
  • Using GIT to sync Emacs config - TuxeduTV
  • jcastp/emacs.d: My personal emacs config - Codeberg.org (@jcastp@mastodon.online)
  • patpatpat's Emacs Config
• Emacs Lisp:
  • Starting a temporary fork of Lispy until abo-abo returns (Reddit)
  • Ep586, Emacs Lisp, Rename All Link of HTML Files (01:36:46)
• Appearance:
  • Emacs w/ auto-dark-mode
  • Sacha Chua: Quickly adding face properties to regions
  • J.e.r.e.m.y B.r.y.a.n.t: Emacs and redisplay on the terminal (TTY). ``Because the true color of computing is phosphorescent green on black.''
  • Frame Size | Emacs Appearance Panel | Video 9 (01:22)
• Navigation:
  • Emacs tabs as desktops (@summeremacs@fashionsocial.host)
  • super-hint.el: Enhance rg and xref results with `which-function` hints. (Reddit)
  • Alvaro Ramirez: Emacs bubble mode (HN, Reddit)
• Dired:
  • James Dyer: Adding Disk Usage Reporting to Emacs Dired
• Org Mode:
  • Marcin Borkowski: Irregular recurring TODOs in Org mode, part II
  • How I track workouts using `org-agenda-list'
  • On switching to Org-Mode for my website
  • My Solution to Handle excalidraw in Orgmode
  • Using TikZ in Markup Languages like Org mode
  • book_generator: générateur de fichiers orgmode pour rédaction de livre - book_generator
  • villarragut/small-notes: Create small notes in Emacs and link them (Reddit)
  • Org-roam's weird twin is finally on MELPA: Org-node 1.0! (Reddit)
  • org-roam-gt: a package to speedup org-roam-find-node
  • Tip for faster org-roam - (setq org-roam-db-update-on-save nil)
  • Autoshare -> Tasker -> Termux -> Org Protocol -> Org Capture
  • Org development:
    • ox-icalendar: Fix export of diary-style timestamps
    • ox-texinfo: Support links in headings
• Denote:
  • Protesilaos Stavrou: Emacs: use Denote for meetings or events
  • TIL: Fix use M-x denote can't display list of notes
• Completion:
  • vertico impressions
  • Company 1.0.0 released
• Coding:
  • From vanilla Emacs to Java IDE in 5 minutes
  • Find the closest Makefile in a parent dir and run it
  • build.el: A transient menu for build systems (bazel, make, cmake, cargo) (Reddit)
  • Mickey Petersen: Combobulate: Bulk Editing Tree-Sitter Nodes with Multiple Cursors - lots of deep-dive details (Reddit)
  • Release v1.8 · magit-todos (Reddit)
  • Kubed (Emacs-based Kubernetes interface) user manual, now with pretty images | Hacker News (HN, lobste.rs)
• Math:
  • ESS - Emacs Speaks Statistics (HN)
• Shells:
  • Calling Emacs functions in vterm using vterm-eval-cmds (@thaodan@mastodon.social)
  • Alvaro Ramirez: How I batch apply and save one-liners - dwim-shell-command (Reddit, lobste.rs)
  • Eshell y el comando clear 1 en Emacs - makigas comunidad (@GK_S@mastodon.social)
• Mail, news, and chat:
  • chrmoe/toot-suite: Some Emacs utilities for Mastodon (@noctuaminervae@toot.si) - archive, ol-toot
  • mastodon.el 1.0.27 - v2 filters, remote instance tag timeline
  • Sacha Chua: Collecting Emacs News from Mastodon
  • Sacha Chua: Archiving public toots on my blog
  • Andrea: Adding support to start a huddle from emacs-slack (jumping to the app)
  • Andrea: Enable OAuth for Gmail with Emacs and OfflineIMAP
• Multimedia:
  • Sacha Chua: Using WhisperX to get word-level timestamps for audio editing with Emacs and subed-record
• Fun:
  • bad.el - ascii art, demos and games with Emacs Lisp
• AI:
  • LLMs are handy for discovering Emacs features
  • Using LLM function calling (tool use) in Emacs: demo use-case & code walkthrough (08:47, Reddit)
• Community:
  • Weekly Tips, Tricks, &c. Thread
  • The argument for emacs in just a few minutes (vlog) (2023, Reddit)
  • Irreal: An Early Emacs Paper
• Other:
  • frantic-save: write every change in Emacs buffer to disk (@shapr@recurse.social)
  • rjekker/password-store-menu: A better, more complete user interface for password-store (@kickingvegas@sfba.social)
  • bdiff.el and my other Emacs Packages
  • Buff stacking Emacs (30:23)
  • Use Emacs to Write Go Code (Demos | Utterly Voice). Utterly Voice is a customizable voice dictation and computer control application
  • How to fix the TAB key in Emacs - TuxeduTV
  • hyperdrive.el 0.4.0: org-transclusion integration and easy installation!
  • Tip about load-no-native
  • Protesilaos Stavrou: Emacs: commands in popup frames with ‘emacsclient’ (14:41, Reddit, Irreal)
  • USHIN Software: Emacs for P2P deliberation
  • tfw/Elisp-and-mcclim: My late days emacsconf 2024 pitch for using common lisp mcclim with emacs lisp. (@screwtape@mastodon.sdf.org)
  • manateelazycat/cloel: Cloel is a framework that combines Clojure and Elisp for collaborative programming (@manateelazycat)
• Emacs development:
  • emacs-devel:
    • some considerations for child frames and visibility
    • thinking about order of "About Emacs" page
  • Teach Electric Pair mode about prefix arguments
  • Don't activate midnight-mode when loading 'midnight'
  • New configuration option '–disable-gc-mark-trace'
  • Add option for delete-pair to mark end of region
  • Add user option to enable Doxygen syntax highlighting (bug#72814)
  • Add accessors for treesit-font-lock-settings
  • Re-enable GC mark trace buffer by default
  • Rename project-{find-file-in-root,root-find-file}
• New packages:
  • casual-editkit: Transient UI for Symbol Overlay (MELPA)
  • compile-multi-nerd-icons: Affixate `compile-multi' with nerd icons (MELPA)
  • dbml-mode: Major mode for DBML (MELPA)
  • dup-transform: RGB/XY graphics code helpers (MELPA)
  • el-who: A s-expression html DSL library compatible with cl-who (MELPA)
  • eros-inspector: Glue between eros and inspector (MELPA)
  • flymake-ansible-lint: A Flymake backend for ansible-lint (MELPA)
  • hyperdrive-org-transclusion: Tranclude hyperdrive content (NonGNU ELPA)
  • incus-tramp: TRAMP integration for Incus containers (MELPA)
  • indent-bars: Highlight indentation with bars (GNU ELPA)
  • txl: Provides machine translation via DeepL's REST API (MELPA)

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

Paul Graham has a very interesting post that considers why it’s so hard for most corporations to produce good software. Actually, his post isn’t quite about that. Rather, it discusses why it’s so hard and so important for a programmer to hold a program in his head. Graham gives a list of 8 prerequisites for enabling programmers to do that. He says that:

1. It’s amazing how often programmers manage to to achieve all eight on their own.
2. It’s amazing how often corporations mange to do all eight wrong.

I agree very strongly with everything he says. I program the way he describes and I hate having anyone else touch the particular piece of code I’m working on. I also believe in small teams with a minimum of interference from above. Every manager’s nightmare.

Plenty of people will, of course, disagree but Graham’s article got me thinking about a discussion I had with Perry Metzger in an Irreal comment about pair programming. Metzger had observed that Emacs wasn’t very useful for pair programming and I made a snarky remark about pair programming. Metzger responded that he really enjoyed it and that he’s never been so productive.

That seems to directly contravene what Graham said and it certainly runs against my inclinations. The thing is, Metzger is a serious and accomplished computer scientist as well as a long time Emacs user so nothing he says should be dismissed out of hand. I suppose that this is another case of different strokes for different folks so we can only agree to disagree. In any event, take a look at Graham’s post and Metzger’s comment to see what you think.

Protesilaos Stavrou (Prot) has posted a really nice and useful video on something that I’ve been struggling with off and on for many years. The problem is I often want—usually several times a day—to pop up an Emacs buffer and run some command like org-capture. There’s no problem with that, of course, if I’m in Emacs but what if—as is often the case—I’m in some other app?

The basic strategy is to call emacsclient with a -e parameter specifying whatever Emacs command you want to run. The devil, as they say, is in the details. First of all, you have to arrange for an Emacs frame to hold the popup menu and then you have to arrange for it to be deleted when you’re done.

Prot provides a complete, easy to understand framework for all of this. He defines a macro that lets you specify an Emacs command for a popup. You need only specify an OS level key binding to call the function and identify a hook function that will call the function to delete the popup frame when the command is done—most other situations are handled by the code generated by the macro.

I’ve tried to solve this problem with everything from some Applescript to Alphapapas yequake. I’m currently using yequake and it works well for me but if I were starting out I’d definitely take a look at Prot’s code. It’s self contained, easily understandable and hackable, and can be adjusted to your particular use case. All you need is a way of binding an OS level key sequence to the appropriate command. If you need to execute Emacs commands from outside of Emacs, take a look at Prot’s code.

You can think of today’s post as a coda to last week’s Red Meat Friday. Unlike that post, this week’s offering isn’t really about the editor holy war. It’s about Emacs versus every other editor out there, which is a different thing.

As you know, I don’t really take sides in the Emacs/Vim debate. I think they offer different things to people with different needs. The rest of those editors though… Well, you all know my feelings about editors and their users who prioritize bling over efficiency: If your editor uses menus exclusively and you have to click to do anything then, sorry, your editor isn’t efficient.

Of course, none of that will matter because those editors are “modern” and Emacs is old and used only by boomers. Or something. Doubtless, some of those people will have the scales fall from their eyes and discover true editor efficiency. Sadly many others will never make that discovery and will end their days clicking on menus and using the arrow keys.

But this is, after all, Red Meat Friday so enjoy the red meat now because tomorrow we get back to serious stuff.