CSS

Tuesday, April 26, 2022

Zettelkasten workflow

I was asked about the details concerning my Zettelkasten (last time I checked, I had an estimated 7920 slips in it, which motivated the question). I thought I would write them down here with some examples.

The basic workflow boils down to:

  1. Pick some reading material
  2. Read it
  3. Summarize the reading material in "literature notes"
  4. Write some "permanent notes"
  5. Integrate the "permanent notes" in my "zoo" [Zettelkasten]
  6. Think about stuff
  7. Write a manuscript using my permanent notes
  8. Questions tend to crop up while writing a manuscript, which guides my choice of next reading material

Literature Notes

Definition. A Literature Note is some writing which occurs when reading an article or book. Its contents vary considerably, but are used to create permanent notes. The literature notes are stored in a separate "shoe box".

The content of the literatuer notes varies depending on how familiar I am with the subject, or whether I can "see where it's going". These are written on quarter-slips of printer paper which are 4.25-inches wide and 5.5-inches tall. I also draw a stripe down the "top" using a blue highlighter, so I can see if I accidentally placed one in my Zettelkasten. [I store literature notes in a "bibliography shoe box".]

Figure 1. A [clipped] literature note from Mac Lane's Categories for the Working Mathematician.

If I'm having difficulty with some portion of an article or book, I sometimes look for other explanations, and insert "citations" to them. (Basically, a sentence like, "I honestly cannot follow [author] here, but I found [other reference] useful, specifically [section].")

Note I usually write small, and this is kinda large for my writing: it's about 3/16 inches, or 0.5 centimeters, for a line's height. I do not place lined paper behind the slip as I write, so it can get pretty curvy by accident.

The back of the literature notes include bibliographic information, and when I read it. This may include a "counter" for long chapters, which is just something like "(part 3 of N)". For technical articles, I also include the journal information. The idea is, if I completely lose access to my computer and papers, I can look up the source material again. Consequently, I give as much bibliographic information as possible.

Figure 2. The back of the literature note from figure 1.

I can refer to my literature notes when writing a permanent note, but I do not permit myself to refer to the original source when writing permanent notes. If I find I need to refer to the original source, I take that to mean my literature note-taking process was lacking somehow.

There are times when, for example, I read a source particularly from one perspective. (E.g., "How can this article about quantum hydrodynamics be used in this particular calculation I'm trying to do?") Later on, I realize there was some important aspect I neglected to take notes on (e.g., "Oh, that article about quantum hydrodynamics revived the 'particle model tradition' of Stokes, I totally missed that."). In that case, I re-read the article or book, and because I note when I read it on the back...it acts like a note to "future me" that there was some material I missed in the reading.

(And, for the record, I organize the literature notes by author-year-title.)

Permanent Notes

Definition. A Permanent Note is some writing which is given an ID number, a "title", and some content; it is stored in the Zettelkasten.

Most of my notes are mathematical in nature, or written in a "mathematical style". I mean to say, I write them using "mathematical registers" to classify the sort of statement (theorem, example, definition, remark, etc.) and a summary of the result. Then the body of the slip of paper contains more precise details.

Figure 3. An example of a permanent note [front side].

I write on printer paper cut into quarters, so this is 5.5-inches wide and 4.25 inches tall. When writing on the paper, I have a college-ruled lined paper behind the quarter slip (which is why it appears so straight and clean).

ID Numbering. I should mention the ID number (the "6.1.2/3b" in the upper left-hand corner) has the following BNF grammar:

id⟩ ::= ⟨folder-part⟩ "/" ⟨file-part⟩
  ⟨folder-part⟩ ::= ⟨positive integer⟩
                 |  ⟨positive integer⟩ "." ⟨folder-part⟩
  ⟨file-part⟩ ::= ⟨positive integer⟩
               |  ⟨positive integer⟩ ⟨lowercase letter⟩
               |  ⟨positive integer⟩ ⟨lowercase letter⟩ ⟨file-part

The "folder part" encodes the generic subject of the slip. For example, "6.1.2" is the folder part for the slip given. Here "6" refers to "Mathematical analysis", "6.1" refers to "Real analysis", and "6.1.2" refers to sequences of real numbers. Observe, the folder part moves from more general ("Real analysis") to the more specific; this is characteristic of all "folders" in my Zettelkasten.

The "file part" encodes the placement of a slip "inside" a "folder". The idea is that I'm sending "telegrams" to an intelligent (but completely ignorant) colleague. My colleague has never heard of real analysis (or anything else besides arithmetic and spelling). I need to order my telegrams somehow. The first attempt is to use Threads, just a sequence of numbers.

But when I want to discuss further an aspect of a thread (like, I want to pause and give a "sub-thread" consisting of examples), then I need to Branch off a slip to talk about some specific aspect. A single slip may have zero or more branches. I indicate a branch by writing a letter. Usually "a" is reserved for examples (unless there is a critical remark necessary to clarify something), then "b" is reserved for theorems about a concept.

For the example given, there is some fibbing here. That is to say, the ID number should really be "6.1.2/3b1". Why? Because letters act like different "flavors" of decimal separations. There shouldn't be any privilege to them (e.g., "a" is not more important than "b", there is no sense of priority, etc.). I haven't added any further theorems concerning "6.1.2/3", so I lazily truncated the ID number of the example to "6.1.2/3b".

Title. The title is usually of the form:

title⟩ ::= ⟨register⟩ ". " ⟨summary

As a mathematician, I habitually write "definition", "theorem", "proof", "remark", "example", etc., in an abbreviated form ["defn", "thm", "pf", "rmk", "ex", respectively]. Then I write a human readable summary of the result, like "Convergent sequences are bounded".

I previously mentioned introducing a "code" register when writing literate programs, and discussed the content of such notes.

Recently, I have also been experimenting with writing notes on history, and I have found registers of little use there. So your mileage may vary.

Figure 4. Another example of a permanent note [front side]. NOTE: all links are written in red ink (using non-red ink was a bad idea).

Also worth mentioning, the first "folder" is about my Zettelkasten's conventions. An example slip from this "folder" is given in Figure 4.

Content of a permanent note. This is pretty subjective, and varies considerably. For mathematics, I have a great luxury here since every "register" is a separate slip.

I have found it useful for propositions [lemmas, theorems, corollaries, etc.] to give a "proof sketch" below the theorem statement. If there is some justification for a step, it's a branch off the slip.

Examples are just special kinds of propositions, and the emphasis of its content is the proof.

I can give some concrete "tricks" to writing permanent notes, they both boil down to relating new material to existing content already in the Zettelkasten.

Example: Creating examples. A useful exercise is to go back to examples existing in my Zettelkasten, and try to use it as an example of the new material. For instance, I have the "symmetries of the tetrahedron" as an example of a group. When reading figure 1, I may pose the question: can we express the symmetries of the tetrahedron as a universal arrow? (Not in any obvious way, but there are other gadgets which may be a universal arrow, so I keep looking.) When I find an example [call it a "widget"], I create a new permanent note entitled "Ex: widget (Universal Arrow)", the first sentence is "A widget [link] is a universal arrow." The "[link]" refers back to the original slip which first discussed the "widget".

Usually this process is more "subconscious" than actually flipping through every slip in my Zettelkasten. With the universal arrow example, I usually know an example would be an algebraic gadget, so I can restrict my focus to the "Abstract algebra" folder; even then, I can restrict attention to a few "subfolders".

Example: Translating content. Another way to approach creating permanent notes is to "translate" old material from a different domain to whatever I just read. For example, I learned the Krull-Schmidt theorem for rings and modules. But currently I am studying finite group theory. Could I frame an analogous theorem for groups?

Convention: formal proof sketches. I try to write the mathematical proofs using Wiedijk's Mathematical Vernacular, which is a synthesis of formal proof steps while leaving the terms and formulas informal. I also write "big step proofs", or what Wiedijk calls formal proof sketches.

Back of permanent notes. On the back side of a permanent note is a list of references which were used for the slip. They are lists to where one could find the same result in the literature which I've read, and refer to entries in my "bibliography apparatus".

Integration into the Zettelkasten. Usually, I study a book or article, write a number of permanent notes without any ID number. Only after I have extracted all the information needed from my source, I begin to think about how to integrate the new permanent notes into my Zettelkasten.

Sometimes the order of material in a book is "backwards" (it happens a lot in physics), so the corresponding permanent notes are in reverse order. Less frequently, the material needs to be partitioned to separate parts of the Zettelkasten. There has not been a time when the material is "completely disconnected" from each other, in the sense that they are added to different threads.

How big will the classification of finite simple groups be?

Gorenstein, Lyons, and Solomon (and friends) have been writing a consolidated proof for the classification theorem for finite simple groups...since 1994. In fact, it's still a work in progress. How big will it be?

Thus far, nine volumes have been published:

Volume NumberFirst page numberBibliography Page numberNumber of Pages
122159137
214216202
3 18 420 402
4 18350332
5 14476462
6 14532518
7 12352340
8 14498484
9 12530518

At present, there is an expected additional 4 volumes coming (according to David Roberts's answer on Mathoverflow). Performing a linear regression on the number of pages in terms of the volume number, we can make a prediction. Here's the R code doing this:

df <- data.frame(vol = 1:9, 
      start=c(22,14,18,18,14,14,12,14,12), 
      bib=c(159,216,420,350,476,532,352,498,530));
df$pgs <- df$bib - df$start;
lm0 <- lm(pgs ~ vol, data=df);
summary(lm0);

# Call:
# lm(formula = pgs ~ vol, data = df)
# 
# Residuals:
#     Min      1Q  Median      3Q     Max 
# -118.29  -53.62  -14.82   84.78  105.84 
# 
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)   
# (Intercept)   174.56      62.68   2.785   0.0271 * 
# vol            40.53      11.14   3.639   0.0083 **
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# 
# Residual standard error: 86.27 on 7 degrees of freedom
# Multiple R-squared:  0.6542,	Adjusted R-squared:  0.6048 
# F-statistic: 13.24 on 1 and 7 DF,  p-value: 0.008296

predict(lm0, data.frame(vol=c(10,11,12,13)))
#        1        2        3        4 
# 579.8889 620.4222 660.9556 701.4889 

sum(df$pgs)
# [1] 3395

sum(predict(lm0, data.frame(vol=c(10,11,12,13)))) + sum(df$pgs)
# [1] 5957.756

Tl;dr: the classification of finite simple groups is expected to be around 5957 pages, but could ostensibly reach up to 8243 pages.

Addendum

We can improve the estimate by dropping the constant term, and estimating the "additional pages" compared to the first volume:

df <- data.frame(vol = 1:9, 
      start=c(22,14,18,18,14,14,12,14,12), 
      bib=c(159,216,420,350,476,532,352,498,530));
df$pgs <- df$bib - df$start;
df2 <- data.frame(vol = 1:9,
      pgs=df$pgs - df$pgs[1]);
lm0 <- lm(pgs ~ 0 + vol, data=df2);
summary(lm0)

# Call:
# lm(formula = pgs ~ 0 + vol, data = df2)
# 
# Residuals:
#     Min      1Q  Median      3Q     Max 
# -122.24  -37.17  -24.70   92.68  125.61 
# 
# Coefficients:
#     Estimate Std. Error t value Pr(>|t|)    
# vol   46.463      4.901    9.48 1.26e-05 ***
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# 
# Residual standard error: 82.75 on 8 degrees of freedom
# Multiple R-squared:  0.9183,	Adjusted R-squared:  0.908 
# F-statistic: 89.86 on 1 and 8 DF,  p-value: 1.263e-05

predict(lm0, data.frame(vol=c(10,11,12,13))) + df$pgs[1]
#        1        2        3        4 
# 601.6316 648.0947 694.5579 741.0211 

sum(predict(lm0, data.frame(vol=c(10,11,12,13))) + 137) + sum(df$pgs)
# [1] 6080.305

This gives us a better fit (adjusted R-squared of 0.908) with a better error estimate (with 95% probability, it will be 6080 pages give or take 40 pages).

Tl;dr2: the classification of finite simple groups can be expected to be around 6080±40 pages long.