Demo/talk: a new peritextual mnemonic medium

Last year, I worked with authors to test the mnemonic medium (i.e. Orbit) in a bunch of contexts beyond Quantum Country. And… it didn't work nearly as well! Since late last year I've been working on redefining the primitives in response to readers' experiences. This demo/talk presents a new framework for the mnemonic medium in the context of a textbook, two non-technical essays, encyclopedic references, and an academic paper.

It's not a product demo video. It's the kind of thing I'd produce to be useful to my peers in the interface invention world, and to get the kind of feedback I need. So there's a lot of discussion about theory and ideas, not just showing off UI designs. But perhaps some of you will find that interesting from the standpoint of learning about how I approach designing something like this.

It feels really lovely to publish this—it's the first significant design artifact I've put out in over a year. My thanks to you all for helping make it possible.

Comments, questions, and criticism are all very welcome.

Script

This is a scripted talk, so I've included that material below for easier searching / reference for anyone who might want to write comments.

---

• Spaced repetition is an incredibly powerful way to learn, so it seems surprising that it’s not more widely adopted.
  • One key reason is that it’s actually quite difficult to write good prompts—that is, the questions and answers you review over time.
  • That’s why Michael Nielsen and I developed the mnemonic medium: maybe we can make it almost effortless to internalize a text if we have the author provide the spaced repetition prompts, and if we make those prompts more richly connected by anchoring them in the narrative.
• For the past two years, I’ve been exploring what the mnemonic medium wants to become as it expands to texts beyond Quantum Country.
  • Quite a few authors have adopted the system in various contexts now, but in most of them, readers are much less enthusiastic than they were with Quantum Country.
  • Gary Wolf nicely summarized the problem: by leaning so heavily on authors, we’ve actually created an authoritarian medium.
  • As I wrote in an earlier essay (which I’ll link in the video description) “the current interactions demand not only that you read the text in full, and in order, but that you repeatedly study—and commit to memory—whatever the author things is important, in whatever form the author chooses. The memory system isn’t “yours”; it’s on loan from the author, kept under glass.”
• Now, if you’re trying to study a well-specified topic carefully, and you’re totally new to that topic yourself, it makes a lot of sense to submit yourself to an authority!
  • Quantum Country works because it’s a primer. There isn’t much relevant variation in readers’ prior knowledge. Most readers won’t know enough about the field to choose specific subtopics they’d like to study. They’re unlikely to have strong opinions on how the material should be framed. And it’s okay that they can’t add their own prompts, because they’re too novice to make lots of connections with a creative project of their own. But they still want to internalize the material comprehensively, and deeply. These readers benefit from a highly guided experience.
  • But most reading doesn’t work like that. Most reading is less linear—more contingent on my prior knowledge, interest, and current projects.
  • I want to internalize the ideas I find meaningful, but I don’t necessarily want to grant the author carte blanche to assign me homework.
  • Even in a primer, I want prompts to feel less like fussy property of the author, protected under glass, and more like they’re “yours” as a reader, malleable material in your hands, there to help you deepen your understanding of what you care about most.
    • If I give you a textbook, I want you to break the spine and write all over it.
  • It’s tough to write good prompts, but that doesn’t mean authors should be the only ones with that privilege: we want to encourage active engagement, people making connections between the material and their own experiences or understandings.
  • In an ideal world, you’d have a personal memory genie sitting on your shoulder. And every time you read something you found striking, and every time something you read sparked an idea of your own, you could effortlessly remember that detail—and, if you like, return to that notion over time to stimulate further engagement.
    • That’s science fiction, but we’ll treat it as something like a north star. How close can we get to that degree of effortlessness?
    • More practically, we can think of this as an information problem. How can readers indicate which prompts accord with their interests, using the fewest interactions and making the fewest decisions? And how can they most fluidly capture prompts reflecting their own insights?
• There is no one-size-fits-all answer here. Different readers will want different things from any given text. And the same reader will read different texts—or even passages within the same text—in very different ways.
  • When you’re trying to learn a new subject, you really may want to defer to an author’s guided experience; but after a few projects under your belt, you’ll often read in a more mercenary fashion.
  • Reading a non-fiction essay, you might care only about the high-level claims, or you might want to be able to reconstruct the entire argument from scratch.
  • Reading reference or encyclopedic material, you’ll often jump to a specific subsection of interest and care only about a line or two, or even a few specific numbers or details within those lines.
  • A single text might inspire one reader to write extensively, and another reader not at all, while a different text provokes the opposite reaction, according to how each interacts with the reader’s own creative projects and interests.
  • The great thing about written language is that it copes well with all this ambiguity. The same page layout can serve a gossip column and a literary journal. Readers can skip around and drill in as they like; cues like headings and indexes help guide strategic reading; margins invite scribbling; and so on.
  • How might we make the mnemonic medium feel that versatile?

Demo

• Let’s start with a familiar concept for the medium: a textbook. This is Introduction to Modern Statistics, an open-access textbook I’ve been adapting as a test.
  • If I haven’t studied this topic before, I’ll probably appreciate a guided approach like Quantum Country’s. I’m going to read fairly linearly. I’ll want the author to make opinionated suggestions about what I should be internalizing here, and about when I might pause for a moment to review.
  • So I’ll focus on reading, and on trying to understand the material. After a few minutes, I’ll see that the author has embedded a review box where I can quickly test myself on key details from the text, and bring that material into my Orbit so that I’ll absorb it more deeply over time. So far, so familiar.
  • Now here’s one new detail: if I don’t care about a particular prompt, I can just skip it. We’ll move right on to the next one.
  • If I’m reading more casually, this kind of guided review might feel pretty heavy-handed. I might prefer to quickly get a birds-eye view of the material in this box. Then I can save any prompts which particularly strike me.
  • But if we’d continued with the review, or if we just switch back, then when we finish up, here’s what we’d see:
  • The prompts we reviewed—but not the ones we skipped—are saved to our Orbit. You’ll also see that reflected in the state of the prompts in this list here.
    • This default is meant to cover the common case: questions you cared enough to review are automatically saved; and those you skipped are not.
  • We make two quick bulk actions available for other common cases.
    • You might decide after reviewing: you know what? That wasn’t very interesting. I don’t want to go through those again. You can just click “Undo” here to reverse the automatic behavior and remove all the prompts.
    • Alternately, you might think: you know, those prompts I skipped… I didn’t feel like reviewing those right this second, but I do want to hang onto them, maybe see them at least once more. You can just click “Add all.”
  • Apart from the bulk actions, you can of course add and remove prompts individually here.
  • If you didn’t like how the author worded a prompt, you can edit it directly, right here.
    • My goal with this design is to give you the best of both worlds: you benefit from the author’s meticulous care in constructing this material—but this is your copy of the book, and it’s yours to tear up or scribble on as you like.
• What we’ve seen so far is a relatively incremental change to the existing mnemonic medium, because we’re looking at a reading context very similar to Quantum Country’s—one where I’m quite unfamiliar with stats, and I want to put myself in the author’s hands for a carefully guided experience.
  • But that’s often not how we read. In many situations, we’d be better off exploring the same stats textbook non-linearly, in a mercenary fashion.
    • To take myself as an example: I studied stats years ago in university, but as I analyzed reader data from Quantum Country, I noticed that I’m feeling pretty fuzzy on certain topics in stats. When I opened this book, my natural approach was to jump around, to focus on material which seems relevant to my project, or which feels unfamiliar.
  • In this context, even the new “list mode” we’ve seen in the review areas would feel pretty cumbersome.
    • Reading through, this material was new to me: I’d never learned how to fit a linear model by hand. I’d always used computers, but this gives me a greater sense of tactility for the concept.
    • The rest of this chapter is mostly old news for me, but I’d like to bring these details into my Orbit. How can I do that?
    • Well, I’d need to scroll down until I found a review box, then skip through all the prompts except the ones which relate to the concept I wanted to reinforce.
    • I could switch into list mode instead, but then I’d have to read through all of the questions to find only the ones which correspond to the concept I care about.
    • There are really two problems here: cost—I need to perform a large number of actions to indicate a small subset of interest; and decontextualization—I can’t indicate my specific interest where and when it actually occurs, while reading the text.
  • This new version of the mnemonic medium has an affordance which should help.
    • Let’s jump back to the passage I found interesting. Say I’ve just finished reading, and I think: wow, I really hadn’t understood this, but I think I get it now. I want to make sure I really internalize this concept.
    • When I have a thought like that, I can notice this little Orbit marker in the margin. This marker indicates that there are prompts available about the adjacent text.
  • When I click it, a sidebar appears presenting the relevant prompts. I can add all the prompts in one click.
    • Or if only a few of the prompts interest me, I can click to add each of them.
  • My goal with this design is to capture that instinctual spark in the moment—that feeling where you read something and go “ooh, juicy!” And, hopefully, to minimize interaction cost by using the marker’s position within the text to implicitly indicate your subset of interest.
  • In the context of a textbook like this one, readers can freely move between the two modalities I’ve shown, picking and choosing prompts as they go, or adding them in bulk as part of an end of section review.
    • The marginal Orbit markers offer a sort of peripheral vision as you read—hey, this passage might be worth remembering, and you can do so if you like.
• Let’s switch gears now and talk about non-technical essays, informal articles. These contexts show another way we might combine the two modalities we’ve seen so far.
  • My friend David Chapman very kindly agreed to try using Orbit in this philosophical essay he published early in 2021.
    • I often find David’s essays quite striking. So historically, I’ve gone to the trouble of writing lots of my own Orbit prompts about his ideas.
    • In many cases, it seems like it’d be even better if he’d done that work for me, or at least gave me a head start.
  • Now, David used Orbit for this essay just as I’d suggested, writing prompts to reinforce all of the details of his argument. But while Quantum Country readers were overwhelmingly positive, feedback about this experiment was more mixed. Many readers told us they found the experience unpleasantly overbearing—“like being in school.” I received similar feedback about several other articles which tried using Orbit.
    • I think the first issue here is reader stance. Quantum Country is explicitly instructional—as a reader, you’re there to learn. When the text asks you to review what you’ve read, it feels like it’s doing its job.
      • But when a non-technical essay like this one starts demanding partway through that you’ve memorized the author’s arguments, that can easily feel presumptuous.
      • My stance towards an essay like this will evolve while I’m reading it. I’ll often start quite casually, reading mostly for edification or curiosity. If the author really catches my interest, I might decide to study the text more carefully. The medium will offend if it demands the latter stance prematurely.
      • This phrase “like being in school” is telling. I’d guess this person means that what’s being asked of them isn’t aligned with the shape of their own interest. That’s the essence of being in school for most people.
    • The second issue is level of detail. David wrote prompts tracing all the fine-grained details of his argument, just as I suggested. That’s a great service for anyone who wants to really internalize this essay in extreme depth.
      • But mixed in with these detailed prompts are a handful of high-level prompts—by my count, 7 of the 60—which represent the main ideas of the essay.
      • It’s totally reasonable for readers to be interested in this essay at different levels of detail.
        • As an author, I believe it’s usually best to write for your most serious, demanding readers. They’ll want to internalize enough detail to reconstruct your full argument.
        • But that doesn’t mean you should actively alienate your (say) 80th percentile readers: those who are less invested but still quite attentive, and who want to carry your main ideas with them into future weeks.
      • In the initial design for the mnemonic medium, you could only offer the highest level of detail to everyone. To provide full coverage, you need to embed a review area every few hundred words.
        • I believe it would be better to follow the familiar design principle of progressive disclosure—to reveal the presence of detail, but to avoid having it take center stage unless the reader directs their attention that way. Ideally, again, with as little interaction cost as possible.
      • In our discussion of the stats textbook, we talked about people being interested in specific parts of a text, rather than the whole. That’s not quite the same as people being interested in different levels of depth, but I think the same primitives can help address both needs.
  • Here I’ve made a prototype of David’s essay using the new design elements.
    • Instead of confronting a review box after every few paragraphs, readers will see just one review box, offered at the very end of the essay.
      • This review box is intentionally high-level, focused on the main ideas of the essay. Just 7 prompts here.
      • Casual readers can easily treat this final review box as an appendix, if they aren’t interested any review at all.
    • But readers interested in more detail will see Orbit markers scattered throughout the essay.
      • Say that I’m particularly struck by this point David makes about marshes causing trouble for a correspondence theory of maps and territory. I can click this marker to easily bring that idea into my Orbit.
      • Likewise throughout the essay. This approach might sound fiddly, but it doesn’t mean there are 60 markers for the essay’s 60 prompts.
        • The markers indicate prompts about the adjacent passage. It’s an intentionally rough association. Prompts tend to cluster, and there are often several describing a single idea from different angles.
        • By my count, we’d use 22 markers for the 60 prompts in this 6,500 word essay. Spread uniformly, that would amount to roughly one marker every minute and a half of reading, though of course in practice some sections are denser than others.
      • Readers interested in more detail can indicate that interest as they’re reading, by clicking the Orbit markers.
        • They’ll see any prompts they add in their next review session, whether that’s in a review box at the end of the essay, or in the Orbit app alongside prompts from other essays.
          • Readers can also stop and review at any time, which might be helpful with a longer text.
        • As in the statistics textbook, the presence of the markers provides a sort of peripheral vision—more here, if you want.
      • Readers can incrementally opt into more detail.
        • Apart from the markers and sidebar, if you read through and complete the review box at the end, you’ll see the other available prompts here in the list tab.
  • Conceptually, none of the prompts “live” in the review box anymore. The review box is just the author’s curation of prompts which appeared throughout the preceding text.
    • So if you read through the text and complete the review box, you’ll notice that certain Orbit markers throughout the text are now marked as added, or partially added.
    • The unifying concept is this sidebar surface, which is now where all prompts “live."
      • The review boxes are like a window into parts of this sidebar. From a review box, you can always jump back to where a prompt “came from.”
      • Note that we don’t surface a literal character range associated with these prompts—they’re just associated with a general region.
        • I think this is important: in my design experiments, I’ve found that many prompts resist a precise association with the source text. This is particularly true of prompts which synthesize, distill, and connect—and these are often the most important ones.
        • A better metaphor is often a post-it note stuck to the side of a page, gesturing at a vicinity.
      • The sidebar gives the prompts a sort of “object permanence” they never had in past versions of the medium. Before, once you’d reviewed a prompt, it was gone. It didn’t “live” anywhere you could get your hands on it.
        • That ephemerality certainly didn’t foster the sense of malleability I want to create, but it created practical problems too. Sometimes people wanted to give feedback on a prompt that hadn’t worked well for them, but they couldn’t find it again!
  • The sidebar also provides a home for your own prompts.
    • For instance, when I read this passage on adventure rationality, I was struck by an observation my friend Alec Resnick has made about the phrase “bicycle for the mind.”
    • How does adventure rationalism echo Alec’s criticism of “bicycle for the mind”? As usually employed, it implies that you know the destination already!
    • I can add a prompt to bring that connection into my orbit right here as I read, and it’s a first class citizen, like those the author provided.
    • I can also add prompts during or after a review: I find that’s often a context where I’ll realize what prompts I wish I had.
  • The Orbit markers offer a relatively quiet peripheral vision. They just say “hey, there are prompts here.” But if I’m reading quite carefully, I can pin the sidebar open to keep the prompts in my periphery while I read.
    • Now I get a continuous extra channel of information about what the author views as important.
    • I may also find this mode more convenient if I’m writing a lot of prompts of my own—like keeping a notebook open alongside a text as I read.
      • I can just click in the margin alongside the text and start typing.
  • This design deliberately resembles margin notes in books and digital annotation tools, so it naturally suggests sharing.
    • Maybe if I find an article very interesting, I can mark it up using the Orbit sidebar and share my prompts with others, even if the author hasn’t provided their own.
    • Or maybe we should think of this as a shared, wiki-like surface, Genius-style.
    • I’d like to explore these angles, but at least initially I’m going to focus on expert-authored prompts: Quizlet and Ankiweb have amply demonstrated the challenges of crowdsourced prompt-writing.
• In my experiments so far, the pattern I’ve shown seems to work well for a variety of essays, informal articles, and blog posts.
  • Here’s another example: Donella Meadows’s famous essay on leverage points for complex systems.
  • As in David’s essay, I’ve put one review box at the end with a few curated questions on the main ideas, and Orbit markers throughout for more detail.
• But I’ve found that we’ll often want an even finer-grained affordance for texts with very high density of detail—texts like references, encyclopedias, and technical papers.
  • Consider this article on life expectancy from Our World in Data.
    • This is an encyclopedic, or reference-like resource, covering the topic from many different angles. Most readers will jump around, focusing on the aspects of life expectancy which most interest them.
    • Now take a moment to scan the first few paragraphs of this section. Notice how data-dense they are: “no country in the world had a longer life expectancy than 40 years”; “Global inequality in health was enormous in 1950: People in Norway had a life expectancy of 72 years, whilst in Mali this was 26 years.”
  • As a reader, what sort of affordances might be most helpful for me to carry away the details which most interest me?
    • If you add prompts to cover all this fine-grained detail, you’ll end up with one Orbit marker per paragraph.
    • I think that would feel pretty overwhelming. Visual noise aside, I’d be deciding every few seconds whether or not to click on an Orbit marker, and evaluating the prompts behind each.
    • But I think it would also be too ambiguous. Look at this third paragraph. There are so many details here! Say I’m particularly struck by this one sentence—“The global inequality in health was enormous in 1950: People in Norway had a life expectancy of 72 years, whilst in Mali this was 26 years.”
    • If this text weren’t so dense, I could act on my interest by noticing that there’s an adjacent Orbit marker and clicking on it.
    • But in a paragraph this detailed, the marker no longer does that job: I’d have to sift through a bunch of prompts about the other sentences to find the ones corresponding to the bit I’m interested in.
  • I think a more practical mental model for these Orbit markers is: “here the author has curated the key things to remember about this subtopic.”
    • Unless I’m reading very closely, I don’t want to be making constant decisions. I want the author to help me stay focused on what’s most important.
  • Here’s how that might look. This section has one Orbit marker, adjoining the paragraph which sums up the discussion.
    • If I found this section striking, and I’d like to carry it with me, I can click this marker, and I’ll see these five prompts which summarize the most important points.
  • Now, these prompts are still quite detailed. But sometimes I find myself wanting more detail. So I’ve designed an interaction meant for bringing fine details into your Orbit: things like numbers, proper nouns, dates.
    • When I first read this section, I was really surprised to learn that no country in the world had a life expectancy longer than 40 years.
    • I can easily capture that by selecting 40 and clicking “New fill-in-the-blank prompt.”
    • In the sidebar, I see a preview of the prompt which would be created. Orbit guesses that I’d want the surrounding sentence as context. But I can drag these handles to give myself more or less context.
      • Or I can manually edit the context down myself.
  • If I create multiple fill-in-the-blank prompts with overlapping context, Orbit merges them for me.
  • I’ll make the same interaction available for images, so that I can easily make fill-in-the-blank prompts for details in images like these maps, or figures like these from Cell Biology by the Numbers.
  • Now, this “fill-in-the-blank” type of prompt is often called a “cloze deletion.”
    • They’re easy to make, so new spaced repetition users often use them for everything: just copy and paste huge text passages verbatim and delete chunks. It’s an easy way to make prompts, but these often don’t work very well. The context is too noisy, or it’s not clear what’s being asked for, or you end up pattern matching, just parroting phrases back without really absorbing anything.
      • https://www.brainscape.com/flashcards/experimental-design-8660066/packs/15958967 A team of scientists in Stockholm are able to repeat an experiment originally conducted by a team in Atlanta. However, the results are dramatically different. This experiment was , but not .
      • 
        

• But in my experience fill-in-the-blank prompts have a higher success rate when used for focused, precise details, like the ones I’ve demonstrated. Numbers, dates, names, terms, diction. Ideally with tight context.
• I find that it’s often best to focus on higher-level prompts which synthesize and summarize; without those, fine-grained fill-in-the-blanks tend to create knowledge which feels atomized and brittle.
• I’m hoping that we’ll create a good balance by having authors focus on those higher-level prompts—which are harder to write—and giving readers this way to easily create super-fine-detail prompts.
• Now, in this kind of encyclopedic article, I wouldn’t expect a review box at the end of the page. Reading patterns are just too non-linear.
  • So for this type of content, I think readers would be well served by a handful of Orbit markers scattered throughout, plus the fill-in-the-blank interaction for finer details.
  • If it makes sense to offer even coarser, article-level summary prompts, you might put those in an Orbit marker associated with the abstract, or the conclusion.
• My experiments suggest the same pattern applies to technical papers. For instance, here’s the paper introducing IFPS, a distributed file system.
  • The structure here is pretty typical. The paper begins with a high-level motivation, then provides background situating this work relative to its predecessors, a summary of the high-level design, sections containing progressively higher levels of detail, and some concluding discussion of the work’s implications.
  • Different readers will want very different things out of this paper. Am I just curious about how IPFS works? Am I trying to implement a client for IPFS? Am I trying to understand the design so that I can build my own distributed file system?
  • Orbit markers let me indicate my interest on a topic-by-topic basis. Maybe I’m not really interested in the motivation or background—I’ve understood that already from other sources—but what I actually want to know is how the file system works.
    • I’ll focus on section 3, clicking the Orbit markers about the topics where I learned something new.
    • For the project I have in mind, this threshold on the size of “small” values stored directly in the distributed hash table is actually quite important. It’s not something most readers would care to take away from this section, so the author didn’t write a prompt about it, but I can capture that fine detail quickly with a fill-in-the-blank prompt.
    • Reading this list of potential use cases, I’m struck by a connection to another topic of interest, and I can add a prompt about that as I read.
      • In what sense might IFPS and TimBL’s Solid complement each other? You could use IPFS to implement a distributed Solid data pod.
    • This way I’ll be nudged to think more about my idea a few times in the coming weeks and months.

Summary and conclusions

• Wrapping up, let’s review the primitives of the proposed new mnemonic medium.
  • Prompts—both yours and the author—live in a sidebar, loosely positioned alongside the relevant source material.
  • Authors can surface these prompts in review boxes. As in today’s mnemonic medium, these give readers the opportunity to review what they’ve read while also saving prompts to return to later.
    • In this new design, readers can triage while they review by skipping prompts.
    • And a new list screen offers a birds eye view and bulk actions.
  • But authors can also surface prompts while you read, via Orbit markers. These mean: “click me to bring the main ideas from this passage into your Orbit.”
    • These markers let you jump around the text, picking and choosing which topics to take with you.
    • And if you like, you can keep the prompts in view while you read by pinning the sidebar open.
  • Authors can mix and match review boxes and Orbit markers to shape the reading experience:
    • In an essay, authors might embed a single review box covering the most important details at the end, while also possibly surfacing detailed per-topic prompts via Orbit markers.
    • In explicitly instructional texts like Quantum Country and the stats textbook I’ve shown, authors can embed review boxes regularly, at the end of each section, to help readers stay on track.
    • Meanwhile, encyclopedia articles and technical papers will probably use no review boxes at all.
  • Readers can write their own prompts which live alongside those of the author.
  • And for detail too fine-grained for authors to reasonably curate, a lightweight interaction allows readers to quickly extract fill-in-the-blank type prompts.
  • The process of triaging, editing, and writing prompts doesn’t end in the reading context: I’ll be adapting these same primitives for the Orbit app as well.
    • So if a prompt comes up that doesn’t resonate with you, you can just skip it, and it’ll back off for increasingly longer intervals. Or you might improve the wording. Or, in many cases, it’ll be best to simply delete it.
    • The list representation of prompts we’ve seen in the sidebar and in the list tab of review boxes will have an analogue in the app for browsing and editing your library.
• So far, these new primitives exist only in smoke-and-mirrors prototypes.
  • Next, I plan to build higher-fidelity prototypes to test with readers in some real contexts.
  • I also have a lot of tricky design work to do in designing onboarding flows for everything I’ve shown. In this video, I’ve focused on the primitives, playing the role of a familiar user. But of course the real system will have to help introduce itself to new users.
• Thanks for watching. Criticism, questions, confusions, and riffing are all welcome.
  • My special thanks to Ozzie Kirkby, who prototyped some early approaches to this problem last summer, and to Nick Barr, Taylor Rogalski, and Gary Wolf for helpful discussions.