Demo/talk: breaking the mnemonic medium out of its box

In this talk, I present a new round of primitives for the mnemonic medium, focused on the issues and opportunities I distilled out of the last round of prototyping (see that talk here).

For those who aren't interested in this specific project, you may still find it interesting to see me unpack how I think about iteration on the architecture of an unusual design problem like this one.

Feedback is very welcome. I've included the text of the talk below to make it easier to point to passages (but don't expect it to make much sense without the accompanying visuals).

Script

• Welcome.
  • Today I’ll be sharing a new iteration of the mnemonic medium.
  • Like the designs I presented in May, this work is trying to solve a specific problem: the medium created for Quantum Country felt rigid and unpleasant when used outside of technical primers.
    • How might we create a more flexible mnemonic medium—one which could be used in non-technical books, informal articles, reference material, papers, and so on?
    • In this video I’ll be assuming you’ve watched the May talk. If you haven’t, I’d suggest watching the first five minutes, and minutes 10-13, for background on the problems I’m trying to solve.
• Now, here’s our plan: first, I’ll quickly recap the core elements of the design I presented in May; then I’ll describe the problems I uncovered through reader observation and design crit; and finally I’ll present a new version of the medium intended to address those problems.
• OK! First, here’s a two minute recap of the design from my May talk. [play 26:05-27:41 from May talk]
• To test this design, I implemented a prototype in the context of a relatively informal product management book called Shape Up.
  • I chose this book because I thought it would exacerbate many of the problems I’ve seen with the medium.
  • The material is often contingent, arguable, heuristic, a matter of values or opinion.
  • Readers’ experience with this book will vary dramatically with their background: relatively inexperienced test readers will want to study it very carefully, while more seasoned readers will skim, mining for pearls to carry away.
  • It’s not formal, not technical; and yet it’s a serious treatment which, for some readers, will reward study.
• In June and July, I observed a dozen people read the prototype book live, while speaking their thoughts and reactions aloud. I also received asynchronous think-aloud diaries from a few dozen additional readers, and benefited from thoughtful crit sessions with some generous designer friends.
  • What I learned from those readers was encouraging—I think the medium can be made to work in a context like this book, and I think doing that would be extremely valuable to many readers.
  • And yet, unsurprisingly, these sessions also uncovered many problems. Some might seem like surface-level design issues—certain things just need more polish—but the troubles I’ll be talking about today are truly structural. The right way to fix them is with changes to the conceptual architecture of the medium.
• Now let’s dig into some of those problems. Broadly, they can be divided into two groups: first, issues with the peritextual Orbit markers and sidebar, and second, issues with the embedded review boxes.
• The Orbit markers and sidebar presented an interconnected series of issues:
  • First, a broad observation: the prompts feel unnecessarily “far away” from the text.
    • That’s true literally, if you’re using a large monitor—they’ll be horizontally separated by a big white space.
    • But it’s also true conceptually: there’s a surface boundary in the way. The prompts are trapped in this sidebar box, separated from the text they reference.
    • Niko Klein pointed out to me that if you watch how people actually mark up their books, they’re not nearly so orderly. Their annotations happily impinge upon the text.
  • In this design, Orbit prompts are timid, hesitant to touch the text. There are markers in the margins, but no indications in the text itself of connections to associated prompts. Readers must mentally draw connections between the prompts and the source material.
    • Sometimes that’s appropriate: some prompts synthesize or summarize from a high level, and there’s no manageable length of text one could visually associate with them. A certain indirection is necessary.
    • But plenty of prompts focus on verbatim phrases from a text passage. It feels strange to make that connection so indirect.
  • Another issue of inappropriate indirection: for Orbit markers which hold just one prompt, the distinction between marker and prompt feels far too heavy.
    • I notice this marker here; I click it—and it reveals both this single prompt over there, and also a contextual menu here with a single button to save this prompt.
    • But the prompt itself also has a button I can click to add it.
    • Here we have two different abstractions which both effectively represent the same conceptual object. At least in this case, the distinction between these two roles isn’t meaningful enough to warrant the doubling. And the connection between them is inappropriately indirect, indicated only by the prompt’s colored trim and its rough spatial position.
    • Can’t these two things be the same thing?
  • Of course, the main reason I distinguished between Orbit markers and the prompts themselves is that in many cases, Orbit markers represent several prompts, collectively reinforcing the same concept from different angles.
    • The first chapter of Quantum Country has 112 prompts. You wouldn’t want 112 of these markers floating in the margin, particularly since many of the prompts are about the same idea.
    • Bulk operations are important for controlling the interaction cost of the medium. That’s really what motivated these Orbit markers—bulk operations.
    • That actually illustrates another issue with the markers: they feel subtly like they’re hiding information. Does this marker represent one prompt or six? I have to interact with it to find out. But that information is a useful cue: if there are lots of prompts here, that tells me this is an important passage. If I find this passage only mildly interesting, I might skip right over a six-prompt marker—too much bother—but I’d consider a one-prompt marker.
  • Perhaps in part because of the way that markers “hide” the prompts on this separate surface, many readers found themselves fiddling with the sidebar, uncomfortable both with leaving it pinned open and with leaving it closed.
  • Finally, the markers create unintended asymmetries—which are often a sign for a designer that you haven’t got your abstractions quite right.
    • The first asymmetry is that if you write your own prompt, there’s no indication of that in the text itself. The marginal Orbit markers are only for the author’s prompts.
    • A second asymmetry falls out of this: the interface readers get for writing prompts can’t be the interface authors use for writing prompts. In the May design, authors also need to place the Orbit markers and define how the prompts are divided between them. Asymmetry aside, that extra work seems unfortunate and perhaps unnecessary.
• The markers and sidebar work in tandem with review boxes embedded directly into the text. These elements had their own separate conceptual problems.
  • As a reminder, in the context of Shape Up and other relatively informal texts, the idea was that review boxes placed at the end of a chapter would contain a curated selection of the most important ideas—maybe 5-10 prompts. But readers could opt into saving much more detail while they read by clicking Orbit markers next to passages they found particularly important. These prompts would be added to the review box at the end of the chapter.
  • One problem with this scheme was that the review box’s behavior confused attentive readers who saved extra prompts while reading. Those embedded reviews include both the prompts curated by the author and also the prompts you yourself saved, without any clear distinction.
    • It makes sense for these review boxes to show a curated selection of prompts as a sort of guided scaffold, but including reader-saved prompts muddies the conceptual identity of the review box as an authored object.
    • Worse, when you’ve gone to the trouble of evaluating prompts as you read, choosing which to save—and more importantly, consciously deciding not to save some—it feels intrusive to have extra prompts inserted into the review, because it feels ambiguously like the point of the review is to look at the prompts you saved.
  • Apart from this confusion, the reviews felt “too soon” for some readers.
    • We didn’t get this kind of feedback with Quantum Country. One reason for the difference is probably that the material in Shape Up is much more straightforward. You’re less likely to have forgotten so quickly.
    • But there’s a deeper issue: if you’re examining and evaluating prompts in the sidebar as you read, that means you’ve already done one informal review by the time you reach the embedded review box. It makes sense that this would feel too soon—you already saw those prompts just a few minutes ago.
      • This is very different from the experience in Quantum Country, where you might have just read the text a few minutes earlier, but you haven't seen the actual questions before your first review.
  • Other readers were highly mercenary. Much of the material was already familiar to them, and they were scanning for useful pearls they could take away. These mercenary readers found an embedded review session overly demanding, even though it can be switched to a list or simply skipped. The implication that they’re “supposed” to review the material felt misaligned with their stance towards the book.
  • I think the meta-problem here is this: with the last prototype, I started by assuming Quantum Country’s basic design and added new primitives to handle more situations.
    • But that led to a conceptual rift of sorts. There’s the world of the sidebar and the inline prompts, and then there’s the world of the embedded review box. I built a bridge of sorts between the two, but they weren’t yet conceptually unified.
    • With this new design work, I explored what would happen if I didn’t start by assuming an embedded review box. Instead, I tried to design first for readers of a book like Shape Up, then I worked out how those design elements might be elaborated to accommodate a more demanding textbook like Quantum Country.
• With that, let’s take a look at a new iteration of the mnemonic medium, where I’ve tried to address all these issues.
  • As in the May talk, these designs will assume that the reader is already familiar with the medium and with spaced repetition. The designs will require extra cues and flows for onboarding new users, but I’ll focus on those once I’m happy with the primitives.
• First and most dramatically, I’ve unified three primitives—the marginal prompts, the Orbit markers, and the sidebar—into one.
  • A subdued prompt representation lives directly alongside the main text, rather than in a separate sidebar.
  • When there’s a single prompt associated with a passage, we don’t need any kind of bulk indirection. We just see that prompt directly.
  • Its full contents are unfurled on hover, and we can simply click it to save it to our Orbit.
    • We’ll see it in our next review session, or we can start a review anytime through the floating Orbit menu.
  • This prompt reinforces a verbatim phrase in the text, so that phrase is now highlighted. Hover interactions reinforce the association.
    • Not all prompts will be associated with a specific phrase like this, but we can surface that connection, where it exists.
  • In previous designs, I tried so hard to make the prompts unobtrusive, so that they wouldn’t bother readers who didn’t care about memorizing every detail. But the insight which really unblocked me here was that once a reader has taken action to save a prompt, that prompt can absolutely assert itself visually. When you mark up your own book with a colored highlighter, you don’t need to worry about the mess which you yourself make.
• I believe bulk interactions are still important for the mnemonic medium. Most of the time, readers will want to indicate their interest in a concept rather than in a specific prompt—and good coverage of a concept will often require several prompts.
  • In this iteration of the medium, overlapping prompts are collapsed into a “bulk” representation alongside the text.
  • Without any interaction, we can see how many prompts are involved. If we move our cursor over here, we can quickly save all the prompts with one click, or more selectively evaluate which we might like to keep.
• The authoring experience can now be made symmetrical for both readers and authors: if we add a prompt of our own, it’s presented alongside prompts we’ve saved from the author, with the same highlighting affordances in the text. No need to place Orbit markers and define their associations with prompts.
• A tighter connection between the text and the prompts lets us play one more trick: if we’re excited by this phrase, and we find ourselves wanting to internalize it, we might select the text as if to write a prompt. But if it’s something important, the author may have already provided a prompt about that idea. So when the anchor of an authors’ prompt intersects a user’s selection, we surface the author's prompt and make it easy to save.
  • And of course, you can edit the author’s prompt to better match how you think about the idea, if you like. Editing is often easier than creating.
• Obviously, this style of floating prompts isn’t going to automatically work in every web layout.
  • In simple cases, like where there’s a sidenote or figure in the margin, we can try to automatically shift the prompts to dodge those elements.
  • In cases where the body text is laid out tightly against the edge of the screen—and, of course, on mobile—we’ll need some kind of “collapsed” representation.
  • And the user should be able to hide these things if they’re feeling obtrusive.
  • But at least for the moment, I’m interested in exploring the best case, where the author lays out their text with this medium in mind, or perhaps where someone has create a user stylesheet which can adapt the site’s layout appropriately. I’ll be interested in dealing with less than ideal cases once I’m convinced that the ideal cases are really working.
• So those are the new floating prompt primitives, replacing the Orbit marker, the sidebar, and the sidebar prompts.
  • Now, these are all secondary interactions, occupying the same structural place in the hierarchy of the text as marginal sidenotes would. They offer extra depth to highly attentive readers, without asserting themselves in the primary linear flow of the text.
  • But just as figures can appear either in the margin or within the body of a text, it makes sense for authors to be able to present prompts both as marginalia and as objects in the body of the text—for instance, at the end of a chapter, or after a passage summarizing the takeaways of a section. How often and how many prompts should be presented this way will depend on the book and on its intended audience.
• In the case of Shape Up, I think it would be quite valuable to offer a short list of curated prompts at the end of each chapter, covering the most important ideas.
  • Because this book is relatively informal, and its material is quite intuitive, I don’t think we need to strongly push readers to review these prompts immediately by default, as the previous iterations of the medium did.
  • The trouble with a book like this—as I’ve learned talking to readers—is not in understanding the content while you read, or in remembering it later that day. The trouble is remembering the details days or weeks later, when you first try to apply them.
    • So the primary job of the mnemonic medium in this case should be to make it easy for people to circle back to the key points in review sessions on a future day.
  • Here’s how that might look: a lightweight module which can be integrated directly into the body of the text.
    • Structurally, this module is similar to the kind of summary callout you might find in a printed book, except of course that it’s part of a larger interactive workflow.
    • I might scan my eyes over these prompts, decide they look generally good, and click to save them all.
    • Or I might do a quick triage, scrubbing my mouse down the list to take a look at each, adding those which seem surprising or meaningful.
      • Note that this scrubbing interaction doubles as an extremely lightweight informal review.
    • This module’s contents are defined by the author. It doesn’t try to double as a list of the other prompts you’ve saved earlier in the text, as the previous design did.
  • Now, if I’m studying this book seriously, and its material doesn’t feel so intuitive, I probably would benefit from doing a quick review right now.
    • I can click here to make that happen.
      • And now the review experience doesn’t need to be trapped in a little inline block—we can create a focused modal context.
    • I can review as normal, or skip prompts which don’t interest me.
    • I’m only reviewing the prompts I was just looking at in that box—other prompts I’ve saved aren’t mixed in.
    • But once I finish reviewing, the interface inquires whether I might like to additionally review the other prompts I’ve saved so far.
      • If I’ve opted into reviewing these curated prompts, I’m probably reading carefully, so I probably want to review everything else too. But by making this a separate “phase” of the review session, we avoid the confusion introduced by the previous review box’s design, which mixed curated and saved prompts together.
    • After the review session is complete, the new prompts I reviewed are automatically saved to my Orbit. That's probably the most common desire for someone who’s opted into inline review. But of course, I can undo this behavior or modify the set of prompts I’ve saved in the list below.
• By embedding prompts directly into the body of the text, rather than into the margin, authors can be a little more assertive. It’s a way to say: hey, you should probably consider carrying some of these prompts with you. The reader gets the sense of a “default.” More attentive readers can up the stakes by choosing to review that material on the spot.
• But for more formal instructional texts like Quantum Country or this stats textbook—particularly where the material is more difficult to grasp—authors should be able to express a more demanding default. They should be able to say: reader, you should probably review the key ideas from this section, right now.
  • For that situation, the mnemonic medium's traditional embedded review box still makes a lot of sense.
    • These interleaved reviews do more than just reinforce the material on the spot. Many readers tell us the embedded reviews make them slow down and read—or reread—more carefully. Other readers tell us the reviews create a feeling of safety: they’re in the author’s hands, and they know that they won’t be able to read too far ahead of their understanding.
      • More subtly, I suspect that inline review anchors the prompts in the context of the broader explanatory prose, so that when you see a prompt, it brings to mind that section of the text and all the connected ideas within. That may help avoid a key problem people have with separately downloadable spaced repetition decks: a feeling that the prompts are disconnected, atomized, or arbitrary.
    • So we shouldn’t abandon the idea of embedded review. It’s a significant imposition on readers, but it also offers significant benefits. When those benefits are likely to be higher than the costs for the primary audience of a text, the author should make interleaved review the “default.”
  • But we can still refine that primitive.
  • In this iteration of the design, embedded reviews include only a set of prompts curated by the author, rather than intermingling any extra prompts you might have written or saved. But if you’ve added some extra prompts besides those, we’ll ask the reader if they’d like to tackle those after they finish reviewing the curated prompts, just as we saw a few minutes ago in the modal reviews.
  • As before, when you finish reviewing, those prompts you reviewed (but not those you skipped) are automatically saved. And you can easily undo this.
  • Readers who have some familiarity with the text’s materials may not want to bother with this sort of guided review. They can switch the inline review module to the prompt list module we saw previously, and save any prompts which seem unfamiliar.
• In a text like Shape Up, which is relatively informal, and whose readers will vary enormously in background and interest, it makes sense to offer multiple levels of detail in the prompts.
  • The prompt lists at the bottom of the chapter are tightly curated to include the most important ideas—things which any serious reader should probably take away from the book. Readers interested in more detail or in secondary ideas can save prompts about those ideas as they read using the floating prompts.
  • But in a highly guided textbook, I think it would usually make more sense for the embedded review module to include every prompt from that section. Readers will tend to express their interest by choosing which sections they read and review, rather than by picking and choosing individual prompts.
    • It may not make sense to distract readers of highly guided textbooks with floating prompts—they’re going to review all those prompts anyway, in a few minutes.
    • And so perhaps authors can express a site default for this floating prompt visibility setting. A textbook might set this to show only saved prompts by default, so that floating prompts won’t distract while you read initially, but after you’ve completed that section’s embedded review, you'll see the saved prompts in context above.
    • Of course, a reader could change that setting away from the default if they prefer.
• In summary, this design collapses the Orbit markers, sidebar prompts, and sidebar into a single floating prompt primitive. I’ve more crisply defined an embedded prompt list primitive, and refined the behavior of the embedded review primitive.
  • Next up, I’ll do another round of design crit and revision, then build out this prototype so I can learn how it makes readers feel and behave. Your feedback is very welcome.
• I’d like to thank all the readers who tested the most recent prototype and provided feedback, synchronously or asynchronously. And I’d particularly like to thank the designers who generously offered extended crit: Cameron Burgess, Shan Carter, Gray Crawford, Joe Edelman, Niko Klein, Marisa Lu, Rob Ochshorn, and Yiliu Shen-Burke.