Working with authors: text-writing requires prompt-writing requires text-writing

A few weeks ago, I had something to celebrate: Orbit had reached the point that authors could publish texts using it. Normally, when software projects reach a major milestone, there’s something highly visible that “goes live”—something to link others to! But in this instance, it’s a piece of infrastructure that’s “gone live”; now the work is in helping others use it to create something meaningful.

As I’ve worked with authors, I’ve realized something important I should have seen much earlier: authors who don’t already have a successful personal spaced repetition practice have a really hard time writing effectively with Orbit. This makes perfect sense in hindsight. Michael and I had already been using Anki for years; we drew heavily on that experience when creating Quantum Country. If we’d come in cold, without ever having written spaced repetition prompts, there’s no way we could have done it.

I’d been thinking of the challenge for authors in terms of “communicating effectively with prompts,” and that’s real—but they also need the basic skills of knowledge modeling that come with a personal practice. The challenge is that now my dependency graph has a cycle in it.

One of Orbit’s main theses is that writing good prompts is a huge barrier to widespread adoption for spaced repetition systems, but we can significantly lower that barrier by interleaving expert-authored prompts into contextualizing narrative. As you read more expert-authored prompts, you’ll absorb how to write in the medium yourself. Then the barrier will be lower to writing your own prompts and establishing a personal practice.

But if you need a strong personal spaced repetition practice to write texts with Orbit, and you’re meant to develop a personal practice by reading texts written with Orbit, we’re in trouble. We’ll need to bootstrap the cycle. Maybe we can start with many authors who already have a strong personal practice… but there are so few such people that this seems unlikely.

I think a more plausible approach is to put a lot of effort into helping authors develop the skills needed for a strong personal practice. I expect this to be an easier task than helping the general population learn to write prompts well: authors already think deeply about representing ideas in words, and they care a lot about precision.

I had thought I’d start working with authors by writing resources about how to communicate well with prompts. But now I see that the first step to writing good texts with Orbit for others is being able to write good prompts for yourself. So I’m spending this month writing a small handbook on prompt writing. I’ve invited a number of authors to join me for a small workshop with authors to activate those concepts and to help me iterate on the handbook. If these go well, I may hold more workshops for wider audiences. In any case, I’ll publish the handbook publicly (and first for you all), assuming it ends up worthwhile enough to publish. I’m planning to use Orbit when writing the handbook, to help readers retain the key ideas and techniques.

There have been a number of articles about writing good spaced repetition prompts, but they all feel too structureless—loose bags of tips like “avoid lists.” I think it’s possible to understand in terms of more fundamental principles what makes some prompts work and others fail.

In particular, one model that’s helped me is to understand that when you write a spaced repetition prompt, you are giving your future self a recurring task. Prompt design is task design. If your goal is to build recall, the purpose of those tasks is retrieval practice. That’s a principle from cognitive psychology which I’ll unpack in more detail in the handbook, but in short, you must design tasks which, when enacted, require you to retrieve the knowledge in question.

The process feels surprisingly similar to translating text between languages. When translating a passage, you’re searching for words which, when read, light up a similar set of bulbs in readers’ minds to those which might have been activated by the original language. It’s not a rote operation. If the passage involves allusion, metaphor, or humor, you won’t translate literally; you’ll try to find words which recreate the experience of reading the original for a member of a foreign culture. When writing learning-oriented prompts, you’re performing something similar to language translation: which tasks, when performed, require lighting the bulbs which are activated when you have that idea “fully loaded” into your mind?

If the idea is fairly simple, it may be possible to directly conceive a task which reliably lights all those bulbs (and not other extraneous ones!). But when the idea has too many important facets, it’s hard to design a task which reliably stimulates all those elements. So you have to break the idea down into many tasks. This is where knowledge modeling comes into play. Given a piece of knowledge, how do we express its constituents and relationships? The answers differs for declarative, procedural, and conceptual knowledge, but there are consistent frameworks one can use for each.

————————

Housekeeping note: if you live in Europe, you can now choose to fund your membership payments in pounds or euros instead of dollars. My thanks as always for your kind support. I can’t believe this little lab community now has over 400 people in it!