May Highlight - The Telelibrary

An ongoing interest in my practice is creative data - I try to think carefully and expansively about what data my exchanges create, and what can be learned or made by applying it creatively. The trade off is that data collection can be fairly resource intensive; it’s hard to find the time to properly process, sort, and document everything I’m taking in, and even harder to make time to really sit with that information and start to find what stories emerge beyond the intuitive.

Luckily, thanks to your support, this month I’ve been able to do all of the above, with the help of Dr. Juan Felipe Beltrán, a brilliant, creative, deeply thoughtful (and very funny) PHD working in the fields of programming/bio/data science. You’ve all made the mistake of displaying a tolerance for charts and graphs in previous posts, so we really backed up the truck for this one.

As I shared in my AMA with Sarah Reynolds, a broader goal of this Patreon for me is to use these resources to collaborate with dear friends and colleagues, so I’m so excited to share some early findings with you! You can expect more deep dives on The Telelibrary Data in future Archive Highlights, but until then, The good Dr. Beltrán encourages anyone curious about questions or careers in those fields to drop him a message at https://twitter.com/offbyjuan!

Alright, on to May!

~

A Garden of Branching Paths

“Nor am I the first author of the tale ‘The Library of Babel...”
Foreword, The Garden of Branching Paths. Jorge Luis Borges.

All Data Visualizations are by Juan Felipe Beltrán, P.H.D., and based on an analysis of Users #159 - 534, captured 3/28/2021, unless otherwise indicated.

Warning: Spoilers for The Telelibrary ahead. Most are more structural than they are content related, though both will be discussed. If you prefer not knowing how your sausage is made, this is not the highlight for you!

As I’ve mentioned in a couple interviews and discussions, when I started The Telelibrary, I was gathering data with absolutely no idea what the applications may be. As you’re aware if you’ve ever had a session, I ask each User if I can record the audio of the session, which is one form of documentation (for the AMA, one of you asked me about file sizes. The full Telelibrary drive is currently at 340.65 GB — and yes, it’s backed up).

However, the other major sources of data and documentation are my notes.

Identifying User Info redacted

During a call, I keep it fairly analog. You can see two different systems of notation here, for 3 different calls (on the left are returning Users, which requires a bit more detail). This is both a useful back up and a means of helping me focus during calls; I jot things down and can make a note of content thatI’ll want to return to or link to something else, either during this call or on another. You can also see an “NR” - that’s a User who isn’t recording their call.

Once a call is complete, I then switch to the computer to take down a complete shorthand of the call. First names (System and User, as dictated by the User), then every selection they made and feature they heard, every feature they’ve unlocked so far, and finally additional notes on the call. Sometimes this is a thread of conversation I want to explore, or continue. Sometimes it’s technical notes on some features that need tweaking, or audio levels that have to shift. Sometimes it’s a new idea to try in a future call. All together, it looks something like this:

(But we’ll get into all that another time)

Finally, for each call I do an even more pared-down notation of which Selection numbers Users chose. If you are one of the Users who have taken part in the construction of the User Stock Market, you are familiar with one application of this data (and if you’re not one of those Users, that last statement was probably deeply confusing*). 

*Essentially way back in User Logbook Vol. 3, User #280 invented a way for Users to bet on which Selections other Users would pick, and earn marginal gains in a gradually increasing account every time they were right)

However, this data also gives us a chance to visually track the paths Users take as they explore the library. For starters, here’s a breakdown of the frequencies of any given User’s first choice:

Looking at this chart (the only one generated by me), you’ll note that almost 4% of the time, when told to pick a number between 1 and 7, the User chose something completely different

There’s a few selections favored over others, but all told it is a fairly even distribution (perfectly even would be ~14% each). Looking at the chart below of the number of times each Selection has been visited overall, the initial 7 Selections are fairly close. There’s a few selections favored over others, but all told it is a fairly even distribution (perfectly even would be ~14% each). Looking at the chart below of the number of times each Selection has been visited overall, the initial 7 Selections are fairly close.

Again, you’ll notice Selection 0 has been picked a not-insignificant amount of times, despite not formally existing. 

As a performer, I have a general impression of what gets picked a lot, particularly as the first choice, but having the hard data can help better inform my choices as I rotate and balance selections — choices which ultimately come full circle and affect the data (Selection 6 is consistently a “compound selection,” where Users can return for a few more excerpts, which in part explains how it ends up so high in the ranking).

However, data like this can also paint a picture of what a User’s flow through their experience looks like. In the following chart, you can see the most common pathways between selections, with heavier lines representing more frequently made progressions. There’s a definite bias towards a few numbers here, but just like the data above, the skew isn’t so overwhelming as to tilt things entirely out of balance.

Way back in the first beta tests of The Telelibrary, that last image was a good model for the whole experience for Users - bouncing between selections and letting themes emerge and disappear until time runs out.  However I quickly found that I wanted to give the world of the piece the ability to open up and get larger, and it wasn’t long before I had added two more levels of Selections (8-13, and then 14-19). In addition to adding another dimension to the interaction (you can now listen to AND unlock Selections), this also seems to suggest a sense of momentum, which in turn appears to have a strong effect on User behavior**.

**I do my best to foreshadow this momentum, and to signal that nothing is static: the main menu is introduced with the instructions to “please listen carefully to our menu options, as they change suddenly, and without warning.” From that point on, if I’m doing my job, you’ll never get the same menu options twice. It took a lot of planning and one very ugly hand drawn graph to plot out what features would show up when to make that possible, but that is a graph for another post.

As you take another look at the chart of User behavior from before, notice how much more those pathways converge.

Again, thicker lines represent more frequent choices, which shows us that as soon as Selections 8-13 are unlocked, Selection 13 is far and away a favorite. Maybe it’s because it’s the highest available Selection at the time (or maybe In fact, it is so common a choice that only Selections 11 and 12 are common enough alternatives to register on this scale. When I noticed this early on, I began placing longer, more expensive, and more complex Selections in the 11-13 slots (feed-the-meter, compound selections, catalog entries, etc.), which ensures that most Users are encountering new and different content (and using enough credits to need to add a credit again). You can also see that as soon as Selections 14-19 are unlocked Users again move on, gravitate to a smaller range of Selection numbers, and less likely to return to earlier content, especially as unlocking Selections 14-19 quickly leads to the discovery of many of the User generated content in the piece. The result of all of this is the suggestion of 3 “zones” or areas of content, and while the boundaries around them aren’t so hard as to mandate certain choices, there’s enough momentum and suggested flow to safely give each zone different characteristics and qualities.

Same graphing idea, but giving a clearer picture of what happens each time the range of selections increases (unlock_1 is 8-13, and unlock_2 is 14-19). Hopefully with more data and as I keep track of more details, like User content).

While much of the data above affirms intuitions and impressions I had already formed as a performer, one particular visualization from Dr. Beltrán represents a lens for understanding User behavior I had never considered before. In the visual below, you can see the correlation between choosing any two given Selections (e.g. users who visited S_ also visited _), with positive correlations in green, and negative in pink, and darker colors representing stronger correlations.

Dr. Beltrán has kindly indicated some of the clusters that emerge when you do this, though I’ll freely admit that there are many here that I really can’t account for (check out Selections 1 &13, and the whole cluster around them and 3,5, and 17. However, when we instead look at Selections in plain order, there are a couple trends that stand out to me.

For starters, You can see that choosing Selection 13 pretty strongly precludes choosing Selections 9,10,11, and 12. As I mentioned above, Selection 13 tends to be time/credit intensive, so Users may simply not have the resources for the other middle options. However, the clearest pattern to me is the really strong tendency for Users to not pick adjacent numbers, particularly in the initial 7 selections (look diagonally from the top left to the bottom right. Obviously I can’t say for certain what causes this, but I can’t help but think about the visual of the initial 7 selections above, and the ways I tried to write the first 15 minutes of the piece to train people to explore and hop around. It’s not just about building an expectation I’m about to break; it’s about fostering a curiosity and willingness to explore that will serve the User no matter which “zone” of the piece they are living in.

It’s worth noting that these flow charts and conclusions can’t be said to be definitive on the subject of “picking a number between 1 and 7” (this is not going to help you win any lotteries). In fact, it’s not even definitive for The Telelibrary.  Most of these visuals only represent the majority of User actions. The Telelibrary invites a significant degree of agency for Users to chart their own path, and the true mapping of User experiences isn’t nearly so simple.

When we lower the limits to visualize less common choices (frequency<10), things get pretty messy

There’s a rule I repeat to myself often, particularly when Users begin trying to test the limits of the System, seeing how far they can break what is suggested above; “never offer a participant a choice you don’t welcome them to take.” I consider it my job to ensure that I’m not giving people options that neither of us would want to happen. Similarly, looking at the visualizations above, I think it is important as a designer of interactive and non-linear experiences to take responsibility for the kinds of pathways that are suggested and implied by your design choices, intentional or not; not all of the choices we give a participant are equal, and if we’re stacking a deck in any direction, we had better stand by that angle (and be prepared for the alternative).

I’ve had the enormous luxury of being able to tinker and adjust this piece over the course of more than a year, which has provided me plenty of chances to ask myself: “What am I inviting Users to do? What’s the effect? How can I improve it?”