March Updates: I made a neural network from scratch!

(⏱ 6 min read)

Three updates!  Two big updates, and one kinda-update!

🧠 Update #1: I'm making a neural network toy!

I'm making an interactive visualization of an Artificial Neural Network, from scratch!  Look, you can directly click & drag to change its inputs, weights, and biases... and see all the intermediary activations.  GIF below:

(Not pictured above: it can also learn by backpropagation, and you can even see the "errors" propagating backwards through the network!)

What's Left To Do: A sidebar with more settings to edit the network.  I also want to visualize the cost function as a nice spinning 3D graph, so you can actually see gradient descent in action.

Mockup of the sidebar so far:

Motivation for this project: As Feynman's blackboard said when he died: "What I cannot create, I do not understand".  I've seen a lot of theory & math of neural networks before, and I have played with code libraries that handle the neural networks for you... but I'd never made my own neural network fully from scratch.  It forced me to learn... stuff.

(The three stages of learning about backpropagation; 1) First impression: Woah what is this complicated nightmare. 2) Learning the theory: Oh, it's just the chain rule! That sounds simple! 3) Implementing in practice: I HATE INDICES AND I HATE SADDLEPOINTS.)

Anyway, what with all the hype and/or fear around neural networks — some justified, some not — I thought I'd actually learn, very concretely, what they are.  Then I can share those lessons with you, too!

Future Plans: I may release this first as a standalone toy, no "explanation" attached, then make it into an "explorable explanation".

Alternatively, I'd like to try making "Educational VTuber" videos: I play with a simulation I've made, while explaining ideas, all while I'm represented on-screen like a Twitch streamer's cartoon avatar. Then, I publish that video as the "product".

It'd be a more novel format for educational stuff, video does "better numbers" than text/websites, and it might be more fun / sustainably-motivating for me to make!

. . .

🤖 Update #2: AI Safety for Fleshy Humans: Part One is reeeeeeeeeeally close to launching

Part One: The Past, Present & Possible Futures (15 thousand[!] words total) has been fully written, illustrated, and 90% polish-draft edited.

A few random illustrations from the Intro:

(There's also ~70 spaced repetition flashcards, so you can remember these facts long-term! The cards are embedded in the text with Orbit, but there'll also be a downloadable Anki deck.)

What's Left To Do: Finish editing to send it to a dozen reviewers (Laypeople & Domain experts); Incorporate reviewer feedback; Animate & draw a bit more art; Finish setting up the website.

Future Plans: As mentioned a few updates ago, Part Two is already ~70% done, because I realized very late, that the series worked way better Part One became Part Two and Part Two became Part One and... anyway, that caused a delay, but the next instalment should not take a frickin' year this time.

If all goes ideally, Part One ships late April, Part Two ships late May, Part Three (final part) ships in late June or early July.

. . .

 🧓 "Update" #3: My first Explorable Explanation is now 10 years old???

[RAMBLE INCOMING. There's no structure or polish, sorry.]

. . .

On February 2014, my videogame's crowdfunding campaign was failing, so I needed a last-week promotion.  I was torn between either A) making a My Little Pony parody-mod, or B) making an interactive explanation of how I coded the lighting effect in my game.

Alas, I didn't have enough time to do the more fun Option A, so I settled for Option B.

That was my first ever explorable explanation, Sight & Light (Feb 2014). It hit the frontpage of Hacker News, got me in contact with Bret Victor, who then connected me with Vi Hart, who I collaborated with on Parable of the Polygons (Dec 2014), which was a huge hit, and

-- several dominos later --

I indirectly affected a few major Western news site? Washington Post's MOST-viewed article was their interactive(ish) thing on Covid/epidemiology models; its creator, Harry Stevens, told me he's been a fan of mine for a while.

Aaaaand that's ten years, folks.

. . .

On one hand, wow? And thank you folks – (and specific thanks to Bret Victor, Vi Hart, also hi Alan Kay we met only a few times but you were very encouraging) – for supporting me in this strange experiment for the last 10 years. <3

On the other hand, ugh, I feel like we could've done so much more? I'll admit, I kinda dropped the ball on Explorables, due to:

• Some mental health stuff, partly exacerbated by the pandemic & gender-dysphoria-related chronic throat pain.

• Some interpersonal bullshit in my Explorables-related community group, that led me to be sad and slightly angry and detached.

• Also, I got a bit blackpilled on the value of "increase the public's scientific literacy". First, it's not clear that changing one's deepest beliefs even changes one's actions much. (a study, replication 1, replication 2) Second, increased scientific literacy leads to polarization on hot topics — to believing more strongly in whatever your political/religious identity already says to believe — not to understanding & agreement. (paper 1, paper 2)

. . .

"Explorable Explanations" was first coined by Bret Victor, who recently updated his original Explorable Explanations webpage with this note:

1) Aw thanks Bret <3 and 2) Yeah, fair, "Explorables" drifted a lot, for... better and worse and neutral? (Some of my interactives do give the reader a full sandbox, to edit the models & challenge my conclusions. Some of my interactives, uh, don't.)

"Explorables" is/was a mix of two dreams:

• A tool for thinking: A computational medium to critically think in, for generating new ideas, not "just" transmitting ideas. (Bret's original direction; notebooks like Observable & Jupyter are in this vibe.)

• A tool for teaching: Explorations in how to best use digital/interactivity for pedagogy & sharing ideas/research. For example: Simulations to let you play with scientific/computational models (most of what's on the Explorables site, Distill.pub); Embedded spaced repetition flashcards (e.g. Quantum Country); Sandboxes with live feedback to help you actually train & practice skills (e.g. the now-defunct Try Ruby by the now-vanished _why). And so on.

Of course, thinking & teaching aren't enemies, or anything. But I think the lack of precision on Explorables' purpose led to...

...actually I'm not sure what it did or didn't lead to. There's no way to see the counterfactual, but I can't shake off the feeling I missed some kind of opportunity.  But I'm not even sure if that opportunity, whatever it is, is right for me specifically, anymore.

It's been ten years. Someone else can take this challenge?  I don't like being tied to one thing.  I want to feel playful & experimental in my creation again.

There's so much I want to do.

🐣,
~ Nicky Case