XaiJu
Andy Matuschak
Andy Matuschak

patreon


(Recording) Discussion: AI-generated ad-hoc UIs and malleable software; Saturday, November 25th @ 9AM PST

Thank you all for a very interesting discussion! See original post for premise.

Please don't share this recording publicly.

And here's the chat transcript with some links which were mentioned:

00:20:19.962,00:20:22.962

Taylor Rogalski: quip had a nice chat-as-changelog-alongside-artifact pattern https://www.smartsheet.com/sites/default/files/2020-02/5cba0c3ca6e1a17be8045cc0.jpg


00:27:12.684,00:27:15.684

Andy Matuschak: This is my #1 favorite UI + AI paper! https://www.pnas.org/doi/10.1073/pnas.1807184115


00:27:18.199,00:27:21.199

Andy Matuschak: (the Jeff Heer one)


00:45:41.614,00:45:44.614

Maggie Appleton: I forget if mentioned in the call notes, but Adpet's latest release very relevant here: https://www.adept.ai/blog/experiments


00:46:44.369,00:46:47.369

Maggie Appleton: At the moment it's essentially keyboard maestro in the browser, but the unreleased stuff they're working on much more interesting and includes lots of on the fly UIs: https://www.youtube.com/watch?v=PAy_GHUAICw


00:46:55.387,00:46:58.387

Andy Matuschak: ^ this is new to me—thank you, will watch!


00:50:26.475,00:50:29.475

Andy Matuschak: https://dynamicland.org/links/2023-04-27


00:55:15.232,00:55:18.232

Tim Riherd: https://www.sidefx.com/tutorials/foundations-overview/


00:55:17.112,00:55:20.112

Taylor Rogalski: https://en.wikipedia.org/wiki/5D_Chess_with_Multiverse_Time_Travel


00:57:02.054,00:57:05.054

Maggie Appleton: @Geoffrey have you written anywhere about those hypothetical primitives of a world where we're generating UIs? Want to think about that more


00:57:30.416,00:57:33.416

Geoffrey Litt: hmm i dont think so;  maybe soon :)


00:57:45.575,00:57:48.575

Maggie Appleton: 👍 please do


00:57:51.926,00:57:54.926

Tim Riherd: +1


00:58:20.816,00:58:23.816

David D: https://github.com/inkandswitch/cambria-project


01:00:59.230,01:01:02.230

Maggie Appleton: Walking treadmill is a sad solution to that dream ;)


01:01:29.100,01:01:32.100

Tim Urian: for real!


01:01:53.136,01:01:56.136

Geoffrey Litt: Currently 1% of people make software; getting to 10% would be huge progress. 100% not necessary!


01:01:58.290,01:02:01.290

Andy Matuschak: ++


01:03:11.572,01:03:14.572

Gabriel Nunes: The problem of adapting data between interfaces for malleable software was the subject of my master’s thesis (not AI though): https://www.kronopath.com/blog/atypical-a-differently-optimized-type-system/


01:03:17.299,01:03:20.299

Maggie Appleton: ++ thanks for hosting


01:03:18.279,01:03:21.279

Daniel Dosen: Thanks!


The org-mode-for-sermons video Taylor was referring to: https://www.youtube.com/watch?v=Q8AqHdZTgNI

(Recording) Discussion: AI-generated ad-hoc UIs and malleable software; Saturday, November 25th @ 9AM PST (Recording) Discussion: AI-generated ad-hoc UIs and malleable software; Saturday, November 25th @ 9AM PST

Comments

I found this 'how to generate UIs with LLMs' video fascinating! Thought I'd share here: https://www.youtube.com/watch?v=8w0hUcQSDy8

Parker

That does seem right! It reminds me the way that Bret's UI in "Drawing Dynamic Visualizations" needs to surface and disambiguate multiple possible interpretations: https://vimeo.com/66085662

Andy Matuschak

I missed this, but enjoyed the video! One element that I'd like to discuss more is the interplay between automation and inspectability. Looking at Adept Workflows demo videos, for example, I kept thinking about what the user might do when these go wrong. Of course, there might be a chat interface where you could say, for example, "No, no, Workflows -- when you are contesting an insurance claim rather than filing one, go to this URL instead!" But this brings you to the problem of poor addressability in verbal interfaces that several people named in this session. It seems to me like there needs to be a visual representation of the major steps and logical decision-making elements, with some ability to replay decisions in a sandbox environment, and intervene at the point where the agent should have behaved differently.

Benjamin Wheeler


More Creators