Sebastien Bubeck Qs
Added 2023-12-15 17:24:22 +0000 UTCSpeaking to Sebastien Bubeck in an hour, for AI Insiders. Bubeck is Sr. Principal Research Manager at Microsoft, co-author of Sparks of AGI and Phi-2 (Textbooks are All You Need) and Yann LeCun debater: A New Kind of Intelligence. A bit late notice I know, timing was just confirmed, but wanted to give Insiders a chance to suggest questions. No guarantees, but will strongly consider those upvoted! Video derived from discussion up after at some point soon.
Comments
Thank you sir!
Prajay Tipre
2023-12-22 20:38:32 +0000 UTCThis should solve it! https://support.patreon.com/hc/en-us/articles/212052266-Getting-Discord-access
Philip
2023-12-22 19:19:28 +0000 UTCUnrelated but can anyone drop the discord link? Was having a hard time finding it.
Prajay Tipre
2023-12-22 18:25:16 +0000 UTCNo, but the Boston Dynamics AI Institute will be working on exactly this problem!
Jie Yu
2023-12-17 00:20:38 +0000 UTCI asked something similar, video coming before Christmas
Philip
2023-12-16 10:01:35 +0000 UTCThis is a question form chaGPT to Sebastian: As someone deeply involved in AI research, what are the next big milestones or challenges you anticipate in the field? What areas of AI research excite you the most?
Michal Babula
2023-12-16 08:11:38 +0000 UTCAsked!
Philip
2023-12-15 23:21:22 +0000 UTCDidn't get to this but next time!
Philip
2023-12-15 23:21:03 +0000 UTCBriefly touched on this but his time was short alas
Philip
2023-12-15 23:20:53 +0000 UTCAsked!
Philip
2023-12-15 23:20:40 +0000 UTCWe touched on reasoning quite deeply!
Philip
2023-12-15 23:20:35 +0000 UTCAsked and answered. But you shall have to wait for the video! Haha
Philip
2023-12-15 21:57:01 +0000 UTCAsked!
Philip
2023-12-15 21:56:20 +0000 UTCI asked something similar. Instruct actually hurts perf. apparently!
Philip
2023-12-15 21:56:13 +0000 UTCWould love to know what he thinks a world post-agi would look like and where society/technology will develop towards or what his person plans will be once it is achived.
Nazzaroth
2023-12-15 20:10:19 +0000 UTCWill true multimodality only be achieved through robots that can actually, see, touch and interact with -- and therefore understand the real world?
Jeff Thom
2023-12-15 18:54:07 +0000 UTCDo we need new benchmarks for measuring capability for reason? What would those look like and how are they different than what we already have?
Jeff Picel
2023-12-15 18:30:06 +0000 UTCVery basic one, but what does he think that the path to AGI looks like?
solarapparition
2023-12-15 18:08:37 +0000 UTCWhat does he think about the new Mamba architecture and if it has the potential of replacing transformers in the near future
Jörg Eitner
2023-12-15 17:42:38 +0000 UTCOn the tune of Yann LeCun I would love to hear a take on his JEPA models (and his opinions around it: https://twitter.com/ylecun/status/1735648731855310896). Maybe also somewhat related, using diffusion models instead of transformers to generate text (eg CodeFusion from Microsoft).
Blixt
2023-12-15 17:41:56 +0000 UTCAlso…selfishly so forgive me…, can the standard GPT architecture be used to predict the next image (rather than word)? Is there an “image embeddings” database perhaps?
Shaun McDonogh
2023-12-15 17:39:56 +0000 UTCUsing high-quality training data, what is the smallest size of model does he think could build to be on the level of a GPT4V or Gemini Pro
Rakesh Murria
2023-12-15 17:35:04 +0000 UTCThe blog post says "Phi-2 is a base model that has NOT undergone alignment through reinforcement learning from human feedback (RLHF), nor has it been instruct fine tuned." How is the model getting such good results without any RLHF??
Daniele Moro
2023-12-15 17:35:01 +0000 UTCI'd like to know what is the most top of mind for him regarding AI in the next year. What he looks forward to and what he sees being accomplished in the AI space.
Anouar Mansour
2023-12-15 17:33:46 +0000 UTCLifelong lurker and newbie to AI from totally separate field, please forgive any misunderstanding in my questions. Current benchmarks and decontamination methods are flawed. Synthetic data seems particularly at risk for beating traditional decontamination methods. Are the phi models susceptible to this, and any thoughts on the next likely outflow of synthetic data for training generated by already contaminated LLMs Nice paper that looked at better methods for evaluating this “Llm decontaminator” https://arxiv.org/abs/2311.04850
G Subs
2023-12-15 17:33:29 +0000 UTCDoes he foresee some kind of inflection point where the best models are good enough to synthesize training data for other models and is this a viable path forward towards AGI? (I still need to understand more of exactly how the synthetic data was generated, and regardless I'm guessing you can phrase this question a lot better)
Eli T. Drumm
2023-12-15 17:30:50 +0000 UTCAmazing! Excited to find out 1) what are the likely biggest contributing factors to a model discovering “new knowledge” (also the recent deep mind post was interesting) and 2) do they have any thoughts as to what Q* was about. Not sure i get 2 questions but worth a shot lol.
Shaun McDonogh
2023-12-15 17:30:07 +0000 UTC
