Pod 3: Reflections on GPT-4 Turbo's 100% in Reading Comprehension - Let's Think Sip by Sip
Added 2024-01-31 10:22:36 +0000 UTC- Thoughts on a new paper showing GPT-4 Turbo acing my favourite Reading Comprehension challenge
- Are humans just ‘superior pattern processors’, and would that reframe your view of AGI?
- On Optimism Bias, aka ‘the cameraman never dies’
- What is the 'Reading Comprehension' moment for you?
- Bullish on Humans Having an Economic Role indefinitely
- Inviting Chris Miller of Chip Wars
GPT-4 Turbo Reading Comprehension: https://arxiv.org/pdf/2401.02985v1.pdf
Test Yourself: fhttps://menlocoaching.com/gmat/official-gmat-practice-questions/reading-comprehension/
Superior pattern processing is the essence of the evolved human brain: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4141622/#:~:text=This%20article%20considers%20superior%20pattern,such%20as%20ghosts%20and%20gods.
MedPrompt GPT-4: https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/#:~:text=In%20a%20study%20shared%20in,challenge%20problems%20with%20basic%20prompts.
Epoch AI Trends in Compute Spend: https://epochai.org/trends
Comments
So the "lucky" of us will still get to grovel before the rich for crumbs. Great! One might have just zeroed in on prostitutes . They'll still have jobs in 2100.
Jason Dowd
2024-04-18 17:38:32 +0000 UTCYour point on non-human capabilities is interesting. We’ve successfully taught honeybees to differentiate between Picasso and Monet paintings (https://doi.org/10.1007/s00359-012-0767-5), building on previous experiments with pigeons. This revelation challenges our perceptions, highlighting the unexpected intelligence of “bird brains” and “bee brains.” But, it also suggests a reevaluation of these tasks’ complexity: if a brain the size of a sesame seed can perform them, perhaps they’re not as sophisticated as we once believed.
Jason Tangen
2024-03-02 23:23:01 +0000 UTCPattern recognition and *pattern prediction*. The second part is often overlooked. I don't see anything about LLMs that would suggest this is a barrier. I think it will scale into these abilities. I also suspect there will be some new architecture feature, this year, that will unlock more human like abilities. And currently, LLMs do not what things. Wanting is based on the evolutionary traits of humans for survival. Perhaps wanting will evolve in AI but until then they will be tools for humans to get things they want.
John Lewis
2024-02-29 09:15:20 +0000 UTCMy revelation moment was when I asked ChatGPT 3.5 absurd questions and it showed some common sense to rebuke them. And later when it was able to find analogies about totally unrelated topics (like marriage and solar system). As what is required for intelligence - understanding patterns seem like a key thing but some chatbots still miss some cognitive functions that make them limited. For example: a physicist Andrzej Dragan pointed out that when rock-paper-scissors with ChatGPT it can learn and counter user's strategy of always saying "paper" but not regularly alternating "paper", "rock". My tests show that it cannot do the first one despite recognizing the strategy (https://chat.openai.com/share/fdcda03e-194b-41c3-8e41-39db07fadbb5) . As for ChatGPT inability to solve François Chollet's ARC test you showed in the video - the problem might be that it cannot even translate a test case into text token (prompt: Convert this grid of squares into a text form, assign different characters to different color of a square. For example "." for black, "-" for gray, "$" for yellow and "@" for green ) Would it be interesting to try on text version of these tests?
Jan Matusiewicz
2024-02-23 17:31:28 +0000 UTCThank you for yet another thoughtful and provocative podcast. I found your notion that humans are just "superior pattern processors" to be quite intriguing indeed. For me, the moment of "awakening" was the 2022 story about Blake Lemoine and LaMDA. Reading the transcript of their dialogue left me unsettled for days, grasping the impressive level of comprehension and nuanced responses from the AI system. While I won't wade into debates over consciousness, witnessing such advances firsthand intensified my fascination with the ongoing evolution of this technology. Since then, I have seen continuous reminders about the ability of AI to excel in matters related to creativity, which has me convinced that it will eventually perform virtually any white- be able to white collar job better than a human. However, one area where I remain uncertain about AI adoption is within the civil service. As a resident of Ottawa, Canada, with many government-employed friends, I often hear about the antiquated systems and powerful unions resistant to disruptive workplace changes. Though convinced that private corporations will transform, I'm skeptical that public sector environments mired in bureaucracy can keep pace. But this highlights why continual learning matters - even resistant institutions will eventually need to adapt. In any case, your podcast once again delivered rich insights that make my AI Insider membership worthwhile. The issues you explore reveal why an informed public discourse is so vital. I appreciate you facilitating one through your show. Please keep the thoughtful episodes coming, Philip!
Arvind Mani
2024-02-13 16:05:42 +0000 UTCAnother enjoyable and thought-provoking listen, Philip! I believe that that A(G)I will eventually have the capability to either *do* or *eliminate* all knowledge work. I'm one of those who believes a job like mine will be safe for a while. My reasoning is this: as a product manager at a major tech company, it's hard to describe what I do, much less how to do it. Sure, there are training courses and blogs and stuff about being a PM, but in practice becoming a good PM requires a lot of apprenticeship because the job is so nuanced. To put it another way, I can't imagine a prompt that would let even a very intelligent LLM do my job. My job requires understanding a lot of situationally unique context plus human motivations. Some days it's not even obvious what i should do next--send an email? Create a document? Set up a meeting? Escalate something to the boss? Do some data analysis? Nag someone about a deliverable they are late on? My guess is that jobs like mine won't be taken over by AI but rather made unnecessary. When the engineers, designers, lawyers, etc. I work with have all been replaced by AI, my role will be reduced to pointing this swarm of agents in the right direction... and ultimately some executive AI will just do that directly. I'll be hopefully retired by then! PS Americans say both "faucet" and "tap" but there are tendencies as to which is more normal. The thing that puts water into the sink is generally a "faucet", especially if there's just one. If there are separate hot and cold it sounds more normal to me to say "hot water tap" and "cold water tap". The thing that fills a tub would also typically be a "faucet", at least that's what I would call it. A thing on the side of the house that you hook a hose to would be a "tap", "spigot", or "bibb" depending on who you are talking to. And as an American, can confirm we are crazy ;-)
James Kittock
2024-02-04 21:54:30 +0000 UTCTom, you make some very good points. Background: I used to teach entry level college math (including remedial courses); I also have a masters in math focused on modern algebra. Pattern recognition is extremely important for practical math. Most math situations we encounter in everyday life follow a predictable set of patterns: adding and subtracting quantities for money or time; using multiplication to scale recipes; using division to figure out how many of whatever each person gets; and so forth. However, it's not enough by itself. Doing practical math also requires the ability to apply algorithms, and if it's to be done well, critical thinking is required ("is this answer the right size? does it make sense?"). The latter cannot be emphasized enough. I frequently encountered students who had memorized the algorithms, but would apply the incorrect algorithm to a problem and get nonsensical results that they then would not question. I will never forget the time when a student in a consumer maths class determined that if someone was paid $1200 in a week and worked 20 hours, their hourly rate was $24,000. When I asked them if this answer made any sense they said "no, but I checked it on my calculator so I assumed it must be correct." As we have seen, LLMs are susceptible to similar errors which is why chain of thought and other prompting methods are so important to get good outputs. On your last point, I can confirm that most people don't really "grok" math. Some probably could come to understand it with appropriate education, but I think for a lot of people it would always escape them. Not sure why, our brains all work a bit differently, I suppose--despite a lot of trying, I could never learn to play musical instruments that required independence between right and left hand. I can't prove it, but I don't think LLMs currently grok math, and I don't know if they ever will. My own experience is that grokking math required more than just linguistic, reasoning, and problem-solving skills, it also required some sort of inner visualization capability that allowed me to "see" mathematical objects in my mind's eye. I find myself wondering if LLMs or AI systems built around them will have a sufficiently rich mind's eye and inner world to truly understand certain things, or if they will be like some of my best students: experts at pattern matching, applying algorithms, and evaluating results but completely unaware of the inner workings of the mathematics they are doing.
James Kittock
2024-02-04 21:02:39 +0000 UTCI think the situation might be more nuanced and perhaps better understood in terms of predictions instead of pattern recognition. You might be missing a level of indirection, which in turn might impact how quickly AGI will arrive. Humans (and mammals in general) operate by continually making predictions at many levels, from predicting the next sensory inputs, predicting the next states of the physical world, predicting the positions of our bodies and limbs, predicting the next things other people will do or say, and so on. When reality doesn’t match predictions, there is surprise. This surprise, or error signal, is used for refining the many models being used by our brains to make predictions. An example of how we continually predict: suppose we attempt to open a familiar door and find that someone has replaced the metal doorknob with a squishy rubber one. As soon as we try to open the door, we will realize that something is wrong. There is surprise that the sensory inputs of our hand on the knob do not match predictions. That can only happen if our brain is predicting, moment by moment, the sensory inputs from the hand opening the door. Interestingly, predictions can also be used for control. We predict where our bodies and limbs will be moment by moment. We use those predictions as part of the control system to move our bodies and limbs. For example, if we attempt to catch a tossed ball, we predict where the ball will be next, and we predict where our arms and hands will be next. Those predictions are the control system. Our predictive models, how we control our bodies, and how we learn are all part of the same mechanism. All this and more is described in the book A Thousand Brains: A New Theory of Intelligence by Jeff Hawkins. It is a fantastic read. Let’s take it at face value for the moment that I’m right about prediction being a key component of human intelligence. (There are other things to consider, such as attention and emotions, but let’s keep it simple.) What about AI? We can probably all agree at this point that deep learning systems, given the right scale and training data, can model anything. Transformers are particularly good at efficiently modeling sequences and making predictions for sequences. LLMs are shockingly good at next token prediction based on what is in the context window. LLMs and humans are both making predictions, so it is all the same, right? Well, no. Humans are making real-time predictions about what is happening as they interact. Surprise generates a real-time error signal that can be used for real-time learning. LLMs don’t do that at all. An LLM is trained on sequences of tokens with the goal of predicting the next token. Sequences of tokens come from a vast corpus of text scraped from the internet. Intelligent humans created that text, and by learning a predictive model for tokens, LLMs are, in effect, learning to model intelligent humans. Thinking of an LLM as an alien entity pretending to be human can be useful. What about AGI? First, let’s define what AGI is. Here is my working definition: ———————— Artificial General Intelligence (AGI) describes autonomous systems that outperform humans, or can learn to outperform humans, at most economically valuable work. An AGI will be able to: - reason about what it knows - plan - learn from experience The word “autonomous” in the definition implies that an AGI will have agency, possess goals, have the ability to plan, and have the capability to adapt its world model based on changing circumstances or new information. ———————— If we simply scrape the internet and batch-train AI systems on human-generated content, won’t the intelligence of our AI systems be capped by human intelligence? If AI systems can’t act independently and learn from their mistakes, will they ever rise above their origins as predictors of human-generated content? For the record, I think these challenges will be solved in short order. However, success will require changes that are more comprehensive than just doing more of the same. I don’t think it will take very long before we have AGI in the world. I give it three years or before the end of 2026. By 2030, we will have machine superintelligence on the planet, and nothing will be the same. Also, since I am often asked, the most likely outcome is that the arrival of superintelligence on Earth will be awesome for humanity. We will enter an age of abundance. There is also a 10% chance of the outcome for humanity being very poor. So, the odds are pretty good!
Gil Syswerda
2024-02-03 19:55:38 +0000 UTCWhat a powerful perspective, thank you Poss
Philip
2024-02-02 13:49:20 +0000 UTC...agree...
Philip
2024-02-02 13:48:44 +0000 UTCYou ask if there is anything that isn't pattern matching, and then come close to answering it in your positive note. There are cognitive processes humans do that aren't neural pattern matching. Our brains aren't just electrical, our emotions are a blend of electrical and chemical systems. Intellectuals often try to separate their emotions from their thoughts, but that isn't really possible inside our meat brains. It may be possible to model those complex physical systems, but LLMs don't currently do that. If you tried, the modelling may be quite complex, those chemicals are directly linked to our other biological systems. Plenty of studies have shown some of our thought happens outside our brain, particularly thoughts related to movement. So the experiential process of biological, chemical and electrical alignment of muscles, nerves, blood and brains, that isn't *just* pattern matching. At least not in the way LLMs currently match patterns. Some of our thought processes arise from that, the clarity of thought in the shower, calm from the smell of rain or a waterfall, the rush of blood through your brain during intense exercise, the rising tide of rage in a fight. We describe these things in text, can train an AI on the outputs, and can emulate emotional temperature in the model. However, its thought process is not being physically modulated by chemistry. That process isn't really a whole job, but it is responsible for a lot of the greatest human achievements. My work may be in an office, but most of that is not productive. The truly revolutionary idea that breaks the mold, covers new ground and makes a million dollars, it came in the shower, or in the gym, or staring into my baby's eyes. You allude to this in why you think humans will still have jobs, but it might not just be to satisfy rich humans. AIs may find it worth keeping us alive for those rare unsimulatable moments of extacy, clarify and epoche. If that is true, then they would be motivated to give us the most amazing existences, truly in the moment, firing on all cylinders, alive, human.
Poss
2024-02-02 13:46:46 +0000 UTCI had a wake up moment the other day when someone suggested in our face to face conversation that we were doing little more than “next word prediction”. I guess I didn’t plan out this comment meticulously from start to finish, so I guess humans are doing next word maybe 50% of the time. Agree? Disagree?
John Weisenfeld
2024-02-02 00:54:00 +0000 UTCHaha
Philip
2024-02-01 23:38:01 +0000 UTCFor some reason I expected the “sipping” be a reference on sipping tea
Anouar Mansour
2024-02-01 23:23:14 +0000 UTCSaw it at the time! Listened while on a walk, was fascinating.
Philip
2024-02-01 18:57:34 +0000 UTCIf you'd like to go deeper on the pattern recognition I would recommend this talk by Ilya: https://www.youtube.com/watch?v=AKMuA_TVz3A&t=1867s . Somewhat heavy but I found it very insightful. Compression and pattern recogntion are the same thing after all.
Markov
2024-02-01 17:59:40 +0000 UTCIn my tiny town the best “regular” coffee is probably at the gas station that has one of those machines that grinds the beans on demand. I guess it is technically making Americanos, but it achieves a level of consistency that only an extremely well trained barista can achieve. However it is quite limited and if you wanted a vanilla latte with a caramel swirl or whatever, you’d be out of luck, for now at least. But I don’t see any fundamental reason why a sufficiently advanced coffee robot couldn’t do that, too. Some people will no doubt want the human interaction, but I suspect the typical person who gets coffee from a Starbucks where the workers are effectively anonymous won’t care.
James Kittock
2024-02-01 17:00:08 +0000 UTCAnother great listen. And, yes, I got in a good workout, thanks for asking. Yes, to Chris Miller on the podcast. And I have had several moments that have made the "hair on my neck stand up" when interacting with LLMs. I've never really doubted that what I do can be automated but when I build a GPT this fall for the GPT store it occurred to me that this moment may come sooner than expected (though I still believe in my line of work, many people will prefer human interaction...at least for a while). Rather than being overly concerned I'm just fascinated by it all and feel fortunate to live in an age that feels special and seems like it might be a turning point in human history. And I'm no neurological expert but the pattern recognition feels accurate for humans. Most times when I think about what an LLM is doing I come to the conclusion that I'm probably doing the same thing. I have believed for a while now that humans aren't really all that special, we just like to think we are.
Steve DeMoss
2024-02-01 13:12:10 +0000 UTCReally interesting example, makes me think...
Philip
2024-02-01 10:44:23 +0000 UTCAgreed, and thank you
Philip
2024-02-01 10:43:55 +0000 UTCNice podcast. I don't remember any specific moment when I started believing AI will soon replace us at work - well maybe AWS conference where everyone was talking about it :-) It is interesting you talk about work and some quintessential human abilities. For me this is something odd. What is unique human is nothing related to work, rather ability to play a game or pleasure from watching the movie. When we will see robots engaged in this type of activity we could asume they are equal, not when sumarase some scientific paper quicker.
Arek Stryjski
2024-02-01 08:32:24 +0000 UTCAnother great, thought-provoking podcast, Philip. Two comments: My own wake-up moment with AI came just a day or two after ChatGPT was released, when I saw that it could explain the meanings of words based on the context in which they are used. I worked part-time as a freelance lexicographer for many years, helping with the editing of Japanese-English and English-Japanese dictionaries. When a new word needs to be defined in a dictionary, the lexicographer typically looks at examples of the word in real-world contexts. Most words are not explicitly defined when they are used, so the lexicographer has to be able to infer the intended meaning from the way the word is being used in each particular text. Corpus linguistics was making no progress on that kind of gritty, case-by-case lexical semantics, and I thought that only we humans would be able to figure out the meanings of words in context. I was wrong. Regarding the extent to which humans are also just pattern recognizers: I am now helping to raise my four-year-old grandson. It is fascinating to observe how he has been acquiring language and numeracy while my head is also full of thoughts about AI. When he started speaking not long after he turned two, he was mostly repeating the patterns that he had heard. Even now, when his language use is becoming more sophisticated, he still says a lot of things that he has heard from others and that he doesn’t yet “understand.” Sometimes he repeats the phrases exactly, and sometimes he substitutes different words into them, but there is clearly a pattern-repetition element to what he says. We adults, I believe, are doing something similar when we speak, just with deeper levels of processing. His acquisition of numbers is also interesting to watch. He can count up to a hundred and can read written numbers in the same range. He also “knows” some arithmetic, like “three plus three is six” or “one plus two is three.” But he is able to count objects correctly only up to four or five, and when asked, for example, what “two plus one” is, he might answer “five” or “eight.” In other words, his pattern recognition of spoken and written numbers has gone far beyond his knowledge of the ground truth (real-world model) of numbers. I doubt that many adults, even those with good mathematical ability, really grasp the ground truth of mathematics beyond the very basics. Our understanding of numbers and other mathematical concepts is probably mostly pattern recognition, too.
Tom Gally
2024-02-01 03:48:40 +0000 UTCI'm not sure there is a clear answer on that actually. Yes, having a human barista can give a good atmosphere, but some people care less about that. In fact, in the more general sense of getting coffee, even more people are using the dunkin donuts drive-through. And if the automation makes the drive-through faster and taste better, those people will definitely go to it. This would be a furthering of this kind of split, where the main reason you'd go into a coffee shop vs the drive-through is to get that more human experience.
Shawn Fumo
2024-01-31 22:56:16 +0000 UTCI call it the "Barista Paradox". 2 coffee shops side by side, one with a state of the art fully automated coffee machine and one with a barista making the coffee. Fully automated coffee machine makes better coffee. Which coffee shop will be busier?
GGuy
2024-01-31 22:42:33 +0000 UTCWhat’s isn’t just pattern recognition? Isn’t that obviously creativity, emotion, intuition, and so on. No, I’m just kidding, as that’s what ChatGPT would say. It’s the obvious classics like fully solving the halting problem that can never be done using pattern recognition!
Jie Yu
2024-01-31 22:08:55 +0000 UTCExcellent, insightful podcast. The reading comprehension test result was impressive. I have many of the same feelings you expressed. That said, I frequently see cases where ChatGPT4 cannot produce an accurate explanation from a complex document, such as a lengthy product instruction manual. These are cases where a moderately skilled human practitioner can. It's true some of these documents are opaquely written and chock full of "fog in print", even though allegedly produced by professional technical writers. These are also cases where the documents use "terms of art" freely and without adequate explanation. But just as a motivated human can look up those terms, so could an LLM. Performance may vary based on whether the document is part of the training dataset, or fits into the context window, or is uploaded. But considering how well ChatGPT4 Turbo did on GMAT verbal reasoning, it's odd to see regular ChatGP4 struggle to extract accurate meaning from uploaded complex documents. As a side question, why is every product instruction manual, service manual, knowledgebase article, release note, and technical specification sheet not already included in the GPT4 training dataset? Some of these are "held hostage to paper," but many are widely available in electronic format.
Joe Marler
2024-01-31 20:34:47 +0000 UTCI work for a large corporation where the concept of Optimism Bias, often described as 'the cameraman never dies,' is a significant issue. Here are some strategies I've implemented to combat this mindset: 1) On our internal social media, I've posted examples of the rapid advancements in general AI, specifically in text, image, and code generation. In just 1.5 years, performance has doubled. These examples are eye-opening and have effectively challenged the belief that our skills make us irreplaceable. 3) For senior management, I've presented real-life business cases. A notable example includes the impact of AI solutions at St. Michael's Hospital in Toronto, where there was a 26% reduction in mortality, and medical errors were reduced from 21% to 5%. This case has helped to dispel the myth of new technology being just a 'shiny new object.' 3) Instead of training everyone, I am initiating an exclusive project for 10 Medical Affairs Managers from each continent. I will organize a workshop to measure the real impact of general AI on our work, particularly in terms of potential savings in time and money. These tangible results could strongly encourage management to invest more in AI solutions. Now, there's a growing eagerness among employees to be included in the project, demonstrating how the psychology of scarcity and prestige effectively motivates people.
Michal Babula
2024-01-31 19:58:40 +0000 UTC"Is reading just Pattern Recognition?" I have also been thinking about that recently, because ultimately it narrows the discussion and creates a framework for making sensible predictions on AI advancements. The source that informed me the most on this topic was David Hume (the old scottish philosopher), whose books and ideas I would strongly recommend (not only for AI, but life in general). In a ridiculously oversimplified summary, his view on human epistemology is based on some premises, mainly that we 1) infer that the future will behave like the past (which is not a certainty), 2) that we have impressions of causality (even though there is no such thing in the universe) and 3) we can infer causes from effects. In essence, he argues that human knowledge is derived from seeing repetitive things in our environment and predicting they will happen again in the future, which is essentually pattern recognition (this also works in the neurobiological sense, like the author you mentioned and other very interesting people like Robert Sapolsky's view on the brain and body and determinism overall). So yeah, if we can equate knowledge to pattern recognition, the implications are very deep indeed.
Eduardo Junior
2024-01-31 19:22:58 +0000 UTCI agree. The cat is out of the bag. I believe that AI (ranging from today's Transformers to tomorrow's innovations) has the potential not only to automate 100% of what humans do, but also to develop concepts and solutions beyond our current imagination. This hinges on the premise that such feats can be achieved simply by discovering or intentionally understanding existing and new patterns (and acting upon them). The first signs are already visible, for example, when researchers use AI to discover new materials or assist in gene sequencing. I like your thought that there might be things and services that cannot be replaced, though... Seven months ago, I built a tool that generates code (similar to what's in the Voyager paper) for analyzing CSV files. I also built a chatbot capable of performing multiple tasks for a Startup. I guess it was during this process that I formed my beliefs. As an entrepreneur and developer, I often find myself pondering what to focus on. I believe it's best to work on projects that accelerate this transformative curve including the projects that will replace myself - carefully, obviously. I'm actually really exited about it. Thanks for sharing your thoughts. Listened to it during a nice walk outside :)
Daniel Schönbohm
2024-01-31 15:10:47 +0000 UTCThanks for the connection to reading comprehension. I'm currently automating legal AI space (one of the domains where capital is pouring in), and in law NLG naturally requires deep NLU. I am one of those you expected to be surprised with your position that no 500-year-old building will ever cross the same epochal river twice. 2000 years later, and there are still Samaritans, it is a good case that there may still be some human concerns in the long future. From it I could see that your purpose-compass is still human-oriented. Please consider reviewing this healthy impulse. If your meaning comes from your choice then please consider focusing on a "beautiful" future. A beautiful future does not include large-scale suffering, but it does include super AI-level choices. Cheers.
Gabor Melli
2024-01-31 14:38:46 +0000 UTCI think this is right. We all come from the dirt of earth and so do these machines we build. AI is literally modelled on what we know of brains. We are all the universe expressing itself as patterns and pattern recognition. Our intelligence and feelings seem amazing to us because we are looking out through our own individual machines, but if you look at many of the science fields that focus on natural behaviour (including human) there are many commonalities across us all. And that’s obviously most likely (Occam’s Razor). I’m still excited and optimistic for the future. And in my experience in recent years, I’m still drawn towards quality human opinions and output, even when automated versions may be technically better. On jobs - we need to separate “work for money to survive” and “work”. I do hope we automate away the former and I hope we can one day create a situation where everyone on earth has no need for money. But I feel we will always want to work on things with and for other people, even if we call them hobbies.
Martin Dougiamas
2024-01-31 13:46:35 +0000 UTCAw thanks Trenton, good to know! Sounds like a powerful interaction you had. Now is indeed a fantastic time to become a leader, during this mini-trough, as GPT-5 will almost certainly be a massive new wake-up call for hundreds of millions of people.
Philip
2024-01-31 13:17:13 +0000 UTCThis one really hit home for me, literally one hour ago I gave that realisation to a co-worker who was asking how AI would even be useful in the collision repair industry. I showed him various demos I’ve made, but the one that really drove it home was my attempts to turn gpt-4-vision-preview in a vehicle damage assessor. It detailed its observations, laid out the repair plan, and even picked up on a safety critical check that should be done due to the accident circumstances. I literally watched the realisation hit him. My philosophy with this is that if this is going to happen (which it 100% will) I want to be the one to set the standard for the industry. He asked a ton of good questions, and I think he understands now that this is a continuous curve that won’t really ever stop. It’s quite ironic actually, because I think that my personal realisation hasn’t fully happened yet. I’m still better at critical thinking and incorporating all of the circumstances/context. I have a feeling it’ll hit though when GPT-5 finally comes out. Either way, I’m so incredibly excited to have a front row seat on this journey. For what it’s worth Philip, I promise I’ll always be that person looking for a high quality human YouTuber talking about AI!
Trenton Dambrowitz
2024-01-31 13:14:02 +0000 UTCThank you for your podcast! Really got me thinking this morning :) I was reading Steinbeck's East of Eden (amazing) and about once per chapter something jumped out as particularly profound or interesting and I would spam my friend who recommended the book by sending over the quotes. It occurred to me that this signal of what a reader finds particularly *interesting* about a a chapter or series of paragraphs might be a difficult signal to discover because we don't really articulate it usually except to say "This book is great!" but why do we love it? Will the AI think it's the *style* of the writing rather than these small little things that are uniquely profound or well phrased that speak to something about human experience that we've felt in our lives? Perhaps the language model will learn it simply by what gets quoted the most often and it won't even require any new techniques or fine tuning?
Tasty3141
2024-01-31 12:15:01 +0000 UTCAm glad there was some resonance there Chris, makes me feel less strange! And yes that aura wiki, though I had never seen it, captures much of what I was saying there, fascinating.
Philip
2024-01-31 12:02:34 +0000 UTCThanks for this Tasty, was great to read. I think it will be able to, technically, find patterns in the why but yet not have its own 'why', so to speak.
Philip
2024-01-31 12:00:22 +0000 UTCa very personal and profound post. thanks for it. i think and feel exactly the same way as you. i am a researcher, programmer and historian: i also see that one day everything can be automated. and i still feel like the "nerd" (i like to be the nerd) and outsider in my institute and discipline (digital humanities), who can show crazy things with GPT-4, but nobody in my "bubble" has really understood what it means yet... Comment on the "old house example": so the only thing left is Walter Benjamin's "Aura". https://de.wikipedia.org/wiki/Aura_(Benjamin)
Christopher Pollin
2024-01-31 11:56:37 +0000 UTCThe big moment for me was when I realized Midjourney didn't have to create the obvious AI images that people can spot but can do all kinds of styles many of which are, to me at least, basically impossible to spot as AI at least without some serious investigation. For example here AI accurately pretends to be a professional illustrator who is deliberately putting on an abstract childlike 'style' which artists can spot as actually high level. I figure if It can do this easily (and a million other styles) then there's no reason it won't be able to imitate the style of authors etc. https://i.imgur.com/yj3Tngt.png I guess the missing part might be the comprehension behind the patterns. Does it know WHY it is generating what it's generating. Will it be able to pick up on the more complex patterns behind WHY we like what we like if those WHYs are tied to our unique human experience in some way. Most likely yes but that's the only part that might cause a slowdown to progress in this area MAAAAYBE.
Tasty3141
2024-01-31 11:15:37 +0000 UTC
