August 15th, 2023
Written by: Jake Parker
In the red corner, weighing in at approximately 3 pounds, with 86 billion brain cells1 and 100+ trillion connections2 between them, yet only using as much energy as a light bulb3,4, the reigning sultan of smart, the human brain! In the blue corner, occupying thousands of servers across many large server buildings and consuming as much energy as hundreds of lightbulbs5 just to answer single questions, the computer-based challenger, ChatGPT!
On November 30, 2022, OpenAI quietly released a piece of software that can mimic human conversation to the public. This was the first version of the now well-known chatbot, ChatGPT. While far from the first chatbot to exist, ChatGPT is far more capable than its predecessors. Upon its release, social media posts spread rapidly, showing that ChatGPT could perform tasks like drafting entire essays, writing computer code, and explaining the basics of quantum mechanics. Naturally, people began to wonder if we had finally created something that is smarter than we are. However, as the fight night style opening suggests, ChatGPT differs substantially from the brain. In fact, comparing the two is like comparing airplanes to birds. They might do the same thing (fly), but they accomplish this in very different ways.
Despite this fact, we can learn a lot about both ChatGPT and the brain by comparing the two. First, we will take a look at what ChatGPT is, how it works, and how its inner workings compare to that of the brain. Then, we will see if ChatGPT can go the distance when pitted against the brain in three classic hallmarks of intelligence – learning, memory, and logical reasoning.
What is ChatGPT and how does it work?
ChatGPT is a type of large-language model6, which is a piece of computer software that is trained to converse with humans by responding to typed messages. To accomplish this, large-language models are made of thousands (or millions) of computer-simulated neurons that mimic the way that real neurons work through basic mathematical operations (Figure 1). This technique is called a neural network and allows computers to do complicated tasks such as identifying objects in an image, translating one language into another, and detecting spam emails.
In comparison, the human brain contains a whopping 86 billion neurons1 that form 100+ trillion connections2 with each other. Furthermore, real neurons are able to perform more complex operations than simulated neurons. In principle, these facts mean that the brain is capable of performing more difficult tasks than neural networks like ChatGPT. This is similar to how a boxer that knows more techniques has the potential to become more successful than one that knows fewer. That being said, let’s see if ChatGPT can overcome these disadvantages in its much awaited matchup with the brain!
Learning – how does ChatGPT learn how to do what it does?
ChatGPT learns primarily by being trained to predict what word comes next in a sequence of words. This is done by giving it incomplete passages from various texts and asking it to predict the next word. Remarkably, it is simply by learning to make these predictions that ChatGPT becomes able to make conversation and write like people do. While we don’t know exactly why this works, we could imagine this is like a boxer learning to fight by learning what exact action to take in every single situation they could find themselves in. This differs from the brain, which appears to learn language through less structured and more natural episodes7. In our boxing analogy, this is like a boxer learning to fight by doing entire practice fights. Regardless of the learning strategy used, both the brain and ChatGPT realize this learning by making small adjustments to the way their neurons connect to each other8.
In order for ChatGPT to get on the same footing as humans, however, it had to learn from vast numbers of texts9 drawn from the internet – think almost all of Wikipedia, large parts of Reddit, entire libraries of online books, huge numbers of news articles, and many more sources. This amounts to orders of magnitude more reading than a person could do in a single lifetime, let alone several! In contrast, humans learn how to speak and write with far less exposure to spoken and written language. While ChatGPT had to spend years training under all of the best boxing coaches, the brain was ready to go after a few days at the local gym. Humans can learn most things with relatively little practice compared to neural networks10. Given how much more efficiently humans learn, the brain decisively takes round one.
Memory – how much does ChatGPT know?
ChatGPT can accurately answer questions about an immense number of topics with an impressive level of detail. In neuroscience, the ability to recall information about topics and events days, months, or years after they occur is called long term memory11. Evaluating ChatGPT’s long term memory in a scientific way is impossible because we don’t know exactly what texts it learned from, so I conducted my own (highly informal) experiment. I asked ChatGPT to recite the first sentence of a few books that vary in terms of popularity (J.K. Rowling’s Harry Potter and the Chamber of Secrets, Kazuo Ishiguro’s Never Let Me Go, Robin Shulman’s Eat the City). It perfectly recited the exact first sentences of Harry Potter and the Chamber of Secrets and Never Let Me Go, and while it wasn’t able to do so for Eat the City, it perfectly recalled the full title (Eat the City: A Tale of the Fishers, Foragers, Butchers, Farmers, Poultry Minders, Sugar Refiners, Cane Cutters, Beekeepers, Winemakers, and Brewers Who Built New York) even though I only asked it about “Eat the City” (Figure 2). If we imagine remembering this level of detail about thousands of books (or topics) that have existed throughout history, ChatGPT appears to have superhuman long term memory. In a shocking upset, round two goes to ChatGPT.
Logical Reasoning – how well can ChatGPT think?
Perhaps the biggest difference between the brain and ChatGPT comes in the area of reasoning and problem solving. On the surface, ChatGPT appears to do well in these areas. It can solve computer coding problems12, pass medical school13 and law school exams14 , and perform well on reasoning problems originally created by researchers to evaluate human thinking15 . However, we encounter a major caveat to this when we dig a little deeper. Namely, people have discovered that ChatGPT performs drastically worse on problems that were not published online until after September 202112,15. This date is significant because ChatGPT only learned from texts that were published online on or before September 2021. Given that these texts were drawn from substantial portions of the internet, it is likely that ChatGPT only performs well on problems from before September 2021 because it encountered them while training. In other words, ChatGPT appears to rely primarily on its extraordinary memory to “solve” these kinds of problems. Returning to the idea that ChatGPT is like a boxer who learned to fight by learning what action to take in every situation they could think of, this makes sense. In a real fight, this boxer would do fine in all of the scenarios they practiced. However, this boxer would immediately struggle whenever they find themselves in a situation they didn’t practice or against an opponent they haven’t seen before.
In comparison, the brain is like a boxer who has developed a strong intuition for how to fight after a handful of bouts. They perform well not only in scenarios they’ve seen before, but also in new situations and against new opponents. The ability to apply previous learning to entirely new situations is called generalization16,17 and is thought to be an essential component of intelligence. Humans are especially adept at generalizing while most neural networks struggle with even small deviations from what they’ve learned18. With this convincing victory in round three, the brain wins the match by unanimous decision!
As it should be apparent now, ChatGPT is a much different machine than the brain. Like how an airplane requires powerful jet engines and large amounts of fuel to do what birds do naturally, ChatGPT relies on incomprehensible quantities of information and exhaustive learning to produce language like a human. Even then, ChatGPT struggles with logical reasoning, appearing to lean heavily on its impressive memory of the texts it learned from. Furthermore, it has a well-known tendency to produce nonsensical or untruthful information and serve harmful content to users14. Given the substantial limitations of ChatGPT , it is safe to say that the brain will get to keep its crown. For now.
References
1. Azevedo, F. A. C. et al. Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain. J. Comp. Neurol. 513, 532–541 (2009).
2. Pakkenberg, B. et al. Aging and the human neocortex. Exp. Gerontol. 38, 95–99 (2003).
3. Balasubramanian, V. Brain power. Proc. Natl. Acad. Sci. 118, e2107022118 (2021).
4. Power of a Human Brain – The Physics Factbook. https://hypertextbook.com/facts/2001/JacquelineLing.shtml.
5. OpenAI’s ChatGPT Reportedly Costs $100,000 a Day to Run – CIOCoverage- Driven for Technology Leaders. https://www.ciocoverage.com/openais-chatgpt-reportedly-costs-100000-a-day-to-run/.
6. Large language model. Wikipedia (2023).
7. Saffran, J. R., Senghas, A. & Trueswell, J. C. The acquisition of language by children. Proc. Natl. Acad. Sci. 98, 12874–12875 (2001).
8. Citri, A. & Malenka, R. C. Synaptic Plasticity: Multiple Forms, Functions, and Mechanisms. Neuropsychopharmacology 33, 18–41 (2008).
9. Gertner, J. Wikipedia’s Moment of Truth. The New York Times (2023).
10. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
11. Squire, L. R. Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory. J. Cogn. Neurosci. 4, 232–243 (1992).
12. ChatGPT-4 Solves 85% of Leetcode Easy Problems | HackerNoon. https://hackernoon.com/chatgpt-4-solves-85percent-of-leetcode-easy-problems.
13. Kung, T. H. et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit. Health 2, e0000198 (2023).
14. OpenAI. GPT-4 Technical Report. Preprint at https://doi.org/10.48550/arXiv.2303.08774 (2023).
15. Liu, H. et al. Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4. Preprint at https://doi.org/10.48550/arXiv.2304.03439 (2023).
16. Generalization (learning). Wikipedia (2023).
17. Gluck, M. A., Mercado, E. & Myers, C. E. Learning and Memory: From Brain to Behavior. (Worth Publishers, 2013).
18. Szegedy, C. et al. Intriguing properties of neural networks. Preprint at https://doi.org/10.48550/arXiv.1312.6199 (2014).
Cover Photo made by Jake Parker in GNU Image Manipulation Program using the following assets:
Image by Peggy und Marco Lachmann-Anke from PixabayImage by OpenClipart-Vectors from Pixabay
OpenAI, Public domain, via Wikimedia Commons
Figure 1 made by Jake Parker on Biorender.com.
PennNeuroKnow 
