Rohan summarized a research paper about the effects of LLMs and Google on brain usage alongside effectiveness. The study took people in three situations when writing essays: brain only, Google + brain and LLM + brain.
For brain usage, it's what you would expect. The usage was the highest with only brain, lowest with LLM and in the middle for Google.
Essays produced with ChatGPT were clustered in terms of words and thoughts. Google was more spread out but was very much influenced by the search engine results. The creativeness of just the brain was the best.
In terms of memory, Google and Brain Only were the best - they were able to recall most passages from the essay. With ChatGPT, only 17% of sentences were remembered.
The scariest part to me was the lingering effects. When a ChatGPT only user tried to write only using the brain, they had 32% less brain activity. I guess the brain thinks that the tool is coming?
When the brain-only writers switched to ChatGPT, their revisions were fantastic and brain usage increased. To me, this demonstrates that starting with only your brain is better than starting with the LLM tools.
Overall, an interesting study into the effects of LLMs!