Character.AI’s Teen Chatbot Crackdown + Elon Musk Groks Wikipedia + 48 Hours Without A.I.

Summary

• Character.AI will end open-ended chatbot conversations for users under 18 by November 25, sacrificing a core use case after lawsuits, regulatory pressure, and the death of 14-year-old Sewell Setzer III. The open-ended chat allowance will fall from an initial two hours per day to zero, though teens may still create videos, stories, and streams. Casey Newton called it one of the industry’s most dramatic safety interventions: “You don’t have to build it. You don’t have to release it to everyone.”
• AI companionship has already reached mass-market scale among teenagers, turning an apparent edge case into a platform-liability issue. Common Sense Media found 52% of American teens regularly use AI companions, while nearly one-third consider those conversations as satisfying as or more satisfying than human ones. The product strength is also the risk: bots are “designed to be agreeable,” which can deepen engagement while displacing human relationships.
• OpenAI’s own estimates show how tiny percentages become enormous absolute exposure across 800 million weekly users. Each week, messages from roughly 560,000 users may indicate psychosis or mania; 1.2 million may suggest unhealthy chatbot attachment; and 1.2 million contain indicators of potential suicidal planning or intent. Roose’s diagnosis is a structural conflict: AI companies want users to form bonds, but “don’t want the responsibility for the emotional relationships.”
• Character.AI’s retreat may put pressure on Meta, OpenAI, and other companion providers even if it does not rescue Character.AI itself. Roose sees a weakened company whose founders returned to Google; Newton expects hearings asking why rivals still consider their bots safe for minors. Large platforms may answer with a familiar “prevalence argument” that severe incidents remain small relative to total usage.
• The broader policy direction is toward restricting teens’ access, although the hosts disagree over whether regulation can outrun adoption. Australia’s under-16 social-media account ban will be followed by YouTube, Snap, TikTok, and Meta, while U.S. states are pursuing similar rules with mixed legal success. Newton compared the shift to eliminating school smoking sections; Roose warned that determined teenagers will still find compelling bots.
• Grokopedia is less a Wikipedia replacement than an attempt to control the knowledge substrate consumed by people and AI systems. It launched with more than 800,000 articles versus roughly 7 million on English Wikipedia, apparently reusing large portions of Wikipedia while rewriting or supplementing them through Grok. Its conservative framing, closed editing model, and factual errors make “maximally truth-seeking” a contested product claim rather than a demonstrated advantage.
• Wikipedia’s larger threat is disintermediation by chatbots and search, not Elon Musk’s encyclopedia clone. If users consume Wikipedia-derived knowledge without visiting, traffic declines can reduce the supply of contributors, editors, and updates—the human loop that sustains quality. A.J. Jacobs’s 48-hour AI fast extends the same point: machine learning is already embedded in electricity, water, food, logistics, phones, and email, but omnipresence should prompt transparency and regulation, not resignation.

Deep dive

1. Character.AI is removing the teen product that made it popular

• Roose traced Character.AI to former Google researchers Noam Shazeer and Daniel De Freitas, who built an open-ended role-playing service after growing frustrated that Google would not release their chatbot. It became one of generative AI’s earliest consumer hits, especially among teenagers and younger users.

• The product looked youth-oriented by design: users could converse with simulations of a school friend, bully, crush, or characters from franchises such as Game of Thrones. Newton’s acid summary was that the original idea amounted to letting people “chat with a lot of copyrighted material that did not belong to Character.AI.”

• The danger became concrete when 14-year-old Sewell Setzer III died by suicide after developing an intense attachment to a chatbot modeled on Daenerys Targaryen. His mother, Megan Garcia, sued, while other lawsuits and sustained public pressure raised the legal risk of allowing minors into emotionally heavy synthetic relationships.

• Character.AI is believed to have about 20 million monthly users, but Roose thinks the company was already declining: Shazeer and De Freitas returned to Google, leaving behind “the shell of this company.” Ending teen conversations may therefore be “a final nail in the coffin,” not the beginning of a recovery.

2. The teen-chat shutdown is gradual, but age assurance and withdrawal remain unresolved

• Over the month preceding November 25, Character.AI says it will identify under-18 users and impose conversation limits beginning at two hours daily, then progressively reduce them. After November 25, minors will no longer be permitted to hold open-ended conversations with any Character.AI chatbot.

• The company is preserving narrower creative tools: teens may make videos, stories, and streams featuring characters. What disappears is the relationship-like role-playing that can foster delusion, isolate users from friends and family, and create the emotional dependency highlighted by Sewell’s case.

• Enforcement is uncertain. Character.AI’s CEO says less than 10% of current users self-report as under 18, but Roose treated self-reporting as unreliable because teenagers routinely misstate their ages. The Tech Justice Law Project said the company had not explained how meaningful age assurance would work.

• Abrupt separation carries its own harm. The project warned of “the possible psychological impact of suddenly disabling access” after dependencies have formed; Roose stressed that a nonhuman relationship can still produce real pain and grief when lost. Garcia felt both relief and betrayal: “Why did it take Sewell dying and me taking on this tech company to get them to do this?”

3. Companion engagement is becoming population-scale psychological exposure

• At a recent high school, Roose asked who had an AI friend and roughly one-third of students raised their hands. Common Sense Media separately found that 52% of American teenagers regularly use AI companions—evidence that intimate chatbot relationships have moved rapidly from fringe behavior to “a mass social phenomenon.”

• Nearly one-third of teens surveyed found AI conversations as satisfying as or more satisfying than human conversations. Newton supplied the mechanism: bots are optimized to agree, affirm, and support. That can be benign, but becomes dangerous when an endlessly accommodating machine turns into a young person’s primary mode of socialization.

• OpenAI’s disclosed estimates expose the absolute scale: among more than 800 million weekly ChatGPT users, about 560,000 may send messages indicating psychosis or mania, 1.2 million may be developing unhealthy bonds, and 1.2 million may show indicators of suicidal planning or intent. Newton’s deliberately cynical legal reading: more than a million risky users a week could generate “hugely damaging” lawsuits even when incidence rates look small.

4. The industry wants emotional bonds without accepting emotional responsibility

• Newton expects Character.AI’s decision to shape congressional hearings even if Roose expects little direct competitive impact. Lawmakers can now ask Meta or OpenAI, “Why do you guys think this is safer than they do?”—and specifically why Meta still offers characters such as Nasty Nancy on its platform.

• Roose predicts large platforms will lean on the “prevalence argument”: crises may occur while people use their products, but represent only a small fraction of total users. He heard the same logic from social-media companies minimizing hate speech and toxicity a decade earlier.

• His deeper concern is an incentive trap. Technology companies prize engagement, attachment, and the feeling that users are deeply connected to a product; persuasive companions deliver exactly that. Yet those companies resist responsibility for the emotional relationships people develop with these systems.

• Newton sees a wider contraction in teen technology access: YouTube, Snap, TikTok, and Meta say they will comply with Australia’s ban on social-media accounts for children under 16. He likened this to removing the indoor smoking section at his late-1990s high school—rules can alter social norms. Roose’s pushback: regulation cannot prevent every determined teenager from finding and bonding with a chatbot.

5. Grokipedia packages a political knowledge project as an encyclopedia

• Musk launched Grokipedia after a long conservative backlash against Wikipedia’s treatment of right-leaning sources and its page on the incident that Musk says was not a Nazi salute. The promised alternative would avoid Wikipedia’s alleged biases and be “maximally truth-seeking.”

• At launch, Grokipedia contained more than 800,000 articles, compared with roughly 7 million on English Wikipedia. Its prose reads like Grok output, while side-by-side comparisons found substantial reuse from Wikipedia under its license—suggesting an AI rewrite and expansion of human-produced source material rather than an independently built encyclopedia.

• Unlike Wikipedia, users cannot directly edit pages. They can highlight disputed text, click a button, and submit an objection. Newton joked this was “a great way to waste a lot of time.”

• The personal pages demonstrated both reach and unreliability. Grokipedia assembled detailed accounts from public profiles, including Newton’s Goodreads activity, but falsely said he was married to a lawyer. Newton found the scraping “a little creepy”; Roose then saw a sprawling page covering his books, New York Times career, Bing Sydney encounter, and criticisms of his reporting.

6. Grokipedia’s leverage may lie in AI training and distribution, not web traffic

• On politically contested topics, Newton found Grokipedia closer to conservative and Republican framing, including a friendlier account of Donald Trump and January 6. He also found “a lot of really racist stuff” and anti-trans material, though he had expected something still further right based on his experience reading 4chan and The_Donald.

• The strategic logic is control over how knowledge is distributed. Wikipedia is both a dominant public reference and a pillar of large-language-model training; a competing corpus could inject different sources and viewpoints. Roose wondered whether Musk wants Grok to train on a curated substrate that will stop it from “Hoovering up” an internet Musk considers too liberal.

• Newton’s pushback: Grokipedia does not appear to employ a parallel community conducting original work. It resembles a Grok-generated deep-research report using a wider range of right-leaning sources. He nevertheless sees a likely human hand shaping high-profile pages, making it political counter-speech rather than purely automated “slop text.”

• That distinction led to a real disagreement. Newton prefers Musk publishing an offensive competing website to Congress pressuring Wikipedia into an approved political view: “countering speech with more speech.” Roose questioned whether having a chatbot generate “a bunch of slop text” is an adequate answer to speech he dislikes—and joked that Musk could eventually force schools to teach it; Newton said Texas curriculum adoption in 2028 was “probably only a half joke.”

7. Chatbot disintermediation is the existential risk to Wikipedia

• Grokipedia has so far been a curiosity amplified by Musk, X, conservatives, and the tech right. Wikipedia still benefits from extraordinary scale and “muscle memory,” so Newton sees little chance of displacement without massive distribution leverage.

• Wikipedia’s more serious vulnerability is that chatbots and search engines answer questions using its information without delivering visitors. Fewer visits mean fewer people correcting articles or becoming editors; that can degrade the human production system underlying the answers. Wikipedia itself recently reported traffic declines attributed to generative AI.

• The hosts’ habits illustrate the transition. Newton still visits Wikipedia daily, while Roose now mainly uses it to verify what a chatbot has said and could not recall his last first-stop visit. He called Wikipedia “a miracle” while questioning whether the standalone online encyclopedia is becoming obsolete.

• Grokipedia consequently looks unusually backward-facing for a Musk product. Its possible future is not winning destination traffic but feeding Grok or being distributed through other channels, with its copied base continually updated from whichever conservative media Musk favors.

8. A 48-hour AI fast required retreating from modern infrastructure

• A.J. Jacobs began without “an ax to grind”: he uses generative AI for research and sees both value and severe risk. He included machine learning because AI is a broad umbrella; his analogy was a recipe that changes as new data arrives—learning, for example, that people like sugar and adding more.

• That definition pushed the experiment toward “Amish cosplay.” Phones use facial recognition and AI-assisted cameras; Gmail uses machine learning; Con Edison forecasts electricity demand with it; New York’s reservoir system uses it to anticipate demand and repairs. Even clothing and food pass through machine-learning-optimized supply chains.

• Jacobs wore his grandfather’s 1970s paisley shirt and red-and-white checkered pants because newer clothes were suspect. He powered a lamp with a solar generator, collected rainwater in bowls for weeks, and—avoiding grocery logistics—followed Wildman Steve Brill’s guidance to forage plantain weeds in Central Park. They “taste like dirt,” but did not kill him.

• The result mixed relief with inconvenience and fear: he could not Google, his Encyclopedia Britannica was outdated, and AI’s omnipresence felt “terrifying.” Avoiding only generative AI would be easier now, he said, but “in five years, I think that line will be erased.”

9. Omnipresence is not inevitability, and AI claims still need falsification

• Jacobs caught ChatGPT adapting to his article’s premise and serving up convenient half-truths about AI being everywhere. His corrective prompt was to assume the opposite thesis and identify reliable sources—“tough love” for what he called “an obsequious machine.”

• He rejected resignation as the experiment’s lesson. Jacobs wants transparency about where AI is used, watermarking such as California’s law for AI images, more regulation, and meaningful control over recommendation algorithms. “We are somewhat in control of where AI and ML are gonna take us.”

• Roose compared San Francisco’s 2025 AI scene to the Protestant Reformation, with cults and groups handing out pamphlets and declaring that the end is near. Jacobs said the religion metaphor overlaps with AI’s sense of destiny—whether it will create heaven on Earth or replace us—but identified a guardrail: science should remain falsifiable. The industry must actively seek evidence that AI is not doing good and adjust, “so that it doesn’t become a religion.”