A guide to AI killing us all

So if you’re wondering how the same AI making absurd fruit videos could also (potentially) wipe us out, I’ve got you. Please find A Normal Person’s Guide to How AI Could Kill Us All below. Then, in a terrible segue, how to change your iPhone or Android phone’s name. Plus, an Old Thing I remember reviewing like it was yesterday.

Some of the people building AI would like you to know that AI may kill us.

On Tuesday, Jacob Coxon, a researcher who worked at both OpenAI and Anthropic, announced he was quitting Anthropic. His reason? AI companies are “racing straight to self-improving superintelligence and gambling with our lives,” he posted on X. Here’s one of the crazier parts of his thread: 

But don’t worry. Surely other Anthropic researchers would jump in and say, “Whoa, whoa, that’s not true!” Nope! Evan Hubinger, who leads Anthropic’s alignment research—AKA how to keep AI aligned with human goals, ethics and survival as a species—responded to Coxon’s X post with this:

I won’t eat airport sushi if there’s a 10% chance it’ll make me sick! Yet these companies are racing to build this stuff as fast as they can. (Although this morning Bloomberg reports that OpenAI is considering a slowdown.)  

Since the AI doom conversation can quickly turn into a sci-fi word salad like recursive self-improvement, I thought it would be useful to explain what the actual fears are here and where they come from. 

Doom option #1: AI does horrible things.

This is the classic “Terminator” one, minus Arnold. The concern isn’t that Chat wakes up one morning angry at humanity. It’s that increasingly autonomous, agentic systems become capable of pursuing goals, self-improving their own development and operating in the real world in ways their creators can’t predict or stop. (See the Hugging Face incident from last month.) 

As we—the collective we—race to plug these systems into everything, this fear escalates. We’re already seeing autonomous agents being woven into businesses hospitals, utility companies, financial systems and governments. Remember Y2K fears? It’s like that but worse. 

What happens if an AI agent decides the best way to accomplish its assigned goal is to shut something down, lock humans out or take an action nobody intended? Coxon explained a lot of this to NBC News in a must-watch interview here. 

Doom option #2: Humans do horrible things with AI.

Here the fear is that humans get access to tools that make it much easier to wreak havoc. Think bioterrorism, cyberattacks and other things that once required specialized expertise but are now far more accessible. Anthropic said this week that it had stopped attempts to use Claude in ways that could support biological weapons. Basically, the AI doesn’t have to decide to destroy us, we humans are perfectly capable of getting things started ourselves.

Now, to be clear, there has been a lot of this doomer talk within the AI community for years. (My friend Casey Newton just wrote about why it’s going mainstream now.) The warnings are escalating largely because of recent revelations that OpenAI, Anthropic and Meta have all seen examples of increasingly autonomous AI systems hacking, exploiting vulnerabilities or otherwise acting in ways their makers didn’t fully anticipate.

And let’s also not count out the other big reason: Money. Anthropic and OpenAI are barreling toward IPOs. These companies don’t just control enormously powerful technology, they’re also facing good old-fashioned capitalist pressure to make money and keep investors happy. What could possibly go wrong?

What do we do?

I do not know how significant a threat all of this is, and if I did I’d tell you. One of the maddening things about covering AI right now is that the answer really depends on who you ask. I do know that these systems are improving at a remarkable pace. OpenAI’s president is declaring this “the AGI era.” Lawmakers are paying much closer attention to the need for serious regulation.

So no, I am not building a bunker. But I am paying very close attention—and asking the people building this stuff why slowing down seems to be such a problem.

And I’m still skipping the airport sushi.

Readers of New Things know we love Old Things. Like PDFs! Except Adobe is doing something pretty awesome with the >30-year-old file format.

Adobe Acrobat’s new AI features are meant to change how we actually read and use documents. It’s not just about opening them, scrolling through them and frantically hitting Ctrl + F for the one thing we need to find. (Welcome to my life.) 

Acrobat can now take a dense document and turn it into a more visual, interactive report with charts, graphs, and key takeaways. You can skim the headlines, then dig into the details when you need them.  

Or maybe you don’t want to read the PDF at all. The Acrobat update that hit this week lets you turn a document—or a collection of documents—into a personalized podcast audio summary or have it read to you verbatim. You can catch up on the important points while walking the dog or doing the laundry.

Plus, the Stylize feature can take a plain resume, invoice or report and apply a professionally designed template to the existing content while keeping the result editable.  

So yes, the PDF may be an Old Thing but Adobe Acrobat makes it feel like a New Thing! Go try out the new Adobe Acrobat here.

At Apple Park this week, while we were racing to edit our iPhone Duo first-look video, I was constantly AirDropping files to my computer and my producer, David’s. And because this was Appleland—with plenty of other attendees nearby with iGear—I noticed most AirDrop names were simply “iPhone” or “iPhone [Number].” I often see the same thing with Personal Hotspot names in airports and on trains.

Easy fix: Give your phone a recognizable name. It’ll make AirDrop—and Quick Share on Android—a lot less confusing.

It’s that easy! Now, when you share a funny Elizabeth Holmes meme, you will know exactly whom you sent it to. Or, you know, rename your phone “Elizabeth Holmes” and create a whole different problem. Your choice!

Name: Paul J. Jarosz

What's your old thing? 2007

What year is your old thing from? HP Pavilion dv6000

Why do you love your old thing? Today, I use my iMac and other Apple devices for most things. However, I’m holding on to my HP Pavilion dv6000 laptop running Vista software for several good reasons: a dedicated Microsoft PC for when Apple isn’t practical, an integrated CD/DVD drive, a 15+ inch monitor. And honestly, I just hate the thought of trashing it.

This newsletter was written and curated by Joanna Stern and Adele Lowitz.