The agentic web is here

In 2001, a decade after he invented the World Wide Web, Tim Berners-Lee articulated a new vision. Together with two colleagues, he published an article in the magazine Scientific American in which he outlined what he called the “Semantic Web” : an internet designed to be navigated by autonomous software agents that could address our every need, without requiring us to muck about with browsers and search engines ourselves.

To illustrate what this might look like, Berners-Lee and his colleagues told a story about a pair of siblings who use their agents to automatically research and book physical therapy appointments for their mother, making sure they would be covered by insurance and fit their busy calendars.

At the turn of the millennium, this vision seemed far-fetched: Google had only been online for a couple of years, and most people were still on dial-up connections. But a quarter-century on, the dream of autonomous software agents that use the internet and carry out tasks for us has finally been realized. New tools such as Meta’s blockbuster Muse agent, OpenAI’s Dots, and Anthropic’s are being given access to people’s email inboxes and bank accounts, Meanwhile, in a series of worrying incidents, agents created by OpenAI broke free of their sandboxes and caused havoc online.

Just as the rise of smartphones sparked the development of a more atomized and attention-grabbing internet, the advent of powerful AI systems heralds a new era: We’re facing not the semantic web but the agentic web. This new web is developing out of sight and in ways that might not be entirely within our control. As it grows, the agentic web will inevitably transform the human web—and whether the human web survives this transition is anyone’s guess.

For Berners-Lee’s original vision to work, internet users would have had to annotate the pages they built with special tags so that a software agent could understand each element’s meaning—say, “street address” for the part of a physical therapy office’s web page that indicates its location and “appointment types” for the list of services. And so the idea never really caught on. “It’s a lot of labor,” says Eni Mustafaraj, a professor of computer science at Wellesley College. “And the benefits of that were not obvious.” For the next two decades, then, the internet remained a place that put humans first, with the odd Twitter reply bot or web scraper to round things out.

Large language models changed that. Unlike the agents that Berners-Lee was imagining, today’s LLM-based agents don’t need any tags to understand which part of the text on a clinic’s site is its address and which details the services it offers. They can search for and process text far faster than any human, and agents with multimodal capabilities can even interpret screenshots of websites and click on specific elements such as hyperlinks and drop-down menus.