Artificial intelligence is becoming the go-to guide for everything from dinner picks to login pages. But what happens when your AI assistant confidently sends you to a fake site? Recent research from Netcraft shows a real problem: AI models often give wrong URLs and send users straight into cybercriminals’ hands. 

This is not a made-up problem. It is happening now, at scale, and it is a serious security issue. It is also a fresh form of AI phishing. 

The Shocking Scale of AI URL Errors 

When cybersecurity researchers at Netcraft decided to test how well AI models handle sWhen Netcraft researchers tested how well AI models handled simple login requests, the results were alarming. They asked a popular large language model direct questions, such as how to log in to [brand name], for 50 well-known companies. 

The AI gave 131 unique website addresses. The split was stark: 

  • 66% were correct — the AI got it right about two-thirds of the time 
  • 29% were unregistered or inactive domains — digital property waiting to be claimed by attackers 
  • 5% belonged to unrelated businesses — sending users to the wrong place altogether 

That means 34% of the time, users were sent to sites not owned by the brands they wanted. For cybercriminals, that is a major opening. 

When AI Becomes a Phishing Tool 

The research found something even worse: AI systems are already pointing users to live phishing pages. In one case, Perplexity — a popular AI search tool — sent users to a fake Wells Fargo login page when asked for the official URL. 

The bad site was not buried deep in search results. It was the top answer, shown with the same confidence as a real one. The true Wells Fargo site appeared below it, which made the scam even more convincing. 

This is a weak point in how people use AI. Unlike a search engine that shows many results, AI gives one answer with authority. People are trained to trust it, which makes false links harder to spot. 

The Perfect Storm for Cybercriminals 

AI errors create a perfect storm for criminals. Here is why the risk is so high. 

Prebuilt victim paths 

When AI models keep giving the same fake URL, attackers can register that domain and wait for victims. It is a simple path from model error to stolen credentials. 

Trust by default 

AI does not present URLs with caution. It gives them with the same certainty as verified facts, which makes users less likely to check if the address is real. 

Scale and automation 

As AI interfaces become the normal way people search, the victim pool grows fast. A single fake URL can reach thousands of users. This is where AI cyber attacks get real leverage. 

Smaller brands face more risk 

Regional banks, credit unions, and mid-sized firms are often less visible in AI training data. That makes wrong URLs more likely, and the damage can be worse. 

The GitHub Poisoning Problem 

The threat goes beyond bad answers. Criminals are also poisoning AI training data so future outputs are easier to abuse. Researchers found a detailed campaign in which attackers created fake GitHub repos with malicious code meant to be consumed by AI dev tools. 

That is a clear example of AI cyber attacks: seed the data, then let the model spread the bait. As AI coding assistants become more common, the risk of wide code poisoning grows too. 

Why Traditional Defenses Fall Short 

Some teams may think they can stop the problem by registering every possible fake domain. In practice, that does not work. 

Endless variations 

AI models can make unlimited URL variants. The combinations are endless, so full domain registration is not realistic. 

Changing patterns 

As models are updated and retrained, they create new error patterns. What looks complete today may fail tomorrow. 

High cost 

The time and money needed to register and keep up thousands of defensive domains can become too much. 

How to Protect Your Organization 

While the AI hallucination problem is complex, there are still practical steps teams can take. 

Use AI guardrails 

Add validation checks before a URL is shown to users. That can include real-time domain ownership checks and reputation checks. 

Watch for fake domains 

Set up ongoing monitoring for domains that may be registered to impersonate your brand. Include common misspellings and AI-style mistakes. 

Train employees and customers to verify URLs on their own, especially for banking or account access. For sensitive tasks, they should type the URL directly or use saved bookmarks. 

The Future of AI and Security 

The URL problem is not going away. As these systems get more capable, criminals will keep looking for new ways to exploit them. As agentic AI for phishing becomes more capable, the same risk can move beyond search and into action-taking assistants. 

Organizations need to prepare for a future where AI-driven threats are as normal as other security issues. The goal is not to avoid AI. It is to use it with better controls, better checks, and more user training. 

Building Resilience in an AI-Driven World 

As AI becomes part of daily work, the cost of errors keeps rising. Organizations that move early to address AI URL errors will be better able to protect users and keep trust. 

The answer is to use AI more carefully, not to throw it out. That means using proper validation, keeping human review for critical calls, and always watching for new threats. 

Q&A 

Question: What is an AI URL error? 

Short answer: An AI URL error is when an AI system gives a website address that is wrong, inactive, unregistered, or tied to the wrong brand. In login flows, that is risky because users may trust the answer and enter sensitive credentials on a fake or malicious site. 

Question: Why are fake URLs useful to cybercriminals? 

Short answer: Fake URLs can create ready-made phishing chances. If an AI model keeps suggesting the same bad domain, attackers can register it, build a convincing fake login page, and wait for users to arrive through trusted AI results. 

Question: Why can’t companies simply register every fake domain? 

Short answer: That is not realistic because AI models can make endless URL variations. Those patterns can also change as models are updated, and keeping thousands of defensive domains would be expensive and hard to manage. 

Question: Which organizations are most at risk from AI-made phishing links? 

Short answer: Smaller and mid-sized organizations, including regional banks and credit unions, may face higher risk because they are less likely to be well covered in AI training data. That can make wrong URLs more likely when users ask AI systems for login pages. 

Question: What steps can organizations take to reduce this risk? 

Short answer: Organizations can reduce risk by checking URLs before showing them to users, watching for fake or look-alike domains, and teaching employees and customers to verify web addresses on their own. For sensitive tasks, users should type known URLs directly or rely on saved bookmarks rather than trust AI-made links.