AI inferencing is headed for the network edge

Processing IoT/OT data as close as possible to the source has been a longstanding goal for IT leaders. Recent technological advances are now making it possible to perform full-blown AI inferencing at the network edge, opening the door for game-changing applications that can respond autonomously to sensor data in real time.

“The combined rapid growth of edge data and the imperative that businesses now have to leverage AI capabilities for business value are leading inevitably toward significant edge AI growth,” says Gartner analyst Thomas Bittman.

By 2028, more than two-thirds of enterprise-managed data will be created and processed outside the data center or cloud, Gartner predicts, and more than two-thirds of all enterprises globally will deploy edge AI by 2029, up from 10% in 2025.

Likewise, IDC expects that half of all enterprise AI inference workloads will run on endpoints or edge nodes by 2030, according to the firm’s 2026 FutureScape IT predictions. “Enterprises are increasing edge IT investments to support genAI/AI inference, with strong momentum in healthcare, finance, and manufacturing,” says Olga Yashkova, IDC’s research manager for edge AI strategies.

What’s driving edge AI?

A variety of factors are coming together to make edge AI a high priority for IT executives.

Data gravity: In 2025, there were about 11.7 billion IoT devices installed, and that number is growing by 9% a year, says Gartner. The volume of data generated by always-on devices like security cameras, traffic cams, or sensors embedded in critical infrastructure, is staggering.

Bittman points out that as much as 90% of edge data goes unprocessed. “The combination of improving technologies available for the edge and the importance of leveraging that data (especially with AI) will significantly increase the percentage processed over time,” he says. “The volume of data and the cost and delay of processing that data elsewhere will put more pressure on finding solutions to filter, process, and even store more of that data locally.”

Data control: Particularly in Europe, where regulations are the strictest, data residency, data privacy, and data sovereignty are key drivers of edge AI, says Paul Schell, an analyst at ABI Research. But beyond regulatory issues, control over data is a growing concern for every type of organization in every location, particularly now that sensor data can include facial imaging, fingerprints and other types of biometric data collected in a safety or security scenario.

Latency: Schell points out that many use cases are “multimodal,” combining embedded sensors, video from cameras, even audio from devices that listen for vibrations in industrial equipment. “Keeping that data on premise saves issues with data transfer and potentially losing that connection, which could impact anything that’s safety related.” He adds, “Having a high-definition stream going between the location of the use case and the cloud is quite difficult in many cases. Locating that at the edge is much easier.”

Cost: Yashkova adds that the networking costs associated with transporting data from the edge to the cloud makes edge AI a more preferable business decision. “AI is moving to the edge because latency, cost, and resilience demand it. Applications in verticals such as industrial automation, mission-critical control, and video analytics require onsite inference; sending data to the cloud and back is too slow, too expensive and too risky when connectivity is unreliable.”

What are the technological advances enabling AI at the edge?

The idea of running AI models at the edge might have seemed impossible a few years ago. After all, AI requires virtually unlimited processing power and scalability of the cloud. But, as Pete Bernard, executive director of the EdgeAI Foundation, explains, “the models are getting smaller and better. The chips are getting faster, and so that enables people to say, ‘Let me just do the processing where the data is created, as opposed to sending it up to the cloud, and then paying for tokens up there.’”

Yashkova agrees. “Better edge silicon and leaner AI models have made this practically achievable, and agentic AI is accelerating adoption further.”

On the chip front, AI edge models run on an advanced type of chip called a neural processing unit or NPU. These chips, such as Google’s Tensor Processing Unit (TPU) or Qualcomm’s Snapdragon, are small, efficient, and don’t use as much energy or create as much heat as traditional CPUs or GPUs. Yet they are extremely powerful, able to perform trillions of operations per second (TOPS).

Another breakthrough is the emergence of “neuromorphic” chips, which attempt to mimic how the human brain works. These chips, such as Loihi from Intel and TrueNorth from IBM, spring to action only when a meaningful event occurs, reducing energy consumption. They also deliver advanced data processing capabilities for real-time applications like robotics or autonomous vehicles.

Then there are AI accelerators specifically designed for edge AI deployments from vendors such as Hailo and BrainChip. And a new generation of small language models (SLM), such as Meta’s Llama 3.2, Google’s Gemma 3, and Microsoft’s Phi series, are now available. These SLMs provide strong performance at reduced scale, enabling organizations to train the AI models in the cloud, and then perform inference at the edge.

What are the use cases for edge AI?

Edge AI is becoming mainstream, says Yashkova. “Early leaders include manufacturing, telecom, and healthcare, but adoption is expanding across retail, government, financial services, media, utilities, and hospitality.”

Schell says: “One of the things about the whole AI revolution is that it has enabled companies to do things that they never thought they could do before. It’s not only doing the same thing faster and more accurately. It’s doing things you never even dreamed of five years ago.”

Edge AI spans operational technology, manufacturing, mobility, automotive, agriculture, smart cities, critical infrastructure, power and water management, Bernard says. “It’s hard to find a vertical market in the operational world that’s not impacted.”

Here are some examples:

• Manufacturing: Cameras and sensors can provide real-time quality control, taking autonomous action, such as shutting down a production line if a problem is detected. On-device intelligence can also deliver predictive maintenance. And video can improve safety by sending an alert in real time if a worker is approaching a hazardous zone or not wearing a hard hat.
• Retail: Cameras can monitor shelves for inventory purposes and detect theft.
• Agriculture: Drones and remote sensors can help farmers improve the yield on their acreage. Robots can drive tractors and other equipment. Bernard says he recently saw a demo of a robot that had the manual dexterity to pick strawberries.
• Pedestrian safety: Streetlights equipped with built-in cameras can detect if someone is crossing into traffic and can immediately emit a warning to approaching cars.
• Medical technology: AI models can provide real-time surgical guidance. Wearables can help monitor patient glucose levels.
• Mobility: Through the use of cameras and radar, today’s vehicles can provide lane correction and emergency braking.

How to get edge AI

The low-power, miniaturized world of edge AI doesn’t rely on the traditional data center infrastructure stack that IT execs are familiar with. Bernard explains that “as you get lighter, things get a little weirder.” Everything is different, from the chips to the software to the networking protocols. And the market hasn’t yet picked vendor winners and losers. “As you get farther out to the edge, there’s more diversity of opportunities and choices,” says Bernard.

Yashkova says options include packaged platforms and appliances, provisioned edge services from colocation providers, telcos offering multi-access edge computing (MEC), or CDN edge AI delivered as-a-service. The hyperscalers are also players in edge AI, particularly when it comes to ‘physical AI’ agents at the edge.

Some of the more interesting vendors in edge AI include Advantech, Aetina, Irida Labs, and AccelerAI, according to Schell.

What is the future of edge AI?

“I think lots of vendors and different parts of the ecosystem I’ve spoken with are looking at 2026, 2027 for there to be an inflection point, because interest has picked up massively, and enterprises are now looking to invest and see what they can do with it,” Schell says.

According to the EdgeAI Foundation, edge AI is the fastest growing segment within the AI wave, with a 37% growth rate through 2030, compared to 28% for the overall AI market.

Yashkova agrees that “edge AI is a genuinely fast-growing market with broad vertical applicability.” While early pilots are now scaling to broader rollouts, constraints include software/integration complexity, CapEx costs, and skills gaps. “The constraint on growth is less about demand and more about deployment complexity. Orchestrating workloads across endpoints, edge nodes, and cloud tiers remains technically challenging, and enterprise readiness varies.”

“But the direction is definite,” she adds. “Inference is distributing outward from the data center, agentic architectures are reinforcing that shift, and the infrastructure investment cycle is already well under way.”