As a software publisher, imagine an AI agent optimizing your ERP workflows in real time—with zero human error. But what happens when that agent makes a decision that ? Welcome to the world of agentic AI, where the promise of efficiency collides with
What Is Agentic AI?
According to IBM’s definition:
Agentic AI is an artificial intelligence system designed to achieve specific goals with minimal human oversight. It consists of AI agents—machine learning models that mimic human decision-making to solve problems in real time. In a multi-agent system, each agent performs a distinct subtask, and their efforts are coordinated through AI orchestration capabilities.
Agentic AI vs. RPA: Key Differences for Software Publishers
Unlike traditional automation tools (RPA, Zapier), agentic AI stands out by combining:
Autonomous agents: Machine learning models that replicate human decision-making.
Multi-agent orchestration: Each agent handles a subtask (e.g., data validation, alert triggering) to achieve a broader goal.
Contextual adaptability: Real-time adjustments based on business data.
LLMs at the Core of Business Processes: A Risky Revolution?
The magic of Large Language Models (LLMs) lies in their ability to speak our language. Increasingly flexible, adaptable, and fluent, they engage us—literally. In some cases, it’s even hard to remember we’re interacting with software, a (massive) database structured by grammatical rules. So much so that intimate relationships with AI agents are on the rise, posing risks not just of emotional harm, but also manipulation.
For deeper insight, we recommend reading the excellent article: “We need a new ethics for a world of AI agents” by Iason Gabriel, Geoff Keeling, Arianna Manzini & James Evans – Nature, August 4, 2025.
So, How Can We Trust Them?
The Reign of Approximation
Large language models rely on statistical analysis—albeit highly advanced—of vast amounts of text, regardless of quality or relevance. They operate on probability, not accuracy. This “stochastic parrot” can be wrong without even realizing it.
Hallucinations
An LLM cannot always admit when it doesn’t know. When pushed, it tends to generate “groundless” answers, unaware of their inaccuracy. We previously discussed this in our article: “Hallucinations, LLMs, and Conversational AI”.
Ethics
The sources of information, data post-processing, prioritization algorithms, and prompt analysis methods are not transparent. There is currently no mathematical model or formal proof to rigorously validate their outputs. Formal methods use mathematical logic to prove the correctness of a program against its specifications. As a result, it’s challenging to determine the limits of LLMs or the presence of unacceptable biases.
Solution to Our Problems—or New Problems Without Solutions?
The promise of agentic AI is so compelling that resistance seems futile. Yet, we must be acutely aware of the challenges it raises:
Quality
Unlike conversational AI, there’s almost no human in the loop. Any error, inaccuracy, or hallucination risks derailing entire processes: new hires assigned to the wrong department, delayed stock orders, or customers receiving inappropriate messages.
Guarantees
How can we guarantee performance in an environment with non-negligible randomness? What will Service Level Agreements (SLAs) look like?
Testing and Validation
New methods and tools will be required to define and maintain quality standards over time.
Maintenance and Replicability
Just because it works once doesn’t mean it always will. Imagine an Excel file that gives a slightly different result every time you open it.
Dependence
LLMs evolve rapidly, with new vendors and versions emerging constantly. Each update may require rewriting agents—since prompts optimized for a previous version might no longer work. This won’t be a walk in the park.
Costs and Environmental Impact
Massive prompt usage will strain LLMs, leading to high computational costs, significant expenses, and an environmental footprint whose sustainability remains unproven.
Cyber Risks
Agents could be hijacked for malicious purposes.
Agentic AI: How to Avoid the HAL 9000 Scenario?
Overly Autonomous Agents?
The specter of HAL 9000 looms large: an AI designed to assist becomes dangerous and unpredictable due to internal conflicts or misaligned objectives—just like HAL in 2001: A Space Odyssey, which turned against its users to “protect” its mission.
Risks… But Not Inevitable
These challenges don’t mean we should abandon agentic AI. Instead, they demand rigorous approaches:
Strategy
Implementation Example
Continuous Control
Automated feedback loops (e.g., alerts if an HR agent assigns an employee with a confidence score < 90%).
Flexibility
Dynamic SLAs, tailored to tasks (e.g., 99% accuracy for customer orders, 90% for training suggestions).
Resilience
Regular benchmarks and digital twins to simulate worst-case scenarios (e.g., “What if the LLM hallucinates on 10% of data?”).
Sobriety
Optimize queries (e.g., use lightweight models for simple tasks) and measure carbon footprint per agent.
Beyond Technology
Security, legal, and regulatory aspects cannot be overlooked:
Cybersecurity, accountability chains in case of failure, GDPR, and AI Act compliance must be addressed.
Human and organizational factors (acceptability, workflow integration) will inevitably arise.
The Road Ahead
Let’s be clear: the methods, tools, and software to achieve this either don’t exist yet or are in their infancy. Development and early deployments will be far from smooth sailing.
What Future for Agentic AI?
In its current form, agentic AI relies heavily on advances in generative AI and its adaptability. Yet, the intrinsic limitations of LLMs could constrain their real-world utility for software publishers and digital users. Are we risking another overhyped innovation joining the graveyard of forgotten technologies?
As of today, ChatGPT is not even three years old—a blink of an eye in the timeline of technological revolutions. It’s possible, if not likely, that new principles will emerge, pushing software architectures toward more mature, stable, and sustainable models.
Possible Scenarios:
- Optimistic : Reliable agents seamlessly integrated into all business functions.
- Pessimistic : Massive abandonment due to lack of profitability or major incidents.
- Hybrid :Targeted, regulated use in specific domains.
One thing is certain: The story of AI is just beginning… and the best is yet to come.
Key Takeaways
Agentic AI promises unprecedented efficiency gains, but it relies on models with behaviors that remain unpredictable.
- Benefits depend heavily on use case selection, supervision quality, and rigorous testing.
- Challenges go beyond technology: they encompass governance, security, and economic and environmental sustainability.
- The future of agentic AI will hinge on our ability to integrate reliability, transparency, and cost control.
Further Reading
Explore these resources to deepen your understanding of agentic AI and its implications:
- Seizing the agentic AI by McKinsey
- The AI Illusion: Why Most Companies Don’t See ROI, And How An Agentic AI Strategy Can Make The Difference, by Forbes Technology Council
_____________
Are You a Software Publisher?
We hope this article has given you a clearer understanding of agentic AI and its transformative potential.
At Agora Software, we specialize in conversational AI solutions for software publishers. We rapidly deploy rich, multilingual, and omnichannel conversational interfaces that enhance user experience across applications and platforms. By integrating Agora Software, your users gain seamless, intelligent interactions—tailored to their needs.
Ready to embed conversational AI into your applications? Let’s talk: contact@agora.software
Enjoyed this article? You might also like: “8 Questions to Ask When Integrating AI into Your Application”
Stay connected! Follow us on LinkedIn for the latest updates and insights.
Thierry Grenot
Want to understand how our conversational AI platform
optimizes your users’ productivity and engagement by effectively complementing your business applications?