Why Enterprise AI Agent Adoption is Still Slow
In this blog post series, Niklas Frühauf, Senior Data Scientist, explores whether AI agents already deliver real value for businesses. In part one, he outlined the core concepts and different types of agents. In part two, he examined SAP’s approach. In this final installment, he focuses on the four key challenges – what is still holding companies back from putting agents into action?
There is clearly a lot of momentum behind AI agents at SAP — and strong customer demand as well. That raises an obvious question: Why is enterprise adoption still relatively slow?
Part of the answer may simply be that setting up and licensing SAP Joule for agent development is not exactly straightforward. But the real issue was perhaps best summarized at last year’s SAP TechEd 2025:
Once companies move beyond proof-of-concepts and hackathons, bringing agents into production quickly turns into a serious engineering challenge. The tooling, architecture patterns, and operational practices around agents are still evolving rapidly, and organizations are actively trying to figure out what reliable production setups should look like.
Challenge 1: Human in the Loop
One of the most important requirements, not least due to regulations such as the EU AI Act, is maintaining some form of human-in-the-loop control.
In enterprise scenarios, businesses typically want expert users to be able to intervene when needed, validate agent decisions, or participate in verify-and-approve workflows. AI agents should support human work, not silently operate outside of established governance processes.
SAP is actually in a strong position here thanks to the integration between SAP Build Process Automation and Joule agents. Agents can trigger workflows, while workflows can also incorporate agents as automated steps within a broader human-centered process. This makes it relatively straightforward to combine AI-driven automation with traditional enterprise approval patterns.
However, a key challenge remains: How does an agent know when to ask for help? Ideally, agents should recognize when they lack the necessary tools, data, or business context to complete a task and escalate appropriately. In practice, designing agents that reliably detect such situations is still an open challenge.
Challenge 2: Evaluation and Optimization
Building an agent that works for a small, well-defined task is relatively simple. Building one that reliably performs across a wide variety of real-world scenarios is significantly harder. While prompt engineering and iterative tweaking can work for simple use cases, production-grade agents require a more structured build–evaluate–optimize cycle, similar to what we are used to from traditional machine learning projects.
This, however, introduces new challenges. Effective evaluation requires:
- labeled benchmark data
- representative real-world tasks
- expected outputs and tool results for comparison
Many organizations simply do not yet have these assets available, which makes systematic evaluation difficult. Beyond technical evaluation, companies must also determine the actual business value of their agents.
That requires visibility into operational metrics such as:
- error rates
- human intervention frequency
- task completion success
- time savings and productivity gains
These metrics should ideally be balanced against operational costs such as model usage or token consumption. Currently, especially in the context of Joule, there is limited transparency at the agent or skill level regarding AI unit consumption, which makes it harder for organizations to evaluate the true return on investment.
From a business perspective, solutions such as SAP Analytics Cloud could potentially be used to track and analyze these metrics. It will be interesting to see how SAP continues to invest in this area going forward.
Challenge 3: Monitoring, Auditing, and Governance
Closely related to evaluation is another critical challenge: monitoring, governance, and auditing.
Organizations need clear answers to questions such as:
- What exactly are our agents doing in production?
- Who is responsible when an agent makes a mistake?
- Can we track which actions were performed by which agent, in which system, and on behalf of which user?
- How do we maintain oversight when our landscape includes a growing mix of low-code agents, pro-code agents, and vendor-provided agents?
- In agent to agent scenarios, how do we correlate each individual agent’s actions?
SAP is starting to address these questions through a combination of tools. While operational visibility may be supported through capabilities in SAP Cloud ALM, governance, architectural oversight and discovery of agents and extensions may be facilitated through SAP LeanIX. However, as enterprise agent ecosystems grow, establishing clear governance models and operational transparency will likely become one of the defining challenges of large-scale AI adoption.
Challenge 4: Security Considerations
Beyond retroactive auditing, proactive security and authentication controls are equally critical when deploying AI agents in enterprise environments. After all, you wouldn’t want an intern to simply ask SAP Joule for the latest unreleased earnings summary and receive a helpful answer. Ensuring that agents respect existing access controls is therefore a fundamental requirement.
A key concept here is principal propagation. Any agent – Joule or otherwise – must know which user triggered the request and correctly propagate that user’s identity and authorization context across all downstream tool calls and data lookups. In practice, this means the agent should operate strictly within the permissions of the initiating user. Today, however, achieving this often requires custom implementations by pro-code developers, especially when integrating with third-party systems.
Relying on large language model reasoning alone is not sufficient for enforcing security boundaries. Prompt injection and data-poisoning techniques are becoming increasingly sophisticated, and security controls cannot depend on an LLM’s ability to “do the right thing.” Instead, technical safeguards must be enforced at the system level. Where strict authorization cannot be guaranteed, tools exposed to agents should default to read-only or non-destructive operations.
Even within the SAP ecosystem, interoperability challenges remain
For example, when agents query data from SAP Business Data Cloud, there is currently no fully native agent bridge that consistently enforces user-level or row-level access controls across all scenarios. As organizations begin experimenting with newer patterns such as the Model Context Protocol (MCP) or agent-to-agent communication (A2A), these challenges become even more complex. Ideally, every agent and tool in the chain must be aware of who initiated the request and apply the appropriate authorization rules accordingly. In particular, MCP-based tool integrations require careful handling of authentication and authorization to avoid unintended data exposure.
Another related question is who is allowed to access which agents in the first place. Within Joule, administrators can restrict certain agents or capabilities to specific user groups. However, for custom pro-code agents deployed on SAP Business Technology Platform, similar governance mechanisms typically need to be implemented manually.
Finally, regulatory and data protection requirements may also influence architectural decisions. In some cases, particularly for workloads subject to stricter GDPR requirements under the General Data Protection Regulation, organizations may need to rely on Europe-hosted or open-source large language models. Pro-code developers can already achieve this through SAP AI Core (“Generative AI Hub”), which support open-source models such as those from Mistral AI. In contrast, Joule currently does not expose configuration options for selecting or controlling the underlying LLM, and the exact foundation model used is not publicly documented.
The result: Are AI agents already transforming business processes?
We love exploring new technologies and adopting them early – always with one key question in mind: how can SAP customers truly benefit? That’s exactly how we approached AI agents.
With our long-standing experience in both traditional machine learning and generative AI, combined with extensive hands-on work with AI agents, we support organizations across the full spectrum of agent development. From low-code agents built with Joule Studio to complex, fully customized pro-code agents, we have successfully delivered production-grade solutions on both ends of the spectrum.
The concept behind agents and SAP’s approach to agents are highly promising and have the potential to transforming business processes. So what’s holding companies back? On the one hand, the right use cases for agents still need to be identified. On the other, organizations require the right expertise and tooling to implement agents effectively and overcome the associated challenges.
Ready to get started with agents? Then the sovanta AI Factory is the perfect platform. Request a demo today.
