AI agents make life easier by automating tasks that would normally require human efforts, such as question-answering, data collection, and decision-making. They play a crucial role in the GenAI space, owing to their accuracy, consistency, scalability, data-driven self-training and self-improving capabilities. They help developers focus on streamlining workflows, making GenAI development more efficient and enjoyable. With the Open Platform for Enterprise AI (OPEA), you can efficiently build robust, scalable and secure AI agents that can be easily integrated into the enterprise sector. In a recent webinar, our experts demonstrated how OPEA enables designing, customizing and deploying Retrieval Augmented Generation (RAG) based AI agents, harnessing the power of Intel® Gaudi® AI accelerators and Intel® Xeon® processors.
The webinar covers the following topics:
-
Basic concepts of an AI agent,
-
Benchmarks illustrating the benefits of Comprehensive RAG (CRAG) over the traditional RAG or non-agentic GenAI solutions, and
-
How to build and deploy hierarchical multi-agent systems on Intel hardware.
The webinar speakers are:
-
Alex Sin – He is an AI software solutions engineer at Intel who enables customers to build AI applications on Intel’s data center products, including Intel Xeon CPUs and Intel Gaudi AI accelerators. He also supports various software tools and libraries including Intel® Extension for PyTorch* and OPEA.
-
Louie Tsai – He is an AI software solutions engineer at Intel.
-
Minmin Hou – She also is an AI software solutions engineer at Intel and leads the development of agentic AI solution on OPEA.
This blog will give you highlights of the webinar, using it to provide deeper insights into the OPEA framework.
Check out the full webinar recording: Build and Deploy OPEA AI Agents on Intel® Xeon® Processors and Intel® Gaudi® AI Accelerators.
What is an AI Agent?
An AI agent is a software program that interacts with its environment, collects data, and autonomously performs tasks to achieve goals set by humans. It operates independently, choosing the best actions to meet user requirements, similar to training a new employee to perform a task.
Watch the webinar recording from [00:05:15] to learn more about the basics of AI agents.
AI agents are capable of managing tasks such as answering questions, making recommendations, troubleshooting issues, and following up. They prove useful across multiple industries, including finance, manufacturing, and commerce.
Self-training of an AI agent involves four steps:
-
Data Collection: The agent gathers data from its environment, such as through customer interactions, transaction histories, or social media.
-
Action Determination: This process analyzes the collected data to identify patterns and decides the best course of action.
-
Execution: It performs the chosen action, like answering a query or processing a request. If it can't resolve an issue, it escalates it to a human.
-
Learning from Feedback: The agent updates its own knowledge based on user feedback to improve future interactions.
From [00:07:55] in the webinar recording, Alex talks about types of AI agents as follows:
-
Simple Reflex: Basic chatbots that detect keywords to perform tasks like password resets.
-
Model-Based Reflex: Evaluates outcomes and fills in missing information to make context-based decisions.
-
Goal-Based: Handles complex tasks in areas such as natural language processing (NLP) and robotics.
-
Utility-Based: Optimizes routes or costs, easing tasks like booking the cheapest flights.
-
Learning: Uses reinforcement learning to improve through rewards and penalties in simulations.
-
Hierarchical: Consists of lower-level agents performing tasks assigned by a supervisor agent, who aggregates results for complex decision-making. Hierarchical multi-agents enhance efficiency, performance, and accuracy in applications like question answering.
The webinar mainly focused on seamless development and deployment of hierarchical multi-agent systems using the OPEA environment. Before we jump into the details, let’s understand what OPEA is and its value to AI developers.
About OPEA in Brief
OPEA is an open-source project introduced by the Linux Foundation in collaboration with 50+ industry leaders, including Intel. The initiative aims to develop open, modular, and production-ready GenAI systems that can be utilized across multi-vendor architectures. It supports a wide range of hardware, including Intel Xeon Processors, Intel® Arc™ Graphics, and Intel Gaudi AI accelerators.
OPEA comes with the following for AI developers:
-
GenAI components in the form of plug-and-play microservices
-
Architectural Blueprints: Comprehensive frameworks for developing GenAI systems, incorporating LLMs, data stores, and prompt engines
-
Ready-to-use GenAI solutions that can be tailored to specific business requirements
Guy Tamir is a technology evangelist at Intel. Refer to his ‘AI with Guy’ playlist on Intel Software YouTube channel for insightful videos based on the latest AI software developer resources. Check out his introductory video on OPEA:
OPEA-built Agentic RAG vs. Traditional Non-Agentic RAG
From [00:13:24] in the recording, Minmin discusses a couple of benchmarks showing how RAG, powered by AI agents, gives better results than the conventional non-agentic GenAI solutions.
The Comprehensive RAG (CRAG) benchmark released by Meta* in 2024 tests RAG’s reasoning and response synthesis capabilities through factual questions. Intel evaluated the results generated by non-agent RAG, single OPEA RAG agent, and hierarchical multi-agent system (with OPEA ReAct agent as a supervisor and OPEA RAG agent as a worker) against human accuracy and using the Ragas* library with Meta Llama 3.1 70B model as a judge. In both scenarios, the single-agent RAG was found to perform better than the conventional non-agentic RAG. The hierarchical multi-agents further enhanced accuracy by accessing additional information from knowledge graphs. This demonstrates the substantial advantages of hierarchical agents over conventional RAG methods.
Another benchmark discussed in the webinar is the Table-Augmented Generation benchmark (TAG-Bench) released by Stanford University* and University of California, Berkeley* (UC Berkeley) in 2024 and advocated by Databricks*. It tests RAG’s ability to answer database-related questions requiring additional world or common-sense knowledge. Intel compared AI agents to Text2SQL and human accuracy and found that the optimized SQL agent nearly doubled Text2SQL performance, though still below human accuracy. This highlights the significant improvement in performance using AI agents over single LLM calls.
Why OPEA AI Agents?
Here are some of the major benefits of OPEA AI agents that make them a better choice over proprietary agent frameworks, as highlighted in the webinar:
-
Fully open-source and no-cost platform
-
Higher degree of data and model privacy since you can deploy OPEA agents on-premises
-
Broad API support across the ecosystem members
-
No code/low code modifications required to Docker Compose YAML file for adding new tools to OPEA agents
Building OPEA AI Agent for Q&A on Intel® Hardware
The webinar speakers discuss developing a question-answering AI agent (AgentQnA) on OPEA using Intel Gaudi AI accelerators and Intel Xeon Processors.
OPEA microservices serve as the building blocks for seamless development of scalable and flexible enterprise-grade AI applications on the platform. They are modular, containerized components that enable cloud-native deployment of the workloads. Each microservice performs a specific task within the application architecture. Some of the key microservices OPEA provides include embedding, reranking, LLM, retrieval, text-to-speech (TTS)/audio-speech-recognition (ASR), and more.
In the following video, Guy talks about the OPEA microservices concept in brief:
For more details about the OPEA microservices, refer to the GenAIComps repository in the OPEA documentation.
From [00:25:35] in the webinar recording, Louie discusses the microservices required for building AgentQnA (a multi-agent application with question-answering capabilities) using OPEA. This is followed by a demo of the application UI and how to customize the application easily by adding tools like web search. Learn more about the OPEA AgentQnA example on GitHub.
Another interesting GitHub example built on OPEA is the ChatQnA, a RAG and LLM-based chatbot application. Check out the following demo video on YouTube where Guy walks you through the ChatQnA example deployed on an Intel Xeon Processor:
Explore more GenAI use-case applications in the OPEA project repo on GitHub.
Agent Microservice: The Brain of LLMs on OPEA
The OPEA Agent Microservice, built on LangChain* or LangGraph* frameworks, integrates the reasoning capabilities of LLMs with actionable steps. It enables creating a sophisticated system that can understand and process information, evaluate and track situations, take necessary actions, and communicate responses. This modular approach allows you to independently develop, deploy, and scale individual components of the application, promoting flexibility and scalability.
The Agent Microservice supports four types of agents as follows:
-
ReAct agents that engage in “reason-act-observe” cycles to solve problems
-
RAG agents that enhance RAG’s performance by rephrasing the query, checking relevancy of the context, and iterating if the context is not relevant
-
Plan and execute agents that create and execute step-by-step plans
-
SQL agents optimized for answering questions about the data in SQL databases
Refer to the webinar recording from [00:36:35] where Alex talks about the Agent Microservice.
Empower Your AI Agents for Accelerated GenAI!
Get started with OPEA – develop efficient AI agents in a transparent environment; let them accelerate and simplify GenAI tasks for you! Be it a chatbot, code generation and translation, document summarization, audio/video, or textual question-answering – the open-source OPEA project has comprehensive guides for building these and several other GenAI applications in the OPEA tutorial. Dive deeper into the GenAI components, OPEA infrastructure and performance evaluation and benchmarking feature of the platform.
We encourage you to explore our GenAI developer resources and a wider collection of AI tools and framework optimizations.
Useful Resources