Published on 00/00/0000
Last updated on 00/00/0000
Published on 00/00/0000
Last updated on 00/00/0000
Share
Share
AI/ML
4 min read
Share
We recently introduced JARVIS, our agentic approach to AI platform engineering at Outshift by Cisco, showcasing how agentic systems significantly boost productivity and reduce platform engineering toil. Building on that foundation, this blog explains JARVIS’s advanced Multi-Agent System (MAS) architecture, orchestrated using LangGraph and guided by Outshift's multi-agent taxonomy.
Jarvis architecture implements the “Semantically Routing (the sandwich bar)” multi-agent system design patterns outlined in Recipes for Multi-Agent System Success. In this blog, we will show that JARVIS Supervisor Agent functions as a semantic router, dynamically interpreting user requests and delegating tasks to specialized Curated Agents (CAs) at runtime.
Each Curated Agent is tailored for specific domains such as Jira, PagerDuty, or GitHub, enabling efficient and modular task execution. Distributed agents are seamlessly connected to the Supervisor Agent using the AGNTCY Agent Connect Protocol, ensuring reliable communication and coordination across decentralized environments.
At the core of each agent’s behavior is the Reason-and-Act (ReAct) paradigm, allowing agents to autonomously reason through user intents, select appropriate tools, and execute complex engineering workflows effectively.
A Multi‑Agent System (MAS) is an agentic AI architecture in which multiple autonomous, domain‑specialized agents independently reason, act, and collaborate — coordinating through communication and task specialization — to solve complex problems that would be difficult or inefficient for a single agent to handle alone.
JARVIS leverages LangGraph to implement the hierarchical supervisor multi-agent system, orchestrating structured workflows across distributed remote agents. These agents are seamlessly connected using the standardized Agent Connect Protocol from AGNTCY, enabling robust communication and coordination across decentralized environments. This architecture streamlines agent collaboration, decision-making, and execution in a scalable, modular, and efficient way.
The reflection agent serves as the decision-making layer in the multi-agent system architecture. It determines whether the system has sufficiently addressed a user’s request, or if further steps are needed. This decision-making is driven by the “LLM-as-a-judge” pattern, evaluating the quality and completeness of each response.
After every interaction:
Consider the user query:
“Who is on SRE on-call and find the JIRA tickets they worked on in the last seven days in OPENSD project?”
One of JARVIS’s core goals is to deliver a highly deterministic platform engineering experience, minimizing unpredictability in how tasks are handled. However, agentic systems inherently grant large language models (LLMs) autonomy over decision flow to solve complex problems. While this flexibility is powerful, it introduces variability in reasoning paths making it difficult to anticipate how one step influences the next.
To address this, JARVIS uses agentevals to analyze agent trajectories, measure the predictability and consistency of those reasoning patterns across large-scale prompt datasets and between various LLMs. This approach provides critical insight into multi-agent behavior, surfacing areas for refinement and helping ensure reliable, reproducible outcomes in automated workflows.
Example trajectory dataset:
pd_jira_combo_query:
input: |
who is on sre oncall and find their latest jiras?
reference_trajectory:
solution_1: __start__;supervisor_agent;pagerduty_agent;pagerduty_tools;pagerduty_agent;reflection_agent;supervisor_agent;jira_agent;jira_tools;jira_agent;reflection_agent;__end__
metadata:
comments: |
PagerDuty and Jira combo questions
The JARVIS system exemplifies the potential of multi-agent systems (MAS) in transforming platform engineering combining supervisor agents that manage decision flow, sub-agents that handle specialized tasks connected via AGNTCY Agent Connect Protocol, and reflection agents that evaluate outputs using the LLM-as-a-judge paradigm
We are at the forefront of integrating agentic AI into platform engineering, building an ecosystem where AI agents amplify human potential, enhance collaboration, and accelerate innovation. Stay tuned as we continue to push the boundaries of AI-powered platform engineering.
Interested in learning more? Explore more from Outshift's platform engineering team.
Get emerging insights on innovative technology straight to your inbox.
Outshift is leading the way in building an open, interoperable, agent-first, quantum-safe infrastructure for the future of artificial intelligence.
* No email required
The Shift is Outshift’s exclusive newsletter.
Get the latest news and updates on agentic AI, quantum, next-gen infra, and other groundbreaking innovations shaping the future of technology straight to your inbox.