Architecting JARVIS: A technical deep dive into its Multi-Agent System (MAS) design

June 2025 Update: JARVIS, the multi-agent system also known as the AI Platform Engineering, has been open-sourced as part of the CNOE Agentic AI Community.

We recently introduced JARVIS, our agentic approach to AI platform engineering at Outshift by Cisco, showcasing how agentic systems significantly boost productivity and reduce platform engineering toil. Building on that foundation, this blog explains JARVIS’s advanced Multi-Agent System (MAS) architecture, orchestrated using LangGraph and guided by Outshift's multi-agent taxonomy. 

Jarvis architecture implements the “Semantically Routing (the sandwich bar)” multi-agent system design patterns outlined in Recipes for Multi-Agent System Success. In this blog, we will show that JARVIS Supervisor Agent functions as a semantic router, dynamically interpreting user requests and delegating tasks to specialized Curated Agents (CAs) at runtime.

Each Curated Agent is tailored for specific domains such as Jira, PagerDuty, or GitHub, enabling efficient and modular task execution. Distributed agents are seamlessly connected to the Supervisor Agent using the AGNTCY Agent Connect Protocol, ensuring reliable communication and coordination across decentralized environments. 

At the core of each agent’s behavior is the Reason-and-Act (ReAct) paradigm, allowing agents to autonomously reason through user intents, select appropriate tools, and execute complex engineering workflows effectively.

What is a Multi-Agent System (MAS)?

A Multi‑Agent System (MAS) is an agentic AI architecture in which multiple autonomous, domain‑specialized agents independently reason, act, and collaborate — coordinating through communication and task specialization — to solve complex problems that would be difficult or inefficient for a single agent to handle alone.

Inside the JARVIS architecture

JARVIS leverages LangGraph to implement the hierarchical supervisor multi-agent system, orchestrating structured workflows across distributed remote agents. These agents are seamlessly connected using the standardized Agent Connect Protocol from AGNTCY, enabling robust communication and coordination across decentralized environments. This architecture streamlines agent collaboration, decision-making, and execution in a scalable, modular, and efficient way.

Reflection agent

The reflection agent serves as the decision-making layer in the multi-agent system architecture. It determines whether the system has sufficiently addressed a user’s request, or if further steps are needed. This decision-making is driven by the “LLM-as-a-judge” pattern, evaluating the quality and completeness of each response.

After every interaction:

• If the result is incomplete or ambiguous, the reflection agent routes the query back to the hierarchical supervisor agent to continue processing.
• If more input is required from the user, it returns the input request back to user by ending the interaction.
• If the output is satisfactory, it finalizes the conversation with the user.

Example user query in action

Consider the user query:
“Who is on SRE on-call and find the JIRA tickets they worked on in the last seven days in OPENSD project?”

1. The hierarchical supervisor agent picks "PagerDuty Agent" as the “next_agent” to solve the task.
2. Upon receiving the on-call SRE details, the reflection agent evaluates whether the full query has been addressed.
3. Realizing the Jira portion is still pending, it returns the query back to the hierarchical supervisor agent.
4. The supervisor agent then routes it to the Jira agent, completing the full chain of execution
5. Upon further reflection, the reflection agent returning the complete response back to the user.

Agent trajectory evaluations

One of JARVIS’s core goals is to deliver a highly deterministic platform engineering experience, minimizing unpredictability in how tasks are handled. However, agentic systems inherently grant large language models (LLMs) autonomy over decision flow to solve complex problems. While this flexibility is powerful, it introduces variability in reasoning paths making it difficult to anticipate how one step influences the next.

To address this, JARVIS uses agentevals to analyze agent trajectories, measure the predictability and consistency of those reasoning patterns across large-scale prompt datasets and between various LLMs. This approach provides critical insight into multi-agent behavior, surfacing areas for refinement and helping ensure reliable, reproducible outcomes in automated workflows.

Example trajectory dataset:

pd_jira_combo_query:
    input: |
      who is on sre oncall and find their latest jiras?
    reference_trajectory:
      solution_1: __start__;supervisor_agent;pagerduty_agent;pagerduty_tools;pagerduty_agent;reflection_agent;supervisor_agent;jira_agent;jira_tools;jira_agent;reflection_agent;__end__
    metadata:
      comments: |
        PagerDuty and Jira combo questions

The future of agentic AI in platform engineering

The JARVIS system exemplifies the potential of multi-agent systems (MAS) in transforming platform engineering combining supervisor agents that manage decision flow, sub-agents that handle specialized tasks connected via AGNTCY Agent Connect Protocol, and reflection agents that evaluate outputs using the LLM-as-a-judge paradigm

We are at the forefront of integrating agentic AI into platform engineering, building an ecosystem where AI agents amplify human potential, enhance collaboration, and accelerate innovation. Stay tuned as we continue to push the boundaries of AI-powered platform engineering.

Interested in learning more? Explore more from Outshift's platform engineering team.