John Jayasankar

JJ-SYS-02Production AI · MCP3 min read

Incident agent that returned 750+ engineering hours a year

When something broke in production, ops still paged senior engineers by default. The agent gathers domain evidence first, sharpens the diagnosis, and escalates only when the evidence is incomplete.

  • Quantile · LSEG
  • Production
  • 2025
  • Lead Product Manager

Incident agent · production support

Incident opens · default path still pages engineering

4.5h  to 11mexpert triage  to ops cycle

Schematic, not the production interface · 750+ senior eng hours / year returned · escalate only when evidence is incomplete

Phases1-3 · select to hold

Reading the schematic interfaceAn incident that pages engineering by default, then run and ops evidence gathered through two MCP servers with six years of support knowledge, then a complete packet resolved in operations in 11 minutes, with escalations down 78%.

What was broken

Institutional production support depended on deep system knowledge accumulated over years. Investigations were slow and routinely escalated from operations to senior engineering.

Product decision

Build for diagnosis. The agent retrieves evidence through two domain-specific MCP servers, applies six years of production-support knowledge, and sharpens the diagnosis before anyone pages an expert.

Flow

  1. 01Before4.5h investigation

    Deep knowledge lived with a small set of experts.

  2. 02DecisionMCP-backed evidence triage

    Retrieve domain evidence, then escalate only if needed.

  3. 03After11 minutes

    Fewer escalations, engineering time returned.

What I owned

  • Framed the product as diagnosis, not chat: success meant fewer engineering pages, not more answers
  • Defined the two MCP domain boundaries and the escalation rule when evidence is incomplete
  • Tied launch criteria to investigation time and escalation rate in production

System pattern

  • Connects the agent to curated operational context through domain-specific MCP
  • Surfaces evidence relevant to the incident instead of generic answers
  • Structures triage so issues can be resolved or sharply scoped before escalation
  • Uses investigation time and escalation rate as production success criteria

Role & tags

Lead Product Manager. Led development and launch, shaped domain boundaries and support workflows, and focused the product on actionable diagnosis, not generic Q&A.

  • Domain MCP
  • Evidence first
  • Escalation boundary
  • Expert time as KPI

Impact

Expert investigation fell from 4.5 hours to 11 minutes. Escalations to engineering dropped 78%. 750+ senior engineering hours a year went back to product development.

1-7 beats · j / k · ← → adjacent · y link · b brief