ServiceNow
Solutions
Architect

The challenge wasn't connecting ServiceNow to an LLM — any developer can write an API call. The challenge was doing it in a way that's secure, resilient, and scales to enterprise ticket volumes without degrading the agent experience.

Scripted REST APIs give fine-grained control over request construction, payload shaping, and response parsing. The key design choice was making all LLM calls asynchronous — the user never waits on an external API response. The UI stays responsive while the heavy LLM processing happens in the background.

Security designed in from day one, not added later. All API credentials live in the ServiceNow Vault — never in script includes, never in system properties. OAuth 2.0 handles the authentication handshake. Credentials rotate without touching code, with a clean audit trail of every external call.

External LLMs have rate limits and occasionally time out. Most implementations handle this with a simple retry loop, which can cascade and make things worse. The Circuit Breaker pattern: after a threshold of failures, the circuit opens, stops hammering the external API, waits for a recovery window, then tests with a single request before resuming full traffic. Graceful degradation instead of falling over.

MID Servers handle the heavy lifting for data-intensive payloads — CMDB context, CI relationships, discovery data for grounding the LLM prompt. Running that processing locally keeps latency low and avoids pushing sensitive infrastructure data to the cloud unnecessarily. Scales horizontally.

The orchestration layer abstracts which model is called. If one provider has an outage, traffic routes to the other without changing application logic.

Prompt construction is where most GenAI implementations fail quietly. Prompts are built using structured context from CSDM — incident description, relevant CI, upstream/downstream relationships, historical resolution patterns. That grounding separates a useful AI response from a hallucinated one.

Instrument the Circuit Breaker telemetry earlier. Latency and failure-rate dashboards added after go-live — having that visibility from day one would have shortened the tuning cycle significantly.

The average service desk agent spends a significant portion of their day doing the same four things: reading a ticket, figuring out what it's about, deciding who owns it, and assigning a priority. None of that requires human judgment 95% of the time.

So — what if we automated the entire triage decision, and only involved a human when the AI wasn't confident enough to act alone?

The first layer is a custom text-parsing engine that extracts entities from Short Description and Brief Description fields — application names, infrastructure components, error codes, service references. That structured output becomes the input to the LLM prompt. The quality of that extraction directly determines the quality of everything downstream.

Once entities are extracted, we cross-match them against the Core Data Dictionary and CI Class models to identify what application or infrastructure the ticket is actually about. The critical detail: we route the match through IRE — the Identification and Reconciliation Engine. Rather than writing a CI name and hoping it's correct, IRE validates the match against existing CMDB records without creating duplicates. It's the difference between an AI that guesses and one that maintains data integrity.

Every triage action produces a confidence score. Above the threshold, the system acts autonomously — Priority, Category, Assignment Group are set, initial troubleshooting steps generated, and the ticket moves forward untouched. Below the threshold, it surfaces for manual review. That gate is what lets us say the system is 100% accurate — accuracy is defined as the AI only acts when it's certain, and defers when it isn't.

30% reduction in average triage time. More interesting: roughly 95% of tickets clear the confidence threshold and are triaged fully automatically. The 5% that go to human review are genuinely the ambiguous ones — agent time is now spent where human judgment actually adds value.

When an agent overrides the AI's triage decision, that correction is captured and fed back into the confidence model tuning. The system gets smarter from every exception.

The same architectural pattern I used for incident triage — LLM extraction, IRE validation, confidence gating — I applied directly to CMDB health.

The CI Relationship Auditor solves a problem that's invisible until it's catastrophic: missing upstream and downstream CI dependencies in the CMDB.

Discovery logs are unstructured. When a MID Server scans your environment, it produces raw text output that contains references to applications, servers, databases, and their interactions — but none of that is automatically structured into CMDB relationships. We were losing insight into dependencies that mattered for impact analysis, change risk scoring, and incident routing.

An LLM-driven parsing engine reads unstructured discovery logs and extracts entity references and implied relationships. The model infers upstream and downstream dependencies from log patterns — for example, recognizing that an application server repeatedly calling a specific database endpoint implies a relationship that should exist in the CMDB.

Extracted relationships don't go directly to the CMDB. They route through the Identification and Reconciliation Engine first. IRE validates that the suggested CI actually exists, checks for existing relationships, and prevents duplicates. Without IRE, an LLM-driven write operation would corrupt the CMDB over time.

Same pattern as Zero-Touch Triage: high-confidence suggestions are written automatically, low-confidence surface to an architect for review. The architect sees the log evidence, the suggested relationship, and the confidence score — not just a raw AI output.

Measurable improvement in relationship completeness on the CMDB health dashboard. It also improved the quality of everything that depends on CMDB accuracy — change risk scoring, impact analysis for major incidents, and the Zero-Touch Triage CI assignment. These two AI features reinforce each other.

IRE is non-negotiable. The LLM suggests — IRE validates. We never let an AI write directly to the CMDB without a reconciliation check. The confidence gate adds a second layer: ambiguous suggestions never auto-commit.

Sits alongside my ServiceNow work at a client, showing a different dimension of enterprise AI — customer-facing conversational AI rather than internal IT automation. The Aisera chatbot was deployed on the client's support portal, serving diabetic patients, caregivers, and healthcare professionals who use blood glucose monitoring products.

The quality of the chatbot's responses is entirely determined by the quality of the KB structure. You cannot train a good conversational AI on poorly organised, inconsistently formatted, or outdated Confluence pages.

My specific responsibility was the knowledge base architecture — designing how Confluence content was structured, ingested, and used to train the Aisera model. That sounds straightforward, but it was the hardest part of the implementation.

I built a KB taxonomy in Confluence that separated content into four structured types: product troubleshooting guides, FAQs, standard operating procedures, and regulatory-compliant device guides. Each type had a defined template with consistent headings, metadata tags, and version control. That structure is what allowed Aisera's ingestion engine to parse content cleanly and assign it to the right intent categories.

The chatbot integrates with three backend systems — Salesforce for customer account and case management, the ERP for order status and returns, and a Complaint Handling System for regulatory-grade complaint intake. The complaint handling integration was the most sensitive: the client sells medical devices, so any product complaint has regulatory implications under FDA and GxP frameworks. The routing logic had to distinguish between a general support question and a formal product complaint, and handle the latter with a completely different workflow.

We established a KB governance process: every Confluence page used as a training source had to pass a structured review — correct template, current version, approved by a subject matter expert, and tagged with the correct product and intent category. Pages that failed the review were excluded from ingestion until remediated. This created a feedback loop — Aisera's response quality drove KB improvement, which drove better responses.

Complaint handling for a medical device company is regulated. A customer saying "my meter gave a wrong reading" is not just a support issue — it may be a reportable adverse event. The chatbot's complaint intake flow was designed with the regulatory team to capture the structured data required for FDA reporting: product serial number, lot number, incident description, patient impact. That structured intake feeds directly into the Complaint Handling System rather than a generic case.