Agentic Banking Assistant — Detailed Production Architecture (Azure, GCP & AWS)
Source diagram: Multi-agent GenAI chat platform for a bank — Coordinator Agent orchestrating Accounts / Transaction / Service agents over MCP servers, fronted by an edge/API layer, with observability, cost tracking, PII redaction, agent evaluation, and a session store.
This document maps every box in the diagram to concrete, current (2026) Azure and AWS resources, plus the open-source/framework layer (LangGraph, MCP, evaluation, Neo4j) you'd wire underneath either cloud. Where a component is genuinely cloud-native (e.g., the bank's own IdP, Azure/AWS aren't interchangeable at the same layer), that's called out.
1. Layer-by-Layer Component Map
| # | Diagram component | Function |
|---|---|---|
| 1 | User Interface (chat) | Web/mobile chat client for bank customers or staff |
| 2 | Edge Layer (WAF, DDoS, Rate Limits, API Gateway) | Perimeter defense + traffic shaping |
| 3 | Authentication (bank's identity provider) | Federates to the bank's existing IdP (Entra ID, Ping, ForgeRock, etc.) |
| 4 | API | Backend-for-frontend / orchestration entry point |
| 5 | Authorization | Fine-grained access decisions (who can ask what, which agent can touch which account) |
| 6 | Coordinator Agent | Top-level orchestrator/router agent |
| 7 | Accounts / Transaction / Service Agents | Domain-specialized sub-agents |
| 8 | Accounts / Transactions / Service MCP Servers | Tool-serving layer exposing core banking capabilities via MCP |
| 9 | Downstream capabilities (Balance Enquiry, Transaction Details, Statement Request, Change of Address, Cheque Book, KYC update) | Actual core-banking operations invoked as MCP tools |
| 10 | PII Redaction | Strips/masks sensitive data before it reaches any LLM |
| 11 | Self-Hosted LLM / Third-Party LLM | Inference layer — open-weight model you host + a frontier model API |
| 12 | Agent Evaluation Suite | Offline/online eval, regression testing of agent behavior |
| 13 | Observability (prompts, agent calls, tool calls, CPU/memory/disk) | Full-stack tracing + infra metrics |
| 14 | Cost Tracker | Per-agent, per-tenant token/compute cost attribution |
| 15 | Session Store (conversation history, inter-agent shared state) | Durable + low-latency state layer |
2. Azure Reference Architecture
| Component | Primary Azure Resource | Supporting Resources | Notes |
|---|---|---|---|
| Chat UI | Azure Static Web Apps / App Service (Node/React) | Azure Front Door (CDN) | SSR or SPA served via Front Door origin |
| Edge Layer | Azure Front Door Premium (WAF + DDoS + CDN) | Azure DDoS Protection (Network layer), Azure API Management (APIM) Premium v2 as the AI Gateway | Front Door WAF handles L7/bot/geo-filtering; APIM enforces token-rate-limit policies, quotas, semantic caching, and request/response transformation in front of the LLM and MCP endpoints |
| Authentication | Microsoft Entra ID (federated to the bank's IdP via SAML/OIDC), Entra External ID if customer-facing | Entra Conditional Access, Entra ID Protection | Diagram explicitly shows "bank's identity provider" — Entra acts as the broker/relying party, doing OIDC federation into whatever IdP the bank already runs |
| API | Azure API Management (same APIM instance, /api product) or Azure Container Apps hosting a FastAPI BFF |
Azure Functions for lightweight routes | APIM is both edge gateway and AI gateway here — one control point |
| Authorization | Microsoft Entra ID + Azure RBAC, plus fine-grained policy via Azure API Management "validate-jwt" + custom policies, or externalize to OPA/Cedar sidecar on AKS | Entra ID Governance (entitlement management) | For agent-to-agent delegation and scoped tool access, layer SPIFFE/SPIRE-issued workload identities or Entra Workload ID on top so each agent/MCP server has its own verifiable identity, not just a shared service principal |
| Coordinator + domain Agents | Microsoft Foundry Agent Service (formerly Azure AI Foundry Agent Service) hosting agents defined via the Microsoft Agent Framework, or self-hosted LangGraph agents on Azure Kubernetes Service (AKS) / Azure Container Apps | Foundry supports "bring your own framework" (LangGraph, Semantic Kernel, AutoGen, CrewAI) if you don't want to rewrite LangGraph agents | Given your existing LangGraph stack, the pragmatic path is: keep LangGraph as the orchestration code, deploy it on AKS, and register it with Foundry's control plane for governance/observability rather than porting to Foundry's native agent authoring |
| MCP Servers (Accounts/Transactions/Service) | AKS-hosted MCP servers (containerized, one deployment per domain) exposed internally, or registered in Foundry's 1,400+ tool catalog as MCP-enabled tools | Azure API Management can front the MCP endpoints too (MCP-aware policies are shipping in APIM's AI Gateway) | Each MCP server should carry its own workload identity (Entra Workload ID / SPIFFE SVID) so the Authorization layer can scope tool calls per agent |
| Core banking downstream calls | Azure Logic Apps / Azure Functions as adapters to mainframe or core banking APIs | Azure Integration Services (Service Bus, Event Grid) for async ops (statement generation, KYC workflow) | These are the actual "Balance Enquiry / Statement Request / KYC update" leaf nodes |
| PII Redaction | Azure AI Language — PII Detection & Document PII Redaction | Deployed as an APIM inbound policy step ("mask PII before it reaches the model") using send-request to call the Language service, splicing redacted text back into the prompt |
This matches the diagram's placement — redaction sits between API and the LLM layer, enforced centrally at the gateway, not per-app |
| Self-hosted LLM | Azure Machine Learning Managed Online Endpoints or AKS + NVIDIA Triton/vLLM for open-weight models (Llama, Mistral, Phi) | Azure Container Registry, GPU node pools (NC/ND-series) | Foundry Local can also run multimodal open models on-prem/edge with zero cloud connectivity if data residency demands it |
| Third-party LLM | Microsoft Foundry Models catalog (Azure OpenAI GPT models, Anthropic Claude on Foundry, Meta Llama, Mistral) | All routed through the APIM AI Gateway for a single metering/policy point | Claude and other frontier models are now available directly inside Foundry with Azure governance wrapped around them |
| Agent Evaluation Suite | Azure AI Foundry Evaluation (built-in agent eval: groundedness, relevance, safety, task adherence) | Complement with open-source Ragas / DeepEval / promptfoo run as an AKS CronJob against a golden dataset in Blob Storage | Foundry evaluation integrates directly with Agent Service traces |
| Observability | Azure Monitor + Application Insights (distributed tracing for prompts/agent/tool calls via OpenTelemetry), Azure Managed Grafana for dashboards | Log Analytics workspace, Container Insights for AKS CPU/memory/disk | Foundry Agent Service emits OTel traces natively into App Insights |
| Cost Tracker | Microsoft Cost Management + Billing (tag-based cost allocation per agent/tenant) | Custom token-cost dashboard in Grafana/Power BI fed from APIM's token-metering policies + Foundry usage logs | APIM's llm-token-limit / azure-openai-token-limit policies emit per-call token counts you can attribute to a tenant/agent tag |
| Session Store | Azure Cosmos DB (conversation history, multi-region, low-latency) + Azure Cache for Redis (hot inter-agent shared state / short-term memory) | Cosmos DB change feed can drive downstream analytics | Matches your pattern of Cosmos/Redis for durable + ephemeral state |
| Policy/entitlement graph (cross-cutting) | Neo4j Aura on Azure Marketplace | — | Fits your existing Neo4j Graph RAG and identity-platform-agents work — model agent delegation chains, entitlements, and tool-scoping as a graph |
3. AWS Reference Architecture
| Component | Primary AWS Resource | Supporting Resources | Notes |
|---|---|---|---|
| Chat UI | Amazon CloudFront + S3 (static SPA) or AWS Amplify Hosting | — | |
| Edge Layer | AWS Shield Advanced (DDoS) + AWS WAF on CloudFront/ALB | Amazon API Gateway (REST/HTTP API) for rate limiting, throttling, usage plans | Shield handles volumetric/L3-L4, WAF handles L7 rules/bot control, API Gateway does per-key throttling and quotas |
| Authentication | Federation into the bank's identity provider via Amazon Cognito (as OIDC/SAML broker) or AWS IAM Identity Center for workforce-facing use | Cognito User/Identity Pools | Cognito plays the same "broker" role Entra plays on Azure — federates to whatever IdP the bank runs |
| API | Amazon API Gateway → AWS Lambda (BFF) or containerized FastAPI on Amazon ECS/Fargate | AWS AppSync if GraphQL is preferred | |
| Authorization | AWS IAM + Amazon Verified Permissions (Cedar policy engine) for fine-grained, per-resource authorization | Cognito authorizers on API Gateway | Bedrock AgentCore's own Policy component is Cedar-based — using Verified Permissions keeps your policy language consistent from edge to agent-to-tool calls. Layer SPIFFE/SPIRE or AgentCore Identity for per-agent workload identity |
| Coordinator + domain Agents | Amazon Bedrock AgentCore Runtime hosting your LangGraph (or Strands Agents) coordinator + sub-agents as serverless, auto-scaling microVM-isolated endpoints | AgentCore natively supports LangGraph, CrewAI, Strands, and raw Python agents — no rewrite needed | Each user session runs in an isolated microVM; this is a clean fit for a regulated banking workload needing per-session isolation |
| MCP Servers (Accounts/Transactions/Service) | Amazon Bedrock AgentCore Gateway — converts existing APIs/Lambda functions into MCP-compatible tools with a single managed MCP endpoint | AgentCore Runtime can also directly host stateful MCP servers (elicitation, sampling, progress notifications) | Gateway also does semantic tool search, returning only 10–15 relevant tools instead of dumping your whole API surface into the agent's context — useful once you have Balance/Statement/KYC/Address/Cheque-book as dozens of underlying operations |
| Core banking downstream calls | AWS Lambda adapters to core banking / mainframe APIs, exposed as Gateway tools | Amazon EventBridge / SQS for async KYC/statement workflows | |
| PII Redaction | Amazon Comprehend — PII detection & redaction | Invoked from API Gateway (via Lambda authorizer/integration) or from the AgentCore Gateway as a pre-processing tool step before any prompt hits Bedrock | Same "chokepoint" pattern as Azure: mask centrally, once, before the model sees the payload |
| Self-hosted LLM | Amazon SageMaker Endpoints (real-time inference) or Amazon EKS + vLLM/Triton on G5/P5 GPU instances for open-weight models | ECR for container images | |
| Third-party LLM | Amazon Bedrock (Anthropic Claude, Meta Llama, Mistral, Amazon Nova, and others) via a single unified API | Bedrock Guardrails for content safety at the model layer | Bedrock is the direct analog of Foundry Models — one managed multi-provider model catalog |
| Agent Evaluation Suite | Bedrock AgentCore Evaluation / Bedrock Agent Evaluation (RAGAS-based) | Open-source Ragas/DeepEval as scheduled Fargate tasks against a golden set in S3 | |
| Observability | AWS X-Ray (distributed tracing) + Amazon CloudWatch (logs/metrics/dashboards) | AgentCore emits OpenTelemetry traces to X-Ray/CloudWatch with minimal instrumentation | Container Insights on ECS/EKS for CPU/memory/disk |
| Cost Tracker | AWS Cost Explorer + Cost Allocation Tags (per-agent, per-tenant tagging via AgentCore Runtime/Gateway tags) | Custom QuickSight dashboard fed from Bedrock's per-invocation token usage in CloudWatch | |
| Session Store | Amazon DynamoDB (conversation history, single-digit-ms latency, global tables for multi-region) + Amazon ElastiCache (Redis/Valkey) for hot inter-agent state | AgentCore Memory (built-in short-term session + long-term semantic memory) can replace a chunk of this custom wiring | AgentCore Memory is purpose-built for exactly this box in the diagram — worth evaluating before building your own Dynamo/Redis layer |
| Policy/entitlement graph (cross-cutting) | Neo4j Aura on AWS Marketplace | — | Same graph-based entitlement/delegation model as the Azure side |
4. Cross-Cutting Concerns (apply to both clouds)
Agent identity, not just user identity. The diagram's Authentication box covers the human/session identity; in a multi-agent system you also need workload identity for the Coordinator, each sub-agent, and each MCP server, so Authorization can answer "can this agent call this tool on behalf of this user." This lines up directly with your identity-platform-agents work — SPIFFE/SPIRE for cryptographic workload identity, IETF AIMS-style agent identity metadata, and OAuth 2.1 token exchange (on-behalf-of) for delegation chains. AWS AgentCore Identity and Azure Entra Workload ID both give you a managed starting point; SPIFFE/SPIRE sits underneath either as the portable, cloud-neutral root of trust if multi-cloud/hybrid is a long-term goal.
PII redaction placement. Both clouds push you toward the same pattern the diagram shows: redact centrally at the gateway (APIM policy on Azure, API Gateway/AgentCore Gateway step on AWS), not inside each agent. This gives one auditable chokepoint for regulators.
Evaluation as a gate, not just a dashboard. Wire the Agent Evaluation Suite into CI/CD for agent prompt/tool changes — both Foundry Evaluation and Bedrock AgentCore Evaluation support automated regression runs; block deploys on groundedness/safety score drops given the KYC/account-data surface here.
Data residency & compliance. For a bank, confirm region pinning for the Session Store, PII redaction service, and any self-hosted LLM (RBI/data-localization rules if this is an India-domiciled bank, given your India context) — Azure Front Door/Cosmos DB and AWS's India regions (Mumbai/Hyderabad) both support in-country residency; Foundry Local / on-prem vLLM are your fallback if a model must never leave the data center.
Cost tracker granularity. Tag every agent invocation and MCP tool call with tenant_id + agent_name + conversation_id at the gateway layer (APIM policy / AgentCore Gateway tag) — retrofitting this after launch is painful; both cost tools above are only as good as the tags you emit.
5. Suggested Deployment Topology (either cloud)
Internet
│
▼
Edge (WAF + DDoS + CDN)
│
▼
AI Gateway (rate limits, PII redaction, token metering) ──► Cost Tracker
│
▼
BFF / API ──► Auth (bank IdP federation) + AuthZ (Cedar/OPA + workload identity)
│
▼
Coordinator Agent (LangGraph on AKS / AgentCore Runtime)
│
├─► Accounts Agent ──► Accounts MCP Server ──► Core Banking (Balance)
├─► Transaction Agent ──► Transactions MCP Server ──► Core Banking (Txn Details, Statement)
└─► Service Agent ──► Service MCP Server ──► Core Banking (Address, Cheque Book, KYC)
│
├─► LLM layer (self-hosted open-weight + Bedrock/Foundry third-party models)
├─► Session Store (Cosmos+Redis / DynamoDB+ElastiCache or AgentCore Memory)
└─► Observability (App Insights+Grafana / X-Ray+CloudWatch) + Agent Evaluation Suite
6. Framework/Tooling Layer (cloud-agnostic, fits your existing stack)
- Orchestration: LangGraph (already your stack) — deploy as containers on AKS or AgentCore Runtime; both platforms explicitly support bringing your own LangGraph code rather than rewriting into Foundry's or Bedrock's native agent-authoring format.
- Tool serving: MCP servers per domain (Accounts/Transactions/Service), registered either in Foundry's tool catalog or behind AgentCore Gateway's semantic search — both reduce "tool sprawl" once KYC/address/cheque-book/statement operations multiply.
- Graph layer: Neo4j Aura (Azure or AWS Marketplace) for entitlement/delegation graphs and Graph RAG, consistent with rdb2graph and RegulatoryRisk GraphAgent patterns.
- Evaluation: Ragas/DeepEval/promptfoo as the portable open-source layer, with Foundry Evaluation or Bedrock AgentCore Evaluation as the managed complement.
- Identity: SPIFFE/SPIRE as the cloud-neutral workload-identity root, with Entra Workload ID or AgentCore Identity as the cloud-native token issuers on top.
Azure
both images generated by Anthropic claude by prompting.
