Risk Management System, Enterprise-grade AI risk analysis on a microservices stack

Project Overview

We built a risk analysis platform that lets analysts ask questions in plain English and get answers grounded in the client's own policy and audit corpus. The system runs as a set of services, with specialized AI agents handling parsing, retrieval, and synthesis, and a vector database providing semantic search.

Client Background

The client runs an enterprise risk management function. Their team reviews policies, audits, and prior decisions to assess new cases. The corpus had grown across years of operation, and tribal knowledge sat with senior analysts. They needed a way for any analyst to surface the right precedent without paging a teammate.

The Problem

Analysts worked across thousands of documents spread over several systems. Finding a relevant prior decision meant searching each system on its own and reading long PDFs to confirm the match. Keyword search returned noise and missed cases where the wording differed. A plain LLM approach was tested earlier and produced confident answers that were not anchored to real documents, which is a non starter in a regulated context.

Challenges

• Large, mixed-format corpus including native PDFs, scanned audits, and structured policy files.
• Strict grounding requirement, with every claim traceable back to a source document.
• Multiple internal teams expected to use the system without re-indexing per group.
• Operational expectation that agents and retrieval can scale independently.
• Sensitive content, so the design had to support enterprise hosting and access boundaries.

Our Approach

We started with the question types analysts actually asked, then mapped each one to the data sources that should answer it. From there we picked a retrieval augmented generation pattern instead of fine-tuning, since the corpus changes and grounding matters more than style. We chose a small set of focused agents over one large prompt, so we could test and replace each step on its own. The platform was split into services from day one so retrieval, parsing, and synthesis could scale and fail independently.

What We Built

User Experience

A Next.js interface where analysts type a question and see an answer with citations next to it. Each citation links back to the exact section of the source document so the analyst can verify the claim quickly.

Automation

A multi-agent pipeline coordinated by Elysia. One agent parses incoming documents, another runs semantic retrieval, and a third synthesizes the final answer with citations. Each agent has a narrow job, which makes the system easier to debug and extend.

Integrations

Document ingestion handles PDFs, scanned audits, and structured policy files. Azure OpenAI provides the language model and embedding calls. Weaviate stores the vectors and serves semantic search. PostgreSQL holds metadata, run history, and references.

Performance

Each service is containerized with Docker and can scale on its own. Retrieval is decoupled from synthesis, so heavy ingestion does not slow analyst queries.

Security

The platform is built to fit inside an enterprise boundary, with the LLM and vector store reachable only through the FastAPI gateway. Source documents stay in the client's environment.

Technical Architecture

At the top is a Next.js frontend that talks to a FastAPI gateway. The gateway forwards analyst requests to the Elysia orchestrator, which routes work to the right agent. The retrieval agent queries Weaviate for semantically similar passages and pulls metadata from PostgreSQL. The synthesis agent calls Azure OpenAI with the retrieved context and returns a grounded answer with citations. Everything runs in Docker so each service can be updated on its own.

Key Decisions

• RAG over fine-tuning. We chose retrieval augmented generation so the system stays grounded in the live corpus and updates as documents change. We gave up the stylistic control a fine-tune would offer.
• Weaviate as the vector store. It gave us hybrid search and a clean operational story. We gave up some of the tight Postgres-only simplicity we would have had with pgvector.
• Elysia for agent orchestration. An open source agent framework let us define focused agents and swap them without rewriting the host service. We accepted a newer dependency in exchange for cleaner separation.
• Microservices from the start. Splitting parsing, retrieval, and synthesis added deployment complexity, but it made scaling and incident response much easier later.
• Azure OpenAI for the LLM and embeddings. It fit the client's existing cloud posture and procurement path. We gave up the option to mix providers without extra abstraction.

Results

Analysts can ask questions in natural language and receive answers tied back to the source documents. Searches that used to require visiting several systems now run in one place. Onboarding a new analyst is easier because the corpus surfaces itself through the assistant rather than living in senior staff's heads. The architecture lets the team add new document sets without rebuilding the platform.

Project Highlights

• Grounded answers with per-claim citations into the source corpus.
• Multi-agent design with separate parsing, retrieval, and synthesis agents.
• Vector-based semantic search across mixed-format policy and audit content.
• Microservices layout that scales retrieval and synthesis independently.
• Enterprise-friendly stack with FastAPI, PostgreSQL, Next.js, and Docker.

Screens and User Flows

The core flow is an analyst asking a question and receiving an answer with citations. The orchestrator delegates retrieval and synthesis, and the UI shows the answer next to the source links.

Future Growth Opportunities

• Add new agents for specific risk domains (sanctions screening, vendor risk, internal audit) without touching the gateway.
• Plug additional document sources into the ingestion pipeline as the corpus grows.
• Layer evaluation tooling on top of the existing run history in PostgreSQL to track grounding quality over time.
• Expose a read-only API so other internal tools can call the same grounded retrieval.
• Add role-based views so different teams see only the documents they are allowed to search.