Architecture

How CortexDB stores, processes, and retrieves knowledge.

High-Level Overview

                    ┌──────────────────────────────────────────────┐
                    │              CortexDB Server                 │
                    │          (Spring Boot 3.5 + Java 21)         │
                    │                                              │
  SDK / curl ──────▶│  /api/setup                   → SetupCtrl    │
                    │  /api/v1/memory/ingest/*      → IngestCtrl   │
                    │  /api/v1/memory/query/*       → QueryCtrl    │
                    │                                              │
                    │  ┌──────────┐  ┌─────────────┐               │
                    │  │ LLM      │  │ Dual Pplns  │               │
                    │  │ Provider │  │ (Prompt &   │               │
                    │  │ (chat +  │  │  Document)  │               │
                    │  │ embed)   │  │             │               │
                    │  └──────────┘  └─────────────┘               │
                    └───────────────────┬──────────────────────────┘
                                        │
                    ┌───────────────────▼──────────────────────────┐
                    │       PostgreSQL 16 + pgvector               │
                    │                                              │
                    │  knowledge_bases ──┐                         │
                    │  contexts ─────────┤ (vector + JSONB)        │
                    │  entities ─────────┤                         │
                    │  entity_context_junction                     │
                    │  relations ────────┘ (weighted graph)        │
                    │                                              │
                    │  Flyway migrations │ NOTIFY triggers         │
                    └──────────────────────────────────────────────┘

The system has three layers:

REST API layer — 3 controllers handling setup, ingestion, and 20+ query endpoints
Processing layer — Async workers for chunking, embedding, entity extraction, and graph building
Storage layer — PostgreSQL with pgvector for vectors and JSONB for metadata

Database Schema

CortexDB uses 5 interconnected tables. All tables use UUID primary keys, JSONB metadata, and created_at timestamps.

knowledge_bases

The entry point — stores the raw user input.

Column	Type	Description
`id`	UUID (PK)	Auto-generated ID
`uid`	VARCHAR	External user identifier
`converser`	ENUM	`USER`, `AGENT`, or `SYSTEM`
`content`	TEXT	Full document/query text
`vector_embedding`	vector(768)	Embedding of the full content
`metadata`	JSONB	`{contentLength, embeddingDimensions, embeddingTimeMs}`
`created_at`	TIMESTAMPTZ	Insertion timestamp

contexts

Text chunks created from the knowledge base entry.

Column	Type	Description
`id`	UUID (PK)	Auto-generated ID
`kb_id`	UUID (FK)	References `knowledge_bases.id`
`text_chunk`	TEXT	The text chunk
`vector_embedding`	vector(768)	Embedding of the chunk
`chunk_index`	INT	Position in the original document
`metadata`	JSONB	`{chunkLength, embeddingDimensions, chunkNumber, totalChunks}`
`created_at`	TIMESTAMPTZ	Insertion timestamp

entities

Concepts, people, technologies, and other named entities extracted via LLM.

Column	Type	Description
`id`	UUID (PK)	Auto-generated ID
`entity_name`	VARCHAR	Name of the entity (indexed)
`entity_type`	VARCHAR	Type (e.g. Technology, Person, Concept)
`description`	TEXT	LLM-generated description
`vector_embedding`	vector(768)	Embedding of name + description
`metadata`	JSONB	`{extractedFrom, contextId, embeddingDimensions, descriptionLength}`
`created_at`	TIMESTAMPTZ	Insertion timestamp

💡 Deduplication:

Entities are upserted by name — if "Java" already exists, the existing entity is reused and linked to the new context.

entity_context_junction

Many-to-many join table linking entities to the contexts where they were mentioned.

Column	Type	Description
`entity_id`	UUID (FK)	References `entities.id`
`context_id`	UUID (FK)	References `contexts.id`

relations

Directed, weighted edges in the knowledge graph.

Column	Type	Description
`id`	UUID (PK)	Auto-generated ID
`source_entity_id`	UUID (FK)	Source entity
`target_entity_id`	UUID (FK)	Target entity
`relation_type`	VARCHAR	Relationship label (e.g. "uses", "manages")
`edge_weight`	INT	Frequency counter — increments on duplicate relations
`metadata`	JSONB	`{extractedFrom, contextId, edgeWeight}`
`created_at`	TIMESTAMPTZ	Insertion timestamp

ℹ️ Edge Weight:

When the same relation (same source, target, and type) is extracted from a different document, edge_weight increments by 1. Higher weight = stronger, more frequently observed connection.

Entity-Relationship Diagram

  knowledge_bases          contexts               entities
  ┌──────────────┐    ┌───────────────┐      ┌──────────────┐
  │ id (PK)      │    │ id (PK)       │      │ id (PK)      │
  │ uid          │    │ kb_id (FK) ───┼──────│ entity_name  │
  │ converser    │────│ text_chunk    │      │ entity_type  │
  │ content      │    │ vector_embed  │      │ description  │
  │ vector_embed │    │ chunk_index   │      │ vector_embed │
  │ metadata     │    │ metadata      │      │ metadata     │
  │ created_at   │    │ created_at    │      │ created_at   │
  └──────────────┘    └───────┬───────┘      └──────┬───────┘
                              │                     │
                    entity_context_junction          │
                    ┌─────────┴──────┐              │
                    │ entity_id (FK) ┼──────────────┘
                    │ context_id (FK)│
                    └────────────────┘
                                               relations
                                          ┌──────────────────┐
                                          │ id (PK)          │
                    entities.id ──────────│ source_entity_id │
                    entities.id ──────────│ target_entity_id │
                                          │ relation_type    │
                                          │ edge_weight      │
                                          │ metadata         │
                                          │ created_at       │
                                          └──────────────────┘

Ingestion Pipeline

CortexDB uses a fire-and-forget async architecture powered by PostgreSQL NOTIFY triggers.

Step-by-Step Flow

API receives input

IngestController routes input to either /prompt or /document endpoints. The IngestService generates an embedding for the full content, creates a KnowledgeBase entity with JSONB metadata, persists it, and begins processing based on the pipeline type.

Dual Pipeline Selection

Prompt (SimpleMem): Restates the prompt with LLM to extract meaning and timestamp context, then merges with highly similar existing contexts (Cosine Similarity > 0.85) to form a flowing continuous memory state.
Document (PageIndex): Generates a hierarchical table-of-contents via LLM, then recursively processes subsections via PageIndexService, establishing HAS_SUBSECTION relations.

PostgreSQL trigger fires

A database trigger on the knowledge_bases table fires NOTIFY rag_events with a JSON payload: {"type": "KB_CREATED", "id": "uuid", "content": "..."}

Listener dispatches to worker

PostgresNotificationListener runs on a dedicated daemon thread, listening on the rag_events channel. When it receives a KB_CREATED notification, it dispatches to IngestionWorker via Spring's @Async — fire-and-forget.

Worker chunks & embeds

IngestionWorker.processKnowledgeBase() splits content into chunks using ChunkingService, generates embeddings for each chunk, and persists them as Context entities with metadata.

Context triggers fire

Each persisted context triggers another NOTIFY rag_events with {"type": "CONTEXT_CREATED", ...}. The listener dispatches IngestionWorker.processContext().

Entity & relation extraction

processContext() calls ExtractionService (which uses the configured LLM) to extract entities and relationships from each text chunk. Entities are upserted (deduplicated by name), linked to contexts via the junction table, and relations are created with edge weight upsert logic.

Console logging

Every persisted row is logged with structured tags for observability:

KB_ROW       | id=... | uid=user-1 | content_length=90 | vector_dims=768
CONTEXT_ROW  | id=... | kb_id=... | chunk_index=0 | text_length=62
ENTITY_ROW   | id=... | name=Java | type=Technology | vector_dims=768
JUNCTION_ROW | entity_id=... | context_id=...
RELATION_NEW | id=... | source=Java | target=GC | type=uses | edge_weight=1

Query Pipeline & Agentic Router

CortexDB uses an Agentic Router to classify queries. The /route endpoint forwards the question to an LLM evaluator to determine the query intent (PROMPT or DOCUMENT). Based on intent, it leverages different traversal strategies.

🧭

Agentic Routing

Dynamically analyzes query intent. Conversational queries trigger Hybrid SimpleMem search, while analytical queries trigger PageIndex graph traversal.

🔍

Vector Search

Generates an embedding from the query text and uses pgvector's <=> cosine distance operator to find the most similar contexts, entities, or knowledge bases.

🕸️

Graph Traversal

Walks the entity-relation graph using 1-hop and 2-hop queries (recursive CTEs). Finds connections, related entities, and relationship patterns.

⚡

Hybrid Search

Combines vector similarity with graph context — first finds relevant entities via vector search, then enriches results with graph connections.

Tech Stack

Component	Technology	Purpose
Runtime	Java 21	Language & runtime
Framework	Spring Boot 3.5	Web framework, DI, config
AI	Spring AI 1.1	LLM provider abstraction
Database	PostgreSQL 16	Relational storage
Vectors	pgvector	Vector similarity search
ORM	Hibernate 6 + JPA	Object-relational mapping
Vectors ORM	hibernate-vector	Vector type support in Hibernate
Migrations	Flyway	Database schema versioning
Build	Maven 3.9	Build & dependency management
Containers	Docker Compose	Multi-container orchestration
Testing	Testcontainers + JUnit 5	Integration testing
API Docs	SpringDoc OpenAPI	Swagger UI at `/swagger-ui.html`