Knowledge Base API
The Knowledge Base Service provides a centralized repository for managing and accessing personal data through files and documents. This service enables document storage, retrieval, and management while integrating with the AI backend for intelligent search capabilities.Base URL
All endpoints are prefixed with/api/v1/kb
Authentication
All endpoints require authentication via Bearer token:Architecture Overview
The Knowledge Base Service is built on a Node.js backend with ArangoDB for graph-based data persistence. The service consists of several key components:- Record Management - Core document storage and metadata handling
- File Storage Integration - Connects with external storage services
- Event Broadcasting - Kafka-based events for system integration
- AI Indexing - Automatic content indexing for search capabilities
- Storage Service - Handles actual file storage and versioning
- AI Backend - Processes and indexes content for search
- Enterprise Search - Provides search capabilities across the knowledge base
- IAM Service - Handles user authentication and authorization
- Configuration Manager - Manages application settings
Data Models
Records
Records represent the core entities in the knowledge base:- Metadata about stored content (name, type, source)
- References to physical files in storage
- Versioning information
- Indexing status and history
File Records
File Records contain file-specific metadata:- File format information (extension, MIME type)
- Size and storage information
- Access URLs and paths
- Checksum and integrity information
Knowledge Base
Knowledge Base represents a collection of records:- Organizational grouping of records
- Permission structure
- Metadata about the collection
API Endpoints
Knowledge Base Management
Knowledge Base Management
POST / - Create Knowledge Base
POST / - Create Knowledge Base
- Request
- Response
POST /api/v1/kb/Request Body Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbName | string | Yes | Name for the knowledge base (1-255 characters) |
GET / - List Knowledge Bases
GET / - List Knowledge Bases
- Request
- Response
GET /api/v1/kb/Query Parameters:| Parameter | Type | Default | Description |
|---|---|---|---|
| page | number | 1 | Page number for pagination |
| limit | number | 20 | Number of items per page (1-100) |
| search | string | - | Search term for filtering knowledge bases (max 100 characters) |
| permissions | string | - | Comma-separated list of permissions to filter by |
| sortBy | string | createdAt | Field to sort by (createdAt, updatedAt, name) |
| sortOrder | string | desc | Sort order (asc, desc) |
GET /:kbId - Get Knowledge Base
GET /:kbId - Get Knowledge Base
- Request
- Response
GET /api/v1/kb/:kbIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
PUT /:kbId - Update Knowledge Base
PUT /:kbId - Update Knowledge Base
- Request
- Response
PUT /api/v1/kb/:kbIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
| Parameter | Type | Required | Description |
|---|---|---|---|
| name | string | No | New name for the knowledge base |
DELETE /:kbId - Delete Knowledge Base
DELETE /:kbId - Delete Knowledge Base
- Request
- Response
DELETE /api/v1/kb/:kbIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
Record Management
Record Management
GET /records - Get All Records
GET /records - Get All Records
- Request
- Response
GET /api/v1/kb/recordsQuery Parameters:| Parameter | Type | Default | Description |
|---|---|---|---|
| page | number | 1 | Page number for pagination |
| limit | number | 20 | Number of items per page (1-100) |
| search | string | - | Search term for filtering records (max 100 characters) |
| recordTypes | string | - | Comma-separated list of record types (FILE, EMAIL, WEBPAGE, MESSAGE, OTHERS) |
| origins | string | - | Comma-separated list of origins (UPLOAD, CONNECTOR) |
| connectors | string | - | Comma-separated list of connector names |
| indexingStatus | string | - | Comma-separated list of indexing statuses (NOT_STARTED, IN_PROGRESS, FAILED, COMPLETED) |
| permissions | string | - | Comma-separated list of permissions to filter by |
| dateFrom | number | - | Filter by creation timestamp (milliseconds) |
| dateTo | number | - | Filter by creation timestamp (milliseconds) |
| sortBy | string | createdAtTimestamp | Field to sort by |
| sortOrder | string | desc | Sort order (asc, desc) |
| source | string | all | Source filter (all, local, connector) |
GET /:kbId/records - Get KB Records
GET /:kbId/records - Get KB Records
- Request
- Response
GET /api/v1/kb/:kbId/recordsPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
GET /record/:recordId - Get Record By ID
GET /record/:recordId - Get Record By ID
- Request
- Response
GET /api/v1/kb/record/:recordIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| recordId | string | Yes | Record UUID |
PUT /record/:recordId - Update Record
PUT /record/:recordId - Update Record
- Request
- Response
PUT /api/v1/kb/record/:recordIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| recordId | string | Yes | Record UUID |
| Parameter | Type | Required | Description |
|---|---|---|---|
| recordName | string | No | New name for the record |
| file | file | No | New file content (creates a new version) |
multipart/form-data (when uploading file)File Constraints:- Maximum file size: 30MB
- Supported MIME types: Based on extensionToMimeType mapping
- File extension must match existing record’s extension for versioned updates
DELETE /record/:recordId - Delete Record
DELETE /record/:recordId - Delete Record
- Request
- Response
DELETE /api/v1/kb/record/:recordIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| recordId | string | Yes | Record UUID |
GET /stream/record/:recordId - Stream Record Content
GET /stream/record/:recordId - Stream Record Content
- Request
- Response
GET /api/v1/kb/stream/record/:recordIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| recordId | string | Yes | Record UUID |
POST /reindex/record/:recordId - Reindex Record
POST /reindex/record/:recordId - Reindex Record
- Request
- Response
POST /api/v1/kb/reindex/record/:recordIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| recordId | string | Yes | Record UUID |
Upload Operations
Upload Operations
POST /:kbId/upload - Upload to Knowledge Base
POST /:kbId/upload - Upload to Knowledge Base
- Request
- Response
POST /api/v1/kb/:kbId/uploadPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
| Parameter | Type | Required | Description |
|---|---|---|---|
| files | file[] | Yes | Array of files to upload (max 1000 files, 30MB per file) |
| file_paths | string[] | Yes | Array of file paths corresponding to files |
| last_modified | number[] | Yes | Array of last modified timestamps (milliseconds) |
| isVersioned | boolean | No | Whether files should be versioned (default: true) |
multipart/form-dataFile Constraints:- Maximum files per upload: 1000
- Maximum file size: 30MB per file
- Supported MIME types: Based on extensionToMimeType mapping
- File paths must be unique and not contain invalid characters
POST /:kbId/folder/:folderId/upload - Upload to Folder
POST /:kbId/folder/:folderId/upload - Upload to Folder
- Request
- Response
POST /api/v1/kb/:kbId/folder/:folderId/uploadPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
| folderId | string | Yes | Target folder UUID |
Folder Operations
Folder Operations
GET /:kbId/folder/:folderId/records - Get Folder Contents
GET /:kbId/folder/:folderId/records - Get Folder Contents
- Request
- Response
GET /api/v1/kb/:kbId/folder/:folderId/recordsPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
| folderId | string | Yes | Folder UUID |
PUT /:kbId/folder/:folderId - Update Folder
PUT /:kbId/folder/:folderId - Update Folder
- Request
- Response
PUT /api/v1/kb/:kbId/folder/:folderIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
| folderId | string | Yes | Folder UUID |
| Parameter | Type | Required | Description |
|---|---|---|---|
| folderName | string | Yes | New name for the folder |
DELETE /:kbId/folder/:folderId - Delete Folder
DELETE /:kbId/folder/:folderId - Delete Folder
- Request
- Response
DELETE /api/v1/kb/:kbId/folder/:folderIdPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| kbId | string | Yes | Knowledge base UUID |
| folderId | string | Yes | Folder UUID |
Administrative Operations
Administrative Operations
GET /stats/:connector - Get Connector Stats
GET /stats/:connector - Get Connector Stats
- Request
- Response
GET /api/v1/kb/stats/:connectorPath Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| connector | string | Yes | Connector name (onedrive, google_drive, confluence, slack, etc.) |
POST /reindex-all/connector - Reindex All Failed Records
POST /reindex-all/connector - Reindex All Failed Records
- Request
- Response
POST /api/v1/kb/reindex-all/connectorRequest Body Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| connector | string | Yes | Connector name to reindex |
| orgId | string | No | Organization ID (defaults to user’s org) |
POST /resync/connector - Resync Connector Records
POST /resync/connector - Resync Connector Records
- Request
- Response
POST /api/v1/kb/resync/connectorRequest Body Parameters:| Parameter | Type | Required | Description |
|---|---|---|---|
| connector | string | Yes | Connector name to resync |
| orgId | string | No | Organization ID (defaults to user’s org) |
| fullResync | boolean | No | Whether to perform full resync (default: false) |
Event System
The Knowledge Base Service broadcasts events through Kafka to notify other services about record changes. These events trigger actions like content indexing, search updates, and audit logging.Event Types
| Event Type | Description |
|---|---|
newRecord | Triggered when a new record is created |
updateRecord | Triggered when a record is updated |
deletedRecord | Triggered when a record is deleted |
reindexRecord | Triggered when a record needs reindexing |
Event Payload Structure
New Record Event
New Record Event
Update Record Event
Update Record Event
Deleted Record Event
Deleted Record Event
Schema Definitions
Record Schema
Record Schema
File Record Schema
File Record Schema
Knowledge Base Schema
Knowledge Base Schema
Upload Schema
Upload Schema
Error Handling
All endpoints return structured error responses with specific HTTP status codes:400 Bad Request- Invalid request parameters, file validation errors401 Unauthorized- Missing or invalid authentication403 Forbidden- Insufficient permissions404 Not Found- Resource not found413 Payload Too Large- File size exceeds limit422 Unprocessable Entity- Validation errors500 Internal Server Error- Server errors, backend service failures
- File extension mismatch for versioned updates
- Unsupported MIME types
- File size exceeds 30MB limit
- Invalid file path characters
- Duplicate file paths in upload
Integration with AI Indexing
When a record is created or updated, the Knowledge Base Service:- Stores the file metadata in ArangoDB
- Uploads the file content to the Storage Service
- Publishes a Kafka event with the file metadata and download URL
- The AI Backend consumes these events and:
- Downloads the file content
- Extracts text and structured data
- Processes the content for search indexing
- Updates the indexing status in the Knowledge Base
Security and Permissions
- All endpoints require valid JWT authentication
- Admin endpoints require additional role verification
- File uploads are validated for MIME type and size
- Access control is enforced at the organization level
- File content is streamed securely through signed URLs
- Upload paths are sanitized to prevent directory traversal
Rate Limits and Constraints
- File Upload: Maximum 30MB per file, 1000 files per upload
- Pagination: Maximum 100 items per page
- Search Terms: Maximum 100 characters
- Knowledge Base Names: 1-255 characters
- File Paths: Must not contain invalid characters or be duplicated













