Enterprise Search Service

The Enterprise Search Service provides intelligent search capabilities across your organization’s data, allowing users to find information through natural language queries. The service consists of two main components:

Conversational Search API - Enables chat-style interactions with your data
Semantic Search API - Powers direct search queries across your content

Architecture Overview

The Enterprise Search Service is built on a Node.js backend with MongoDB for data persistence. It leverages AI models to provide accurate responses and semantic search capabilities. The service integrates with other components such as:

AI Backend - Processes queries and generates responses
IAM Service - Handles user authentication and authorization
Configuration Manager - Manages application settings

Data Models

Conversations

Conversations represent chat sessions with the AI assistant. Each conversation consists of:

User queries and AI responses stored as messages
Citations that reference source documents
Metadata about the conversation (creation time, sharing status, etc.)

Citations

Citations are references to source documents that back the AI’s responses:

Content snippets from source documents
Metadata about the source (document name, type, location)
Relevance information

Searches

Searches represent individual search queries and their results:

The original query
Citation results
Metadata about the search (creation time, sharing status, etc.)

Conversation API

The Conversation API enables chat-style interactions with your organizational data.

Create Conversation Thread

Start a new conversation thread.

POST /api/v1/conversations/create

Add Message

Adds a new message to an existing conversation.

POST /api/v1/conversations/:conversationId/messages

Get All Conversations

Retrieves all conversations for the authenticated user.

GET /api/v1/conversations

Get Conversation By ID

Retrieve conversation by id for the authenticated user.

GET /api/v1/conversations/:conversationId

Delete Conversation

Delete a conversation by its id for the authenticated user. This is a soft delete operation.

DELETE /api/v1/conversations/:conversationId

Share the conversation within your organization.

POST /api/v1/conversations/:conversationId/share

Unshare Conversation

Revoke the shared conversation access.

POST /api/v1/conversations/:conversationId/unshare

Regenerate Answer

Regenerate the answer.

POST /api/v1/conversations/:conversationId/message/:messageId/regenerate

Update Title

Update the Title of a conversation thread.

PATCH /api/v1/conversations/:conversationId/title

Add Feedback

Provide the feedback to the AI Service.

POST /api/v1/conversations/:conversationId/message/:messageId/feedback

Archive Conversation

Archive a conversation thread.

PATCH /api/v1/conversations/:conversationId/archive

Unarchive Conversation

Unarchive archived conversation thread.

PATCH /api/v1/conversations/:conversationId/unarchive

List Archived Conversations

List all the archived conversation.

GET /api/v1/conversations/show/archives

Semantic Search API

The Semantic Search API allows for direct search queries across your organization’s content.

Search

Performs a semantic search across indexed content.

POST /api/v1/search

Search History

Retrieves all of the search history in pagination.

GET /api/v1/search

Get Search By ID

Retrieve a particular search id.

GET /api/v1/search/:searchId

Delete Search

Delete a particular search id.

DELETE /api/v1/search/:searchId

Delete Search History

Clean all search history.

DELETE /api/v1/search

Share the search result within organization.

PATCH /api/v1/search/:searchId/share

Unshare Search

Unshare the shared search.

PATCH /api/v1/search/:searchId/unshare

Archive Search

Archive a search result.

PATCH /api/v1/search/:searchId/archive

Unarchive Search

Unarchive a search result.

PATCH /api/v1/search/:searchId/unarchive

Schema Definitions

Conversation Schema

interface IConversation {
  // User and organization identifiers
  userId: Types.ObjectId;
  orgId: Types.ObjectId;
  
  // Conversation metadata
  title?: string;
  initiator: Types.ObjectId;
  messages: IMessageDocument[];
  
  // Sharing settings
  isShared?: boolean;
  shareLink?: string;
  sharedWith?: Array<{
    userId: Types.ObjectId;
    accessLevel: 'read' | 'write';
  }>;
  
  // State flags
  isDeleted?: boolean;
  deletedBy?: Types.ObjectId;
  isArchived?: boolean;
  archivedBy?: Types.ObjectId;
  
  // Activity tracking
  lastActivityAt?: Date;
  tags?: Types.ObjectId[];
  
  // Source information
  conversationSource:
    | 'enterprise_search'
    | 'records'
    | 'connectors'
    | 'internet_search'
    | 'personal_kb_search';
  conversationSourceRecordId?: Types.ObjectId;
  conversationSourceConnectorIds?: Types.ObjectId[];
  conversationSourceRecordType?: string;
  
  // Timestamps
  createdAt?: Date;
  updatedAt?: Date;
}

Message Schema

interface IMessage {
  // Message type and content
  messageType: 'user_query' | 'bot_response' | 'error' | 'feedback' | 'system';
  content: string;
  contentFormat?: 'MARKDOWN' | 'JSON' | 'HTML';
  
  // Citations and follow-ups
  citations?: IMessageCitation[];
  confidence?: string;
  followUpQuestions?: IFollowUpQuestion[];
  
  // User feedback
  feedback?: IFeedback[];
  
  // Processing metadata
  metadata?: {
    processingTimeMs?: number;
    modelVersion?: string;
    aiTransactionId?: string;
    reason?: string;
  };
  
  // Timestamps
  createdAt?: Date;
  updatedAt?: Date;
}

interface IFollowUpQuestion {
  question: string;
  confidence: string;
  reasoning?: string;
}

interface IMessageCitation {
  citationId?: Types.ObjectId;
  relevanceScore?: number;
  excerpt?: string;
  context?: string;
}

Citation Schema

interface ICitation extends Document {
  // Core citation data
  content: string;
  chunkIndex: number;
  metadata: ICitationMetadata;
  citationType: string;
  
  // Timestamps
  createdAt: Date;
  updatedAt: Date;
}

interface ICitationMetadata {
  // Document position information
  blockNum?: number[];
  pageNum?: number[];
  sheetNum?: number;
  sheetName?: string;
  
  // Categorization
  subcategoryLevel1?: string;
  subcategoryLevel2?: string;
  subcategoryLevel3?: string;
  categories?: string;
  departments?: string[];
  topics?: string[];
  languages?: string[];
  
  // Source information
  connector?: string;
  recordType?: string;
  orgId: string;
  blockType?: string;
  mimeType: string;
  recordId: string;
  recordVersion: number;
  recordName: string;
  origin: string;
  extension: string;
  
  // Positioning (for visual documents)
  bounding_box?: Array<{
    x: number;
    y: number;
  }>;
  
  // Search relevance
  score?: number;
  
  // Additional identifiers
  _id?: string;
  _collection_name?: string;
}

Search Schema

interface IEnterpriseSemanticSearch extends Document {
  // Search parameters
  query: string;
  limit: number;
  
  // Sharing settings
  isShared: boolean;
  shareLink: string;
  sharedWith: {
    userId: mongoose.Types.ObjectId;
    accessLevel: string;
  }[];
  
  // State flags
  isArchived: boolean;
  archivedBy: mongoose.Types.ObjectId;
  
  // User and organization identifiers
  orgId: mongoose.Types.ObjectId;
  userId: mongoose.Types.ObjectId;
  
  // Search results
  citationIds: mongoose.Types.ObjectId[]; // References to citation documents
  records: Record<string, any>; // Map of record details
  
  // Timestamps
  createdAt: Date;
  updatedAt: Date;
}

Welcome To PipesHub

System Overview

Authentication

Mail Configuration

AI Providers

Connectors

User Management

Deployment

Developer

Additional Resources

Enterprise Search

Enterprise Search Service

Architecture Overview

Data Models

Conversations

Citations

Searches

Conversation API

Create Conversation Thread

Add Message

Get All Conversations

Get Conversation By ID

Delete Conversation

Unshare Conversation

Regenerate Answer

Update Title

Add Feedback

Archive Conversation

Unarchive Conversation

List Archived Conversations

Semantic Search API

Search

Search History

Get Search By ID

Delete Search

Delete Search History

Unshare Search

Archive Search

Unarchive Search

Schema Definitions

Welcome To PipesHub

System Overview

Authentication

Mail Configuration

AI Providers

Connectors

User Management

Deployment

Developer

Additional Resources

​Enterprise Search Service

​Architecture Overview

​Data Models

​Conversations

​Citations

​Searches

​Conversation API

​Create Conversation Thread

​Add Message

​Get All Conversations

​Get Conversation By ID

​Delete Conversation

​Share Conversation

​Unshare Conversation

​Regenerate Answer

​Update Title

​Add Feedback

​Archive Conversation

​Unarchive Conversation

​List Archived Conversations

​Semantic Search API

​Search

​Search History

​Get Search By ID

​Delete Search

​Delete Search History

​Share Search

​Unshare Search

​Archive Search

​Unarchive Search

​Schema Definitions

Enterprise Search Service

Architecture Overview

Data Models

Conversations

Citations

Searches

Conversation API

Create Conversation Thread

Add Message

Get All Conversations

Get Conversation By ID

Delete Conversation

Share Conversation

Unshare Conversation

Regenerate Answer

Update Title

Add Feedback

Archive Conversation

Unarchive Conversation

List Archived Conversations

Semantic Search API

Search

Search History

Get Search By ID

Delete Search

Delete Search History

Share Search

Unshare Search

Archive Search

Unarchive Search

Schema Definitions