Configuration Manager Service

The Configuration Manager Service provides a centralized way to manage configuration settings across your organization’s services. It securely stores various configuration parameters in a key-value store (ETCD by default) and provides APIs to manage these configurations.

Architecture Overview

The Configuration Manager Service is built on a Node.js backend that leverages ETCD (or other compatible key-value stores) for secure configuration storage. The service:

Securely encrypts sensitive configuration values
Provides a RESTful API for managing configurations
Integrates with other services via event-driven architecture
Controls access through authentication and authorization

Key Features

Secure Storage: All sensitive configurations are encrypted using AES-256-GCM algorithm
Health Checks: Built-in validation of configuration parameters
Event Notifications: Publishes events when critical configurations change
Centralized Management: Single service to control settings across the system

Supported Configurations

The Configuration Manager handles various types of configurations:

Storage Configurations - AWS S3, Azure Blob, or Local storage settings
Authentication Configurations - Azure AD, Google, Microsoft, and SSO settings
Database Configurations - MongoDB, ArangoDB, and Qdrant settings
Message Broker Configurations - Kafka settings
Key-Value Store Configurations - Redis settings
SMTP Configurations - Email service settings
AI Models Configurations - Settings for various AI model types
Connector Configurations - Google Workspace and other integration settings
Public URL Configurations - Frontend and connector URLs
Metrics Collection Configurations - Settings for metrics gathering

Storage Configuration API

The Storage Configuration API enables you to configure different storage backends for your application.

Set Storage Configuration

Configure the storage service with specific backend details.

POST /api/v1/configurationManager/storageConfig

Get Storage Configuration

Retrieve the current storage configuration.

GET /api/v1/configurationManager/storageConfig

Authentication Configuration API

The Authentication Configuration API allows you to manage various authentication providers.

Azure AD Authentication

POST /api/v1/configurationManager/authConfig/azureAd

Get Azure AD Config

GET /api/v1/configurationManager/authConfig/azureAd

SSO Authentication

POST /api/v1/configurationManager/authConfig/sso

Database Configuration API

Configure database connections for your application.

MongoDB Configuration

POST /api/v1/configurationManager/mongoDBConfig

ArangoDB Configuration

POST /api/v1/configurationManager/arangoDBConfig

Qdrant Configuration

POST /api/v1/configurationManager/qdrantConfig

Message Broker Configuration API

Configure message broker for event-driven architecture.

Kafka Configuration

POST /api/v1/configurationManager/kafkaConfig

SMTP Configuration API

Configure email service settings.

POST /api/v1/configurationManager/smtpConfig

AI Models Configuration API

Configure AI models for various tasks.

POST /api/v1/configurationManager/aiModelsConfig

Connector Configuration API

Configure connectors for integrating with external services.

Google Workspace Configuration

POST /api/v1/configurationManager/connectors/googleWorkspaceOauthConfig

Key-Value Store Configuration API

Configure Redis as a key-value store.

POST /api/v1/configurationManager/redisConfig

Public URL Configuration API

Configure public URLs for frontend and connector services.

POST /api/v1/configurationManager/frontendPublicUrl

POST /api/v1/configurationManager/connectorPublicUrl

Metrics Collection Configuration API

Configure metrics collection settings.

PATCH /api/v1/configurationManager/metricsCollection/toggle

Schema Definitions

Authentication Config Schema

// Azure AD Config
interface AzureAdConfig {
  clientId: string;
  tenantId: string;
  authority: string;
}

// SSO Config
interface SsoConfig {
  entryPoint: string;
  certificate: string;
  emailKey: string;
}

// Google Auth Config
interface GoogleAuthConfig {
  clientId: string;
}

// Microsoft Auth Config
interface MicrosoftAuthConfig {
  clientId: string;
  tenantId: string;
  authority: string;
}

Storage Config Schema

// S3 Storage Config
interface S3Config {
  accessKeyId: string;
  secretAccessKey: string;
  region: string;
  bucketName: string;
}

// Azure Blob Storage Config
interface AzureBlobConfig {
  endpointProtocol: string;
  accountName: string;
  accountKey: string;
  endpointSuffix: string;
  containerName: string;
}

// Local Storage Config
interface LocalConfig {
  mountName: string;
  baseUrl: string;
}

Database Config Schema

// MongoDB Config
interface MongoDBConfig {
  uri: string;
  db: string;
}

// ArangoDB Config
interface ArangoDBConfig {
  url: string;
  username: string;
  password: string;
  db: string;
}

// Qdrant Config
interface QdrantConfig {
  host: string;
  port: number;
  grpcPort?: number;
  apiKey?: string;
}

Message Broker Config Schema

Connector Config Schema

// Google Workspace Individual Credentials
interface GoogleWorkspaceIndividualCredentials {
  access_token: string;
  refresh_token: string;
  access_token_expiry_time: number;
  refresh_token_expiry_time?: number;
  enableRealTimeUpdates: boolean;
  topicName: string;
}

// Google Workspace Business Credentials
interface GoogleWorkspaceBusinessCredentials {
  type: string;
  project_id: string;
  private_key_id: string;
  private_key: string;
  client_email: string;
  client_id: string;
  auth_uri: string;
  token_uri: string;
  auth_provider_x509_cert_url: string;
  client_x509_cert_url: string;
  universe_domain: string;
  adminEmail: string;
  enableRealTimeUpdates: boolean;
  topicName: string;
}

// Google Workspace OAuth Config
interface GoogleWorkspaceOAuthConfig {
  clientId: string;
  clientSecret: string;
  enableRealTimeUpdates: boolean;
  topicName: string;
}

Encryption and Security

The Configuration Manager Service uses AES-256-GCM encryption to protect sensitive configuration data. Key features include:

Secret Key Hashing: The secret key used for encryption is hashed using SHA-256 for added security
Per-Configuration Encryption: Each sensitive configuration is individually encrypted
Fine-grained Access Control: API endpoints require appropriate authentication and authorization

Event-Driven Architecture

The Configuration Manager publishes events when critical configurations change:

LLM Configured Event: When AI models are configured
Connector Public URL Changed Event: When connector URLs are updated
Gmail Updates Enabled/Disabled Event: When Gmail real-time updates are toggled

These events allow other services to react to configuration changes without polling.

Health Checks

The service includes built-in health checks for various configurations:

Kafka Health Check: Verifies Kafka broker connectivity
Redis Health Check: Tests Redis connection
Qdrant Health Check: Ensures Qdrant vector database is accessible
MongoDB Health Check: Validates MongoDB connection
ArangoDB Health Check: Confirms ArangoDB connection

These health checks are automatically performed before saving configurations to ensure validity.

Welcome To PipesHub

System Overview

Authentication

Mail Configuration

AI Providers

Connectors

User Management

Deployment

Developer

Additional Resources

Configuration Manager

Configuration Manager Service

Architecture Overview

Key Features

Supported Configurations

Storage Configuration API

Set Storage Configuration

Get Storage Configuration

Authentication Configuration API

Azure AD Authentication

Get Azure AD Config

SSO Authentication

Database Configuration API

MongoDB Configuration

ArangoDB Configuration

Qdrant Configuration

Message Broker Configuration API

Kafka Configuration

SMTP Configuration API

AI Models Configuration API

Connector Configuration API

Google Workspace Configuration

Key-Value Store Configuration API

Public URL Configuration API

Metrics Collection Configuration API

Schema Definitions

Encryption and Security

Event-Driven Architecture

Health Checks

Welcome To PipesHub

System Overview

Authentication

Mail Configuration

AI Providers

Connectors

User Management

Deployment

Developer

Additional Resources

​Configuration Manager Service

​Architecture Overview

​Key Features

​Supported Configurations

​Storage Configuration API

​Set Storage Configuration

​Get Storage Configuration

​Authentication Configuration API

​Azure AD Authentication

​Get Azure AD Config

​SSO Authentication

​Database Configuration API

​MongoDB Configuration

​ArangoDB Configuration

​Qdrant Configuration

​Message Broker Configuration API

​Kafka Configuration

​SMTP Configuration API

​AI Models Configuration API

​Connector Configuration API

​Google Workspace Configuration

​Key-Value Store Configuration API

​Public URL Configuration API

​Metrics Collection Configuration API

​Schema Definitions

​Encryption and Security

​Event-Driven Architecture

​Health Checks

Configuration Manager Service

Architecture Overview

Key Features

Supported Configurations

Storage Configuration API

Set Storage Configuration

Get Storage Configuration

Authentication Configuration API

Azure AD Authentication

Get Azure AD Config

SSO Authentication

Database Configuration API

MongoDB Configuration

ArangoDB Configuration

Qdrant Configuration

Message Broker Configuration API

Kafka Configuration

SMTP Configuration API

AI Models Configuration API

Connector Configuration API

Google Workspace Configuration

Key-Value Store Configuration API

Public URL Configuration API

Metrics Collection Configuration API

Schema Definitions

Encryption and Security

Event-Driven Architecture

Health Checks