Connector Manager API
The Connector Manager Service provides centralized management of authentication tokens and connector configurations for third-party integrations. This service handles OAuth flows, token lifecycle management, and connector configurations for various external services like Google Workspace, OneDrive, SharePoint, Confluence, and more.Base URL
All endpoints are prefixed with/api/v1/connectors
Authentication
All endpoints require authentication via Bearer token:FETCH_CONFIG
- For configuration updates and token refresh operations
Architecture Overview
The Connector Manager Service is built on a Node.js backend with MongoDB for data persistence, Redis for caching, and ETCD for secure configuration storage. The service consists of several key components:- Token Management - Handles OAuth token lifecycle including creation, refresh, and revocation
- Connector Configuration - Manages connector settings and credentials
- Event Broadcasting - Kafka-based events for system integration
- Health Monitoring - Service health checks and dependency monitoring
- Configuration Manager - Manages encrypted configuration storage
- IAM Service - Handles user authentication and authorization
- Kafka - Event streaming for token and connector state changes
- ETCD - Encrypted key-value storage for sensitive configurations
- Backend Connector Service - Python service that handles actual connector implementations
Data Models
Connectors
Connector configurations that define enabled/disabled state and metadata for third-party integrations.Token Events
Events broadcast when tokens are created, refreshed, revoked, or expired.Entity Events
Events broadcast when connectors are enabled or disabled, triggering synchronization workflows.Configuration
Encrypted storage of OAuth credentials, API keys, and service endpoints.API Endpoints
Connector Management
Connector Management
GET / - Get All Connectors
GET / - Get All Connectors
GET /api/v1/connectors/
Access Control: Admin privileges requiredAuthentication: Bearer token requiredGET /active - Get Active Connectors
GET /active - Get Active Connectors
GET /api/v1/connectors/active
Authentication: Bearer token requiredGET /inactive - Get Inactive Connectors
GET /inactive - Get Inactive Connectors
GET /api/v1/connectors/inactive
Authentication: Bearer token requiredGET /config/:connectorName - Get Connector Config
GET /config/:connectorName - Get Connector Config
GET /api/v1/connectors/config/:connectorName
Path Parameters:connectorName
: Connector name (string, required)
PUT /config/:connectorName - Update Connector Config
PUT /config/:connectorName - Update Connector Config
PUT /api/v1/connectors/config/:connectorName
Path Parameters:connectorName
: Connector name (string, required)
Parameter | Type | Required | Description |
---|---|---|---|
auth | object | No | Authentication configuration |
sync | object | No | Synchronization settings |
filters | object | No | Filter configurations |
baseUrl | string | Yes | Base URL for the connector |
GET /schema/:connectorName - Get Connector Schema
GET /schema/:connectorName - Get Connector Schema
GET /api/v1/connectors/schema/:connectorName
Path Parameters:connectorName
: Connector name (string, required)
POST /toggle/:connectorName - Toggle Connector
POST /toggle/:connectorName - Toggle Connector
POST /api/v1/connectors/toggle/:connectorName
Path Parameters:connectorName
: Connector name (string, required)
OAuth Management
OAuth Management
GET /:connectorName/oauth/authorize - Get OAuth Authorization URL
GET /:connectorName/oauth/authorize - Get OAuth Authorization URL
GET /:connectorName/oauth/callback - Handle OAuth Callback
GET /:connectorName/oauth/callback - Handle OAuth Callback
GET /api/v1/connectors/:connectorName/oauth/callback
Path Parameters:connectorName
: Connector name (string, required)
Parameter | Type | Required | Description |
---|---|---|---|
code | string | No | Authorization code from OAuth provider |
state | string | No | State parameter for CSRF protection |
error | string | No | Error code if authorization failed |
baseUrl | string | Yes | Base URL for the connector service |
POST /getTokenFromCode - Exchange OAuth Code (Legacy)
POST /getTokenFromCode - Exchange OAuth Code (Legacy)
POST /api/v1/connectors/getTokenFromCode
Access Control: Admin privileges requiredRequest Body Parameters:Parameter | Type | Required | Description |
---|---|---|---|
tempCode | string | Yes | OAuth authorization code from Google |
- Retrieves Google Workspace configuration from Configuration Manager
- Exchanges authorization code for access and refresh tokens via Google’s token endpoint
- Verifies the ID token matches the authenticated user’s email
- Stores credentials securely via Configuration Manager
- Creates or updates connector in MongoDB
- Publishes connector enabled event to Kafka
- Determines enabled apps based on OAuth scopes received
Filter Management
Filter Management
GET /filters/:connectorName - Get Filter Options
GET /filters/:connectorName - Get Filter Options
GET /api/v1/connectors/filters/:connectorName
Path Parameters:connectorName
: Connector name (string, required)
POST /filters/:connectorName - Save Filter Options
POST /filters/:connectorName - Save Filter Options
POST /api/v1/connectors/filters/:connectorName
Path Parameters:connectorName
: Connector name (string, required)
Parameter | Type | Required | Description |
---|---|---|---|
filterOptions | object | Yes | Filter configuration object |
Internal Operations
Internal Operations
POST /internal/refreshIndividualConnectorToken - Refresh Token
POST /internal/refreshIndividualConnectorToken - Refresh Token
POST /api/v1/connectors/internal/refreshIndividualConnectorToken
Authentication: Requires scoped token with FETCH_CONFIG
scopeProcess:- Retrieves refresh token from Configuration Manager
- Exchanges refresh token for new access token via Google’s token endpoint
- Implements retry logic with exponential backoff (3 attempts)
- Updates stored credentials via Configuration Manager
- Maintains existing refresh token and expiry times
POST /updateAppConfig - Update Application Config
POST /updateAppConfig - Update Application Config
POST /api/v1/connectors/updateAppConfig
Authentication: Requires scoped token with FETCH_CONFIG
scopeNote: This endpoint reloads the application configuration from the configuration manager and updates the dependency injection container.Health Monitoring
Health Monitoring
GET /health - Get Health Status
GET /health - Get Health Status
GET /api/v1/health
Authentication: No authentication required for health checksEvent System
The Connector Manager Service broadcasts events through Kafka to notify other services about token and connector state changes. These events trigger actions like connector synchronization, indexing updates, and audit logging.Event Topics
Topic | Description |
---|---|
token-events | Token lifecycle events (creation, refresh, revocation, expiration) |
entity-events | Connector state change events (enabled, disabled) |
Event Types
Event Type | Description |
---|---|
TOKEN_CREATED | Triggered when a new token is acquired |
TOKEN_REFRESHED | Triggered when a token is refreshed |
TOKEN_REVOKED | Triggered when a token is revoked |
TOKEN_EXPIRED | Triggered when a token expires |
appEnabled | Triggered when a connector is enabled |
appDisabled | Triggered when a connector is disabled |
Event Payload Structures
Token Events
Token Events
ONEDRIVE
GOOGLE_DRIVE
CONFLUENCE
JIRA
App Enabled Event
App Enabled Event
none
- No synchronization requiredimmediate
- Start synchronization immediatelyscheduled
- Schedule synchronization for later
App Disabled Event
App Disabled Event
Schema Definitions
Application Configuration
Application Configuration
Token Event Schema
Token Event Schema
Entity Event Schema
Entity Event Schema
Google Workspace Types
Google Workspace Types
Health Status Schema
Health Status Schema
Validation Schemas
Validation Schemas
Configuration Management
The Connector Manager Service uses ETCD as a key-value store for managing configurations securely. All sensitive information, such as OAuth client IDs/secrets and access tokens, are encrypted using AES-256 before storage.Configuration Types
The service manages several types of configurations:-
Service Connection Information
- Database connection strings (MongoDB, ArangoDB)
- Message broker details (Kafka brokers, authentication)
- Cache settings (Redis host, port, credentials)
-
Service Discovery
- Backend service endpoints (IAM, Auth, Configuration Manager, etc.)
- Frontend URL for OAuth redirects
-
Authentication Secrets
- JWT signing keys
- Scoped JWT secrets for service-to-service communication
- Cookie encryption secrets
-
Connector Credentials
- OAuth client IDs and secrets
- Access tokens and refresh tokens
- Token expiration timestamps
Security Measures
- All sensitive configuration data is encrypted using AES-256 before storage in ETCD
- Encryption keys are securely managed and not stored in plaintext
- Scoped JWT tokens are used for internal service communication
- Token refresh operations include retry logic with exponential backoff
Error Handling
All endpoints return structured error responses with specific HTTP status codes:200
- Success201
- Created (for new resources)400
- Bad Request (validation errors, missing parameters)401
- Unauthorized (missing or invalid authentication)403
- Forbidden (insufficient privileges, admin required)404
- Not Found (connector or resource not found)500
- Internal Server Error (backend service failures)
- Connection refused → “Connector Service is currently unavailable. Please check your network connection or try again later.”
- Service-specific errors → Mapped to appropriate HTTP status codes with detailed error messages
- Automatic retry logic with exponential backoff for failed requests (3 retries)
Integration Architecture
Backend Connector Service Integration
The Connector Manager Service acts as a proxy to a Python-based Backend Connector Service that handles the actual connector implementations. Most endpoints forward requests to this backend service and return the responses. Request Flow:- Client sends request to Connector Manager Service
- Connector Manager validates authentication and authorization
- Request is forwarded to Backend Connector Service
- Backend Connector Service processes the request
- Response is returned through Connector Manager to client
Configuration Manager Integration
The Connector Manager Service integrates closely with the Configuration Manager for:- Secure Credential Storage - OAuth client IDs, secrets, and tokens
- Service Discovery - Endpoint URLs for backend services
- Configuration Management - Connector settings and preferences
- Encrypted Data Handling - All sensitive data is encrypted before storage
Google Workspace OAuth Flow
For Google Workspace connectors, the service implements a complete OAuth 2.0 flow:- Authorization Request - Generate authorization URL with appropriate scopes
- Code Exchange - Exchange authorization code for access and refresh tokens
- Token Verification - Verify ID token matches authenticated user
- Credential Storage - Securely store tokens via Configuration Manager
- Connector Activation - Enable connector and publish events
- Scope Mapping - Map OAuth scopes to enabled applications (Drive, Gmail, Calendar)
Token Lifecycle Management
The service provides automatic token management with the following features:- Token Acquisition - OAuth flow handling and token exchange
- Token Refresh - Automatic refresh before expiration with retry logic
- Token Revocation - Secure token deletion and cleanup
- Event Broadcasting - Kafka events for token state changes
- Error Handling - Retry logic and fallback mechanisms