OneDrive

​

Step 1: Register Application in Azure Portal

• Navigate to portal.azure.com and sign in with your Microsoft 365 administrator account

• Search for “App registrations” in the top search bar
• Or navigate to Microsoft Entra ID → App registrations

• Click “New registration”

• Enter application details:
  • Name: Enter app name (e.g., “PipesHub Connector”)
  • Supported account types: Select “Accounts in this organizational directory only (Single tenant)”
  • Select a platform: Web
  • Redirect URI: Get the redirect URI from PipesHub configure OneDrive dialog in connector settings

• Click “Register”

​

Step 2: Copy Application Credentials

• Application (client) ID: Found under “Essentials” section
• Directory (tenant) ID: Found under “Essentials” section

​

Step 3: Create Client Secret

• In the left sidebar, click “Certificates & secrets”
• Click “New client secret”
• Configure the secret:
  • Description: Enter a description (e.g., “PipesHub Connector Secret”)
  • Expires: Choose expiration period (recommended: 24 months)
• Click “Add”
• Immediately copy the secret value from the “Value” column

​

Step 4: Configure API Permissions

• In the left sidebar, click “API permissions”
• Click “Add a permission” → “Microsoft Graph”
• Choose “Application permissions” (not Delegated permissions)
• Add the following permissions:
  • User.Read.All
  • Group.Read.All
  • Files.Read.All
• Click “Add permissions”

​

Step 5: Grant Admin Consent

• On the API permissions page, click “Grant admin consent for [Your Organization]”
• Confirm by clicking “Yes”
• Wait for the status to show green checkmarks

​

Step 6: Configure Connector in PipesHub

• Navigate to Settings → Connectors in PipesHub
• Find the OneDrive connector and click “Configure”
• Enter the following details:
  • Application (Client) ID: From Step 2
  • Client Secret: From Step 3
  • Directory (Tenant) ID: From Step 2
  • Redirect URI: Pre-filled (verify it matches your environment)
• Check the “Has Admin Consent” checkbox
• Click “Next” or “Save”

​

Step 7: Enable the Connector

• After saving the configuration, toggle the connector to “Enable”
• The connector will verify credentials and begin initial sync
• Wait for the status to show “Connected” or “Syncing”

​

How Does OneDrive Connector Work?

1. Sync Users
2. Sync User Groups
3. Sync User Drives (Records)

​

1. User Synchronization

• Calls msgraph_client.get_all_users() to fetch all users
• Publishes new/updated users to the data store via data_entities_processor.on_new_app_users()

​

2. User Group Synchronization

• Calls _sync_user_groups() to fetch all groups and their members
• Publishes groups and their members to the data store via data_entities_processor.on_new_user_groups()

​

Initial Full Sync

• Starts with the base Delta API endpoint
• Processes all existing groups in pages
• Each group triggers member fetching and processing
• Saves deltaLink upon completion for future incremental syncs

​

Incremental Sync

• Uses saved deltaLink to fetch only changes since last sync
• Processes additions, updates, and deletions
• Updates sync state with new deltaLink for next run

​

Change Detection

• ADD/UPDATE: Creates or updates group metadata and syncs all members
• DELETE: Removes group from system when @removed marker is present

• Processes member additions and removals via members@delta field
• Fetches user email for each member change
• Removes the group member if @remove field is present
• Member addition is handled as part of the group’s ADD/UPDATE process.

​

3. Records Synchronization

​

Processing Users in Batches

• Calls _process_users_in_batches(users)
• Filtered users are divided into small batches (controlled by max_concurrent_batches = 3)
• Each batch is processed in parallel using asyncio.gather
• A short delay is introduced between batches to rate limit API calls

​

Sync Drive Items for Each User

1. The connector checks the persistent sync point (drive_delta_sync_point) for the last token (either deltaLink or nextLink)
2. If a token exists, the sync is incremental; otherwise, it initiates a full scan
3. The connector calls the Delta API endpoint to retrieve all drive items that have changed
4. Fetched items are passed to _process_delta_items_generator to extract metadata and changes
5. The _process_delta_items_generator yields a RecordUpdate item along with permissions for each change.
6. New file records and permissions are collected into batches (batch_size)
7. Once a batch is full, it’s published via data_entities_processor.on_new_records
8. Updated records are handled via _handle_record_updates
9. After all items are processed, the deltaLink is stored as the starting point for the next sync