Skip to main content
Microsoft OneDrive Logo

​
Microsoft OneDrive

Cloud storage and file management

✅ Ready📚 Documentation Available

​
Overview

OneDrive is Microsoft’s cloud storage service that lets you store, sync, and share files across all your devices, often bundled with Microsoft 365. The OneDrive connector uses OAuth 2.0 via Microsoft Graph API with Application Permissions, allowing background access to organizational files and folders without requiring individual user sign-ins.

​
Configuration Setup

​
Step 1: Register Application in Azure Portal

Sign in to Azure Portal:
  • Navigate to portal.azure.com and sign in with your Microsoft 365 administrator account
Access App Registrations:
  • Search for “App registrations” in the top search bar
  • Or navigate to Microsoft Entra ID → App registrations
Create New Registration:
  • Click “New registration”
OneDrive
  • Enter application details:
    • Name: Enter app name (e.g., “PipesHub Connector”)
    • Supported account types: Select “Accounts in this organizational directory only (Single tenant)”
    • Leave Redirect URI blank
  • Click “Register”
OneDrive
Single-tenant configuration ensures the application only works within your organization for better security.

​
Step 2: Copy Application Credentials

After registration, you’ll see the Overview page. Copy the following values:
  • Application (client) ID: Found under “Essentials” section
  • Directory (tenant) ID: Found under “Essentials” section
OneDrive
Copy the “Application (client) ID” NOT the “Object ID”. These are different values.

​
Step 3: Create Client Secret

  • In the left sidebar, click “Certificates & secrets”
  • Click “New client secret”
  • Configure the secret:
    • Description: Enter a description (e.g., “PipesHub Connector Secret”)
    • Expires: Choose expiration period (recommended: 24 months)
  • Click “Add”
  • Immediately copy the secret value from the “Value” column
OneDrive
The client secret value is only displayed once! If you navigate away, you cannot retrieve it again and must create a new secret.

​
Step 4: Configure API Permissions

  • In the left sidebar, click “API permissions”
  • Click “Add a permission” → “Microsoft Graph”
  • Choose “Application permissions” (not Delegated permissions)
  • Add the following permissions:
    • User.Read.All
    • Group.Read.All
    • Files.Read.All
  • Click “Add permissions”
OneDrive
  • On the API permissions page, click “Grant admin consent for [Your Organization]”
  • Confirm by clicking “Yes”
  • Wait for the status to show green checkmarks
OneDrive
Important: The administrator granting consent should use the same email address as the PipesHub account configuring this connector. Using a different email will cause permission issues on the PipesHub platform.
Admin consent is required for application permissions. Only Global Administrators or Application Administrators can grant this consent.

​
Step 6: Configure Connector in PipesHub

  • Navigate to Settings → Connectors in PipesHub
  • Find the OneDrive connector and click “Configure”
  • Enter the following details:
    • Application (Client) ID: From Step 2
    • Client Secret: From Step 3
    • Directory (Tenant) ID: From Step 2
  • Check the “Has Admin Consent” checkbox
  • Click “Next”
OneDrive
Unlike user-facing OAuth flows, there’s no login screen because the connector uses application permissions with client credentials.

​
Sync Settings

Sync settings allow you to control how the connector syncs data from OneDrive.Sync Strategy:
  • Scheduled - The connector will automatically sync data from OneDrive at the scheduled interval.
  • Manual - The connector will sync data from OneDrive only when Admin manually triggers the sync from the connector settings page.
Sync Interval: How often the connector syncs data (default: 60 minutes).

​
Sync Filters

Sync filters control which files are fetched from OneDrive. Files that don’t match are skipped entirely. Folders are always synced regardless of these settings to preserve directory structure.

​
Created Date

Filters files by when they were created. Accepts operators like Last 365 Days, Is After, Is Before, or Is Between for a specific range.For example, set Last 365 Days to fetch only files created in the last year.

​
Modified Date

Same as Created Date but filters by the file’s last modified timestamp. Use Last 30 Days or Is After to limit sync to recently updated files.

​
File Extensions

Filters files support 2 options: In (only sync listed extensions) or Not In (sync everything except listed extensions). After selecting your desired option, you can select your desired extensions from a list of supported extensions.
Unsupported extensions are excluded under In and allowed through under Not In.

​
Indexing Filters

Indexing filters control which synced files are made searchable. Unlike sync filters, these don’t prevent files from being fetched — files are still recorded in the system but their indexing status is set to Manual Indexing.They can be manually indexed later from the All Records section.

​
Manual Indexing

When enabled, files are synced but not automatically indexed. They won’t appear in search results until explicitly indexed.This can allow you to explicitly control which files and folders are indexed for search, by going to the All Records section and indexing the records manually.

​
Index Shared Items

When enabled, files shared with others are indexed alongside personal files. Disable this to exclude shared content and index only files originating from a user’s own drive.

​
Connector Workflow

​
How Does OneDrive Connector Work?

The run_sync method orchestrates the complete synchronization process in three main steps:
  1. Sync Users
  2. Sync User Groups
  3. Sync User Drives (Records)

​
1. User Synchronization

The main sync function calls msgraph_client.get_all_users() to interact with the Microsoft Graph API and retrieve a list of all users configured in the organization’s Azure Active Directory (or Microsoft 365 tenant).Workflow:
  • Calls msgraph_client.get_all_users() to fetch all users
  • Publishes new/updated users to the data store via data_entities_processor.on_new_app_users()

​
2. User Group Synchronization

Group membership is critical for accurately handling shared permissions (e.g., when a file is shared with “Everyone in Marketing”).Workflow:
  • Calls _sync_user_groups() to fetch all groups and their members
  • Publishes groups and their members to the data store via data_entities_processor.on_new_user_groups()

​
Initial Full Sync

On first run (no saved sync state):
  • Starts with the base Delta API endpoint
  • Processes all existing groups in pages
  • Each group triggers member fetching and processing
  • Saves deltaLink upon completion for future incremental syncs

​
Incremental Sync

On subsequent runs (with saved deltaLink):
  • Uses saved deltaLink to fetch only changes since last sync
  • Processes additions, updates, and deletions
  • Updates sync state with new deltaLink for next run

​
Change Detection

The system handles three types of changes:Group Changes:
  • ADD/UPDATE: Creates or updates group metadata and syncs all members
  • DELETE: Removes group from system when @removed marker is present
Member Changes:
  • Processes member additions and removals via members@delta field
  • Fetches user email for each member change
  • Removes the group member if @remove field is present
  • Member addition is handled as part of the group’s ADD/UPDATE process.

​
3. Records Synchronization

​
Processing Users in Batches

The initial list of all Microsoft 365 users is filtered against the application’s internal list of active users. Only users with active accounts in the core system proceed to the drive sync stage.Workflow:
  • Calls _process_users_in_batches(users)
  • Filtered users are divided into small batches (controlled by max_concurrent_batches = 3)
  • Each batch is processed in parallel using asyncio.gather
  • A short delay is introduced between batches to rate limit API calls

​
Sync Drive Items for Each User

This function manages the synchronization of a single user’s OneDrive content using the Delta API for incremental changes.Process:
  1. The connector checks the persistent sync point (drive_delta_sync_point) for the last token (either deltaLink or nextLink)
  2. If a token exists, the sync is incremental; otherwise, it initiates a full scan
  3. The connector calls the Delta API endpoint to retrieve all drive items that have changed
  4. Fetched items are passed to _process_delta_items_generator to extract metadata and changes
  5. The _process_delta_items_generator yields a RecordUpdate item along with permissions for each change.
  6. New file records and permissions are collected into batches (batch_size)
  7. Once a batch is full, it’s published via data_entities_processor.on_new_records
  8. Updated records are handled via _handle_record_updates
  9. After all items are processed, the deltaLink is stored as the starting point for the next sync

​
FAQ

There are three common causes:
  1. Email mismatch - The email used during connector authentication doesn’t match your PipesHub account email. This causes permission issues preventing you from accessing synced data.
    • Fix: Reconfigure the connector and authenticate using the same email as your PipesHub account.
  2. Records show in “All Records” but not in search - Documents display in the All Records section but don’t appear in search query results.
    • Fix: Verify sync filters in the connector settings and ensure indexing has completed. Check that the data matches your search query criteria.
  3. Data not yet indexed - The connector is still syncing or hasn’t completed the initial indexing process.
    • Fix: Go to Settings > Connectors, check the sync status, and wait for indexing to complete.
Azure AD client secrets expire based on the duration set during creation. Once expired, the connector will fail to authenticate and sync will stop.
  1. Go to your app registration in the Azure Portal
  2. Navigate to Certificates & secrets → New client secret
  3. Set a new expiry and copy the secret Value immediately (it won’t be shown again).
  4. Go to Settings > Connectors in PipesHub, open the OneDrive connector, and update the Client Secret field with the new value
To avoid disruption, set a calendar reminder before your secret’s expiry date. The recommended duration is 24 months.
The initial sync performs a full scan of all users’ drives across your organization. The connector fetches every file, folder, and permission from scratch before it has a delta checkpoint to work from. Depending on the size of your organization and number of files, this can take a significant amount of time.All subsequent syncs are incremental — only changes since the last run are fetched, making them considerably faster.
Permission changes are picked up automatically on the next sync cycle. The connector compares current permissions from the Microsoft Graph API against what’s stored and updates accordingly.If your sync is set to Scheduled, the change will reflect within the configured sync interval. If set to Manual, trigger a sync from Settings > Connectors to pull the latest permissions immediately.