Skip to main content
GitHub Logo

​
GitHub

Code repository and version control

✅ Ready📖 Documentation Available

​
Overview

GitHub is the world’s most widely used platform for hosting, versioning, and collaborating on software development projects. It is trusted by individual developers, engineering teams, and enterprises to manage code repositories, automate CI/CD workflows, perform peer code reviews, and track project progress through integrated issue and project management tools.

​
GitHub Data Structure

The connector understands various entities of GitHub. Repositories → Issues & Pull Requests → Comments/Attachments

​
What Gets Synced

The connector indexes the following content for AI-powered search:
  • Issues: Title, description (Markdown with embedded images), status, labels; assignee, created/updated timestamps
  • Pull requests: Same body, comment, and attachment coverage as issues, plus commit SHAs, changed files with patches (diffs), per-file review comments on the diff, and PR review submissions (approve / request changes / comment); assignee and created/updated timestamps
  • Comments: Full threads on issues and on pull requests (including the PR conversation), with author context where available
  • Attachments: Files and images attached to or embedded in issue/PR bodies and comments

​
Configuration Guide

​
Authentication Method

The GitHub connector uses OAuth Apps for secure authentication.
An OAuth App is used as it provides secure, token-based access without requiring users to share their passwords.

​
Generating Client ID and Secret

​
Step 1: Access GitHub Developer Settings

  1. Navigate to the Developer Console:
    Go to settings/developers and sign in with your GitHub account.
  2. View Your Apps: You’ll see the “OAuth apps” page where you can create and manage OAuth 2.0 apps.
GitHub OAuth Apps

​
Step 2: Create a New OAuth App

  1. Click the “New OAuth App” button in the top right corner
  2. Fill in the application details:
    • Application Name: Enter a meaningful name (e.g., “PipesHub GitHub Connector”)
    • Homepage URL: Enter PipesHub Homepage URL (e.g., “http://localhost:3001”)
    • Application description: Description about app (Optional)
    • Authorization callback URL:
      • You need to enter the redirect URI provided by PipesHub
      • To get this URL, open PipesHub in another tab, navigate to PipesHub (bottom left corner) → Workspace settings → Connectors → Your Connectors, find the Github connector, and click “Setup”
        GitHub OAuth App allows only one callback URL.
      • Copy the Redirect URL shown in the configuration dialog:
    PipesHub GitHub Configuration - Copy Redirect URL
  3. Click “Register Application” to proceed

​
Step 3: Get OAuth Credentials

  1. Copy the following credentials:
    • Client ID: Your app’s unique identifier
    • Secret: Your app’s client secret (click “Generate a new client secret”)
OAuth Credentials
Store the Client ID and Client Secret securely. The secret can be regenerated if needed, but you’ll need to update your PipesHub configuration with the new value.

​
Step 4: Configure Connector in PipesHub

  1. Return to the Pipeshub Github configuration dialog (if you closed it, navigate to Pipeshub (bottom left corner) → Workspace settings → Connectors → Your Connectors, find the Github connector, and click Setup)
  2. The configuration dialog has two steps:
    • Step 1: Authentication - Enter OAuth credentials
    • Step 2: Sync Settings - Configure synchronization
  3. Verify the Redirect URI displayed matches the callback URL you configured in GitHub Developer Settings (Step 2)
  4. Enter your OAuth 2.0 credentials:
    • Application (Client) ID: From previous step
    • Client Secret: From previous step
Enter OAuth Credentials in PipesHub
  1. Click “Next” to move forward

​
Step 5: Authorize the Connection

  1. After saving the configuration, click “Authenticate GitHub to Proceed”
  2. You’ll be redirected to GitHub’s authorization page
GitHub sign-in redirect
  1. Enter GitHub login credentials and click Sign In
  2. User will be shown the set of permissions needed, click Authorize to grant permissions
GitHub permissions page
  1. You’ll be redirected back to PipesHub with a success message
  2. The connector status will update to show “Connected — you can continue to Configure records”
GitHub OAuth completed

​
Step 6: Enable and Configure the Connector

  1. After successful authentication, click “Continue to configuration” to activate the connector
  2. A configuration dialog will appear with a section:
    • Sync Settings - Configure synchronization strategy and schedule
Sync settings GitHub Connector

​
Sync Settings

Configure your synchronization preferences:
  1. Sync Strategy: Choose between “Scheduled” or “Manual”
  2. Sync Interval: Choose how often to sync (default: 60 minutes)
Scheduled sync runs automatically at the specified intervals, keeping your data up-to-date without manual intervention. Manual sync requires you to trigger synchronization on-demand.

​
Save and Activate

  1. Click “Save Configuration” to save your configuration.
  2. User will be prompted to begin synchronization.
  3. The connector will verify credentials and begin initial synchronization
  4. Monitor the Indexing Progress to track sync completion

​
Supported Features

The GitHub connector syncs the following data from your GitHub repositories:
  • Repositories: All accessible repositories
  • Issues: Full issue content including description, status, comments
  • Pull Requests: Full pull request content including description, merge status, commit history, review comments, comments and file changes
  • Comments: All comments on issues and pull requests with author information
  • Attachments: Files attached to issues, pull requests (images, documents, etc.)

​
Troubleshooting

​
Common Issues

Invalid client credentials error:
  • Verify Client ID and Client Secret are correct
  • Ensure you copied the full values without extra spaces
  • Check that the OAuth app is active in GitHub Developer Settings
  • Regenerate the secret if necessary and update PipesHub
Callback URL mismatch error:
  • Ensure the Redirect URI in PipesHub exactly matches the callback URL in GitHub
  • Check for trailing slashes or protocol differences (http vs https)
  • Update both configurations to use the same URL
Authorization failed:
  • Verify you’re signing in with a valid GitHub account that has access to the required repositories
No data syncing:
  • Verify the connector status shows “Active”
  • Check that the authenticating user has owned repositories
  • Review sync logs for specific error messages
Token expired or sync stopped:
  • Disable and re-enable the connector to re-authenticate
  • Check if the OAuth app is still active in GitHub Developer Settings
If you modify OAuth application settings in GitHub Developer Settings (Client ID, Secret, callback URL), you must update the configuration in PipesHub and re-authorize the connection.

​
Connector Workflow

The GitHub connector follows a structured synchronization process to ensure all data is accurately synced with proper permissions.

​
Sync Overview

The run_sync method executes a complete synchronization cycle in a specific order:

​
Full Sync vs Incremental Sync

The connector automatically determines whether to perform a full or incremental sync:Full Sync occurs when:
  • First-time synchronization (no sync checkpoint exists)
  • Manual full sync is triggered
Incremental Sync occurs when:
  • A valid sync checkpoint exists
  • Uses the saved sync checkpoint to fetch or filter for records updated since the last sync time

​
Issue and Pull Requests Synchronization

GitHub considers all Pull requests as issues.
For each repository, the connector fetches issues and pull requests from the GitHub API. Each record is normalized into a hierarchical BlockGroup layout inside a BlocksContainer: the description, conversation, and (for pull requests) code-review context are kept in separate block groups so search and retrieval can target the right slice of content.

​
Body, comments, and attachments

For each issue and pull request, the connector:
  • Description: Stores the body as the first BlockGroup (index 0). Markdown is preserved and inline images are embedded in the indexed representation.
  • Comments: Stores each top-level comment as its own BlockGroup in thread order.
  • Attachments and images: Applies the same extraction and placement rules for issues and pull requests—files and images are mapped to the description or the specific comment they belong to.
Pull requests use the same body, comment, and attachment handling as issues. The section below lists additional content indexed only for pull requests.

​
Additionally for pull requests

  • Commits: Commit SHAs (and related commit context) are synced as dedicated blocks so answers can be tied to the commits on the PR.
  • Changed files: For files touched by the PR, the connector indexes file content together with the diff (patch) for that change.
  • Review comments on the diff: Per-file review comments on the PR changes—including line-associated feedback where GitHub provides it—are indexed separately from regular issue/PR conversation comments.
  • Reviews: PR-level review submissions (for example approve, request changes, or comment-only review) are indexed as distinct material from inline review-comment threads.

​
Sync pipeline (summary)

  1. Fetch all issues and pull requests for the repository (respecting full vs incremental sync rules above).
  2. Batch records for processing.
  3. Build BlockGroups for description and comments; map attachments and images to the description or the correct comment.
  4. For pull requests only, add blocks for commits, changed files with diffs, review comments on changes, and review submissions.
Comments are stored as BlockGroups within the parent issue or pull request BlocksContainer (not as separate top-level records). That keeps threads organized and ensures comments are removed when the parent record is deleted.

​
Checkpoint Management

After each successful sync:
  • Stores the latest issue update timestamp as checkpoint for each repository

​
FAQ

Common causes:
  1. Email mismatch - The email used during connector authentication doesn’t match your PipesHub account email. This causes permission issues preventing you from accessing synced data.
    • Fix: Reconfigure the connector and authenticate using the same email as your PipesHub account.
  2. Data not yet indexed - The connector is still syncing or hasn’t completed the initial indexing process.
    • Fix: Go to Settings > Connectors, check the sync status, and wait for indexing to complete.
The initial sync duration depends on the size of your GitHub repositories:
The most common causes are missing scopes or selecting the wrong GitHub account. Re-run authentication, ensure the PipesHub email and GitHub account email matches.

​
Ready to Get Started?

Connect your GitHub account to PipesHub in just a few minutes. Follow the step-by-step guide above to enable repository issues and pull-requests search across all your GitHub content.