Skip to main content
Released September 14, 2026 · Stable · View on GitHub 0.8.0 brings 103 merged PRs since 0.7.0, and most of the weight is in the indexing pipeline. Each connector now gets its own lane into indexing, so one connector’s backlog no longer blocks every other connector. The consumers size their concurrency from what Neo4j, Redis and the parsing and embedding services can actually take, and they back off when those services struggle. A record whose event is lost or abandoned no longer sits in QUEUED forever. Redis access moves behind one connection provider per backend, which prepares PipesHub for Redis Cluster and AWS MemoryDB and makes TLS settings apply everywhere. The release also scopes more things to the organization: MD5 deduplication, connector access, agents, signed URLs, local storage and notifications. Python services now accept only real service tokens as service credentials. Chat messages move out of the conversation documents into their own collections through a one-time migration. Read the upgrade notes before you upgrade: several Redis, TLS and vector-store settings now fail at startup instead of being silently ignored.

​
Highlights

​
Fair scheduling: every connector gets its own lane into indexing

All record events used to share one queue that indexing drained in arrival order. The first connector to start syncing took the whole pipeline until it finished, and a newly connected source showed no progress at all while an unrelated backlog cleared. Producers now place each record event in a lane chosen from its connector. The indexing consumer reads every lane and dispatches with Deficit Round Robin over per-connector virtual queues, on both Kafka and Redis Streams (#3126 — @abhishekgml1991).

​
An indexing pipeline that sizes itself to what downstream can take

Indexing concurrency used to be derived from CPU alone, with nothing tied to Neo4j’s connection pool, the lease Redis or the parsing and embedding services. The total index budget is now 6 × CPU, between 24 and 96. Downstream symptoms (a 429, an exhausted Neo4j or Redis pool, an unreachable service, repeated timeouts, rising hold time) feed a resource governor that cuts the pools within one sample and holds growth until the service recovers. Heavy documents (PDF, Office, images) and light ones (pages, issues) are admitted per tier, so attachments queued behind slow parses no longer starve the light pool. A cancelled graph transaction rolls back instead of leaking its Neo4j connection, and waiting for a parse slot no longer uses up a record’s processing budget (#3108, #3170, #3174 — @abhishekgml1991). The full write-up is in docs/indexing-service.md.

​
No more records stuck in QUEUED

A record could be stored QUEUED before its event was published, or acknowledged by a consumer that gave up on it, and then nothing ever looked at it again. New connector records now start as NOT_STARTED and become QUEUED only once their event is acknowledged, and the stranded-record sweep is on by default (#3163). A consumer that abandons a message first marks the record FAILED with a reason (#3140), the sweep no longer re-queues healthy records whose timestamps looked old, and storage uploads can be retried safely (#3274) — all @abhishekgml1991. Documents are no longer silently dropped when indexing fails to start (#3226 — @shekharkadyan), and a message that crashes its consumer is dead-lettered by its delivery count (#3037 — @blockgroot).

​
Redis: one connection provider, ready for Redis Cluster and AWS MemoryDB

Node.js and Python each build Redis connections through a single provider (IRedisConnectionProvider). REDIS_MODE selects standalone (the default) or cluster, which speaks Redis Cluster and MemoryDB. The REDIS_TLS* settings are now honoured by every Redis client, REDIS_KEY_NAMESPACE isolates keys, and the crawling queue moves to a hash-tagged key prefix so it works on a cluster (#3155, #3254 — @abhishekgml1991). Several of these changes need attention on upgrade; see below.

​
Vector collection strategy and per-org deduplication

A new VECTOR_COLLECTION_STRATEGY setting can group records into one vector collection per connector type instead of a single collection, with vector membership tracked per connector and record group. MD5 deduplication is now scoped to one organization, and connector access and deletion follow clearer rules (#3112 — @abhishekgml1991).

​
Chat messages get their own collections

Messages are split out of the conversation and agent-conversation documents into chatSessions and chatSessionMessages. New chats write there straight away, and a one-time backfill copies existing conversations on first boot (#3148 — @tushar1245).

​
S3 with an instance IAM role

S3 storage no longer needs an access key and secret: when both are left empty, PipesHub uses the AWS default credential chain, so a deployment on EC2 or ECS can run with its instance role. The key pair is all-or-nothing, the health check uses the same resolution as the provider and says which mode it took, and onboarding makes the fields optional (#3143, #3144 — @abhishekgml1991). The S3 health check also verifies content and signed URLs (#3141).

​
Breaking changes & upgrade notes

  • Redis TLS is now enforced wherever a Redis client is created. Before, only part of the code read REDIS_TLS. A deployment running REDIS_TLS=true against a plaintext Redis will now fail to connect; check that your endpoint supports TLS, or unset the variable. REDIS_TLS_ENABLED and REDIS_TLS_REJECT_UNAUTHORIZED accept true/1/yes/on and false/0/no/off, and any other value fails startup. REDIS_TLS_REJECT_UNAUTHORIZED=false is refused when Redis credentials are set; use REDIS_TLS_CA_PATH instead, which is now read as a file and fails startup if it cannot be read. REDIS_TLS_ENABLED is OR-ed with the TLS flag stored in the admin UI for most clients, but the Node.js Redis Streams and key-value store clients read only REDIS_TLS_ENABLED: if you turned TLS on only in the admin UI, also set REDIS_TLS_ENABLED=true (#3155).
  • REDIS_DB is deprecated. It still works with REDIS_MODE=standalone but fails startup in any other mode; use REDIS_KEY_NAMESPACE for isolation. REDIS_KEY_NAMESPACE together with MESSAGE_BROKER=redis is rejected at startup, because it does not cover Redis Streams names (#3155).
  • The crawling queue moves from the bull key prefix to {crawling}. The first boot after upgrade removes the old keys and re-creates every connector sync schedule from its stored configuration. If the connector service is not reachable within the startup retry window, the schedules are not re-created until the Node.js service restarts: check that your scheduled syncs are back after the upgrade, and restart the Node.js service if they are not (#3155).
  • VECTOR_DB_TYPE=redis is gated on VECTOR_REDIS_MODE (which defaults to REDIS_MODE), so the app can run on MemoryDB while the vector index stays on a standalone Redis (#3155).
  • New VECTOR_COLLECTION_STRATEGY setting (default single, the existing behaviour). It is read only on the first startup that finds nothing stored; after that the stored value wins, and a value that contradicts it fails startup. Changing it on a live deployment is an embedding rebuild, not a config edit (#3112).
  • MD5 deduplication is scoped to one organization. Content that was deduplicated across orgs re-indexes once per org after the upgrade, so expect a one-off rise in indexing volume and vector-store size. Existing records are not rewritten (#3112).
  • Connector deletion is “an administrator, or whoever created it”, for team and personal connectors alike, and connector lookups are scoped to the caller’s organization (#3112).
  • The stranded-record sweep is on by default (STRANDED_RECORD_REPUBLISH_AFTER_SECONDS, 3600s); it was off (#3163).
  • New and changed indexing settings. MAX_CONCURRENT_INDEXING still overrides the new index budget, and MAX_CONCURRENT_PARSING / MAX_CONCURRENT_INDEXING should normally stay unset. NEO4J_MAX_CONNECTION_POOL_SIZE (default 100) is new. Real Neo4j transactions are available behind NEO4J_EXPLICIT_TRANSACTIONS=true, off by default: try it on staging first and watch the indexing logs for DeadlockDetected. The parsing client’s read timeout now follows RECORD_PROCESSING_TIMEOUT (#3170).
  • A document the parsing service cannot parse returns 422, not 500, so it fails as a document error on the first attempt instead of tripping the parsing circuit breaker (#3170).
  • Chat storage migration. The first boot copies every conversation and agent conversation into chatSessions and chatSessionMessages. The copy is resumable per document and per collection, and the legacy conversations and agentconversations collections are kept, no longer written to. A legacy document without an orgId stops the migration instead of being copied without one (#3148).
  • Python services accept a token signed with the scoped secret only as a service token with the connector:signedUrl, record:content, conversation:create or fetch:config scope. Refresh, password-reset, email-verification, mail, storage and user-lookup tokens now get 401 there. Refresh, password-reset and verification tokens are signed with a key derived from the scoped secret that the Python services never hold (#3283).
  • Query-service tuning variables now reach the service. QUERY_UVICORN_WORKERS (new, default 1), PIPESHUB_AGENT_TRANSPORT, PIPESHUB_ACCESSIBLE_RECORDS_CACHE, PIPESHUB_ACCESSIBLE_RECORDS_CACHE_TTL and PIPESHUB_SIGNED_URL_CACHE_SECONDS were ignored by Compose and Helm before. Every default keeps the current behaviour (#3113 — @shrikantNemiwal1).

​
Connectors

  • Google Drive and Gmail — blocking Google API calls move onto a shared thread pool (#2886); items outside the workspace are fetched (#3052); streaming a record from a shared drive works in the personal Drive connector (#3160) — all @R0gue-one.
  • Jira — the Jira connector was refactored (#2775 — @darshangodase). Jira Data Center and Linear send notifications when a sync, a project or team, a permission read or an attachment sync fails (#3123 — @darshangodase).
  • Confluence — “Grant Jira user access” defaults to Yes (#3137 — @tushar1245).
  • GitLab — permission fixes for confidential work items (#3248 — @darshangodase).
  • OneDrive — blocked users are skipped during sync (#3164 — @R0gue-one).
  • Notion — an inherited OAuth app is handled, with better logging (#3222), and nested data sources are handled (#3252) — @shrikantNemiwal1.
  • Slack — the connector icon is served from the path it declares (#3107).
  • All connectors — operating-system metadata files are filtered out before indexing (#3147 — @R0gue-one); the connector factory can unregister a connector (#3162 — @vishwjeet-pawar03); the connector stats endpoint is fixed (#3133 — @vishwjeet-pawar03).

​
Agents & AI

  • Coding sandbox — the sandbox that runs agent code was restructured into Docker, settings, registry and governor modules (#3149 — @abhishekgml1991).
  • Embeddings — embedding calls get a timeout and a shared retry policy (#3161 — @abhishekgml1991), and chunks are sized by tokens against the embedding model’s limit instead of by characters, so a long block no longer fails the whole record (#3286 — @rbales79).
  • Agent builder — a correct default reasoning effort, with its locale keys (#3166 — @vishwjeet-pawar03, #3173 — @Pranjal-wq); an agent with no knowledge source attached says so (#3269 — @rbales79).
  • MCP — fallback MCP configurations are fixed (#3221 — @vishwjeet-pawar03).
  • Retrieval — retrieved images are kept after transport deduplication (#3244 — @TatshSiow).

​
Fixes

  • Users and workspaces — when auto-join is off, people can ask to join a workspace, with notify and decision emails (#3130, #3158 — @Pranjal-wq); Groups move out of People, with permission placeholders for the open-source edition (#3068 — @Pranjal-wq); inactive users are left out of team member lists (#3171 — @darshan-pipeshub); the invite button is disabled when SMTP is not configured (#3249 — @tushar1245); org signup answers 400, not 500, for validation errors (#3098 — @dineshyadav03).
  • Knowledge base — fetching a deleted record no longer answers 500 (#3246 — @Saneajayy).
  • Configuration — AI model and storage settings fixes (#3145) and a rework of how stored secrets are redacted in settings responses (#3091) — @vanshg685; the vector-store actions in Labs show under the right condition (#3127 — @vishwjeet-pawar03).
  • Graph databases — the ArangoDB app schema and an optional Knowledge Hub search parameter (#3119 — @shrikantNemiwal1); agent-template soft delete on ArangoDB passes the right arguments (#3125 — @Anai-Guo).
  • Logging — the query service logs unhandled exceptions in its auth middleware instead of swallowing them (#3224 — @L4XB); a Slack request-verification log line is removed (#3217 — @tushar1245).
  • Smaller fixes in the token manager, the installer, the graph providers and the environment templates (#3288 — @abhishekgml1991).

​
Security & authentication

  • Service tokens and URL fetches — the Python services stop accepting any scoped-secret JWT as a user credential, and the Skills URL import can no longer fetch arbitrary server-side URLs (#3283 — @abhishekgml1991).
  • Cross-org isolation — another org can no longer chat as an org’s service agent or probe whether one exists, signed-URL minting and download check identity, record org and permissions, and local storage rejects any path that leaves the org’s folder (#3118 — @Pranjal-wq). Further org-id fixes in sign-in, mail and configuration (#3109 — @vanshg685), and notification lookups filter by org (#3216 — @R0gue-one).
  • Sessions and sign-in — a role change ends the user’s sessions in every workspace that uses the same email (#3169 — @Pranjal-wq); the one-time code logic is hardened (#3102 — @rish664); the OAuth client secret is no longer returned when JIT provisioning is disabled (#3099).
  • Toolsets — toolset routes no longer fall back to client-supplied headers for the user and org identity (#3103 — @igennova).
  • Reporting — the address for security vulnerability reports changed (#3276 — @rish664).

​
Developer experience & infrastructure

  • Installer and Compose — download progress no longer repeats in the terminal (#3086); a MongoDB segfault no longer fails the install or risks data (#3100); the manual quick-start produces a working stack (#3105); a first-run script creates the organization and writes a personal access token to a file without printing it (#3182).
  • Docs — the README states what PipesHub is once, at the top (#3168), puts the curl install command first (#3089), drops Bitbucket from the code-search claim (#3088), and moves file-format support and the tech stack to the FAQ and docs (#3256, #3257); the contributor setup guide, the manual Compose examples and the connector playbook are corrected (#3104, #3106, #3111); the Spanish deployment FAQ is updated (#3273 — @defawan); the 0.7.0 changelog was added (#3142 — @rish664).
  • Integration and end-to-end tests — Groups Playwright specs (#3087 — @Pranjal-wq); test fixes and updates (#3090 — @jatingaur18, #3101 — @abhishekgml1991, #3156 — @vanshg685, #3172 — @siddhantota); org response validation (#3129 — @R0gue-one); GitHub, GitLab and GitHub Teams environments (#3157, #3146 — @darshangodase).
  • Dependencies — Node.js API: nodemailer 9.1.1, sharp 0.35.4, morgan, js-yaml, hono, multer and @xmldom/xmldom (#3209, #3227, #3211, #3228, #3229, #3210, #3205); frontend: sharp and next, js-yaml, vitest, browserslist, fast-uri, @xmldom/xmldom and @humanfs/node (#3213, #3212, #3207, #3132, #3150, #3152, #3151); Python and integration tests: nltk, thrift and snowflake-connector-python (#3135, #3134, #3131, #3124) — all @dependabot[bot].

​
New contributors

  • @blockgroot made their first contribution in #3037
  • @dineshyadav03 made their first contribution in #3098
  • @Anai-Guo made their first contribution in #3125
  • @darshan-pipeshub made their first contribution in #3171
  • @L4XB made their first contribution in #3224
  • @rbales79 made their first contribution in #3269
  • @defawan made their first contribution in #3273
Full commit-level detail: GitHub release · Compare view