> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipeshub.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Storage Service API

> Documentation for the Storage Service API

# Storage Service API

The Storage Service provides robust file storage, versioning, and retrieval capabilities across your organization's infrastructure. This service abstracts away the underlying storage vendor implementation, allowing seamless interaction with files regardless of where they're physically stored.

## Base URL

All endpoints are prefixed with `/api/v1/document`

## Authentication

All endpoints require authentication via Bearer token:

```http theme={null}
Authorization: Bearer YOUR_TOKEN
```

**Internal endpoints** use scoped token authentication for service-to-service communication with specific scopes:

* `STORAGE_TOKEN` - For storage operations
* `FETCH_CONFIG` - For configuration updates

## Architecture Overview

The Storage Service is built on a Node.js backend with MongoDB for metadata persistence. It provides a flexible storage interface that supports three storage vendor backends:

1. **Amazon S3** - For cloud-based storage on AWS
2. **Azure Blob Storage** - For cloud-based storage on Microsoft Azure
3. **Local Storage** - For on-premises file storage

The service integrates with other components such as:

* **Configuration Manager** - Manages storage configuration settings stored in etcd
* **IAM Service** - Handles user authentication and authorization
* **Key-Value Store** - Provides access to configuration data from etcd

## Storage Configuration

Storage configuration is maintained in etcd under the `/services/storage` path. The configuration determines which storage vendor is active and the credentials needed to access it. The service dynamically loads this configuration at runtime and can switch between storage vendors without restarting.

## Authentication Modes

The Storage Service offers two authentication modes:

1. **User Authentication** - Uses standard user JWT tokens for operations performed by end users
2. **Service Authentication** - Uses scoped tokens with specific permissions for service-to-service communication

## Data Models

### Document

The Document model represents metadata about files stored in the system, including:

* Basic information (name, path, size, MIME type)
* Versioning metadata and history
* Storage location details
* Permissions and ownership
* Custom metadata

### Version History

Each versioned document maintains a complete history of changes, including:

* Previous versions of the document
* Metadata for each version (size, creation time, creator)
* Storage locations for each version
* Version notes and labels

## API Endpoints

<AccordionGroup>
  <Accordion title="Document Management">
    Core document operations including upload, retrieval, and deletion.

    <AccordionGroup>
      <Accordion title="POST /upload - Upload Document">
        Upload a new document to the storage service.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/upload`

            **Request Body Parameters:**

            | Parameter             | Type   | Required | Description                                               |
            | --------------------- | ------ | -------- | --------------------------------------------------------- |
            | file                  | File   | Yes      | The file to upload (multipart/form-data)                  |
            | documentName          | string | Yes      | Name of the document (no extensions allowed)              |
            | documentPath          | string | No       | Optional path for document storage                        |
            | alternateDocumentName | string | No       | Alternative name for the document                         |
            | permissions           | string | No       | Access permissions (owner, editor, commentator, readonly) |
            | customMetadata        | object | No       | Custom metadata to associate with the document            |
            | isVersionedFile       | string | Yes      | Whether the document supports versioning ("true"/"false") |

            **File Size Limits:**

            * User endpoints: 1GB (1024 \* 1024 \* 1000 bytes)
            * Internal endpoints: 100MB (1024 \* 1024 \* 100 bytes)

            **Direct Upload Threshold:** 0MB (all files trigger direct upload for S3/Azure for large files)

            **Validation Rules:**

            * Document name cannot contain extensions or forward slashes
            * File must have a valid extension with supported MIME type
            * File buffer is required in multipart form data

            **Supported MIME Types:** Extensive list including documents (PDF, Office), images (JPEG, PNG, SVG), archives (ZIP, RAR), and many others.
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK` (normal upload) or `308 Permanent Redirect` (large files requiring direct upload)

            **Normal Upload Response:**

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c12345",
              "documentName": "presentation",
              "documentPath": "org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345",
              "alternateDocumentName": null,
              "isVersionedFile": true,
              "sizeInBytes": 2048576,
              "extension": ".pptx",
              "storageVendor": "s3",
              "s3": {
                "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/current/presentation.pptx"
              },
              "versionHistory": [
                {
                  "version": 0,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v0.pptx"
                  },
                  "createdAt": 1626955195000,
                  "size": 2048576,
                  "extension": ".pptx"
                }
              ],
              "createdAt": 1626955195000,
              "updatedAt": 1626955195000,
              "isDeleted": false,
              "currentVersion": 0,
              "mutationCount": 1
            }
            ```

            **Large File Response (308 Redirect):**

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c67890",
              "documentName": "large-video",
              "documentPath": "media/videos",
              "isVersionedFile": false,
              "extension": ".mp4",
              "storageVendor": "s3"
            }
            ```

            **Headers for large files:**

            * `Location`: Presigned URL for direct upload
            * `x-document-id`: Document ID for reference
            * `x-document-name`: Document name
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /internal/upload - Internal Upload">
        Internal endpoint for service-to-service document uploads.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/internal/upload`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Request Body:** Same structure as public upload endpoint.

            **File Size Limit:** 100MB for internal uploads.
          </Tab>

          <Tab title="Response">
            Same response structure as public upload endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /placeholder - Create Document Placeholder">
        Create a document placeholder for future direct uploads.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/placeholder`

            **Request Body Parameters:**

            | Parameter             | Type    | Required | Description                                  |
            | --------------------- | ------- | -------- | -------------------------------------------- |
            | documentName          | string  | Yes      | Name of the document (no extensions allowed) |
            | documentPath          | string  | Yes      | Path for document storage                    |
            | alternateDocumentName | string  | No       | Alternative name for the document            |
            | permissions           | string  | No       | Access permissions                           |
            | metaData              | object  | No       | Custom metadata                              |
            | isVersionedFile       | boolean | No       | Whether the document supports versioning     |
            | extension             | string  | Yes      | File extension (e.g., "pdf")                 |

            ```json theme={null}
            {
              "documentName": "large-dataset",
              "documentPath": "analytics/data",
              "isVersionedFile": false,
              "extension": "csv",
              "permissions": "owner"
            }
            ```

            **Validation:**

            * Document name cannot contain extensions or forward slashes
            * Extension must be provided separately
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c67890",
              "documentName": "large-dataset",
              "documentPath": "analytics/data",
              "isVersionedFile": false,
              "extension": ".csv",
              "storageVendor": "s3",
              "createdAt": 1626955245000,
              "updatedAt": 1626955245000,
              "isDeleted": false,
              "currentVersion": 0
            }
            ```
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /internal/placeholder - Internal Create Placeholder">
        Internal endpoint for creating document placeholders.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/internal/placeholder`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Request Body:** Same structure as public placeholder endpoint.
          </Tab>

          <Tab title="Response">
            Same response structure as public placeholder endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /:documentId - Get Document By ID">
        Retrieve document metadata by ID.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/:documentId`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Request Headers:**

            * `Authorization`: Bearer token (required)

            **Request Body:** Contains fileBuffer field for validation (per DocumentIdParams schema)
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c12345",
              "documentName": "presentation",
              "documentPath": "org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345",
              "alternateDocumentName": null,
              "isVersionedFile": true,
              "sizeInBytes": 2048576,
              "extension": ".pptx",
              "mimeType": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
              "storageVendor": "s3",
              "s3": {
                "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/current/presentation.pptx"
              },
              "versionHistory": [
                {
                  "version": 0,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v0.pptx"
                  },
                  "createdAt": 1626955195000,
                  "size": 2048576,
                  "extension": ".pptx"
                }
              ],
              "permissions": "owner",
              "initiatorUserId": "507f1f77bcf86cd799439011",
              "createdAt": 1626955195000,
              "updatedAt": 1626955195000,
              "isDeleted": false,
              "currentVersion": 0,
              "mutationCount": 1
            }
            ```
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /internal/:documentId - Internal Get Document">
        Internal endpoint for retrieving document metadata.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/internal/:documentId`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.
          </Tab>

          <Tab title="Response">
            Same response structure as public get document endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="DELETE /:documentId/ - Delete Document">
        Mark a document as deleted (soft delete).

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `DELETE /api/v1/document/:documentId/`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Note:** The endpoint includes a trailing slash in the route definition.
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c12345",
              "documentName": "presentation",
              "isDeleted": true,
              "deletedByUserId": "5fb3b7c8e1d8f32a94c98765",
              "updatedAt": 1626958795000
            }
            ```
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="DELETE /internal/:documentId/ - Internal Delete">
        Internal endpoint for deleting documents.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `DELETE /api/v1/document/internal/:documentId/`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.
          </Tab>

          <Tab title="Response">
            Same response structure as public delete endpoint.
          </Tab>
        </Tabs>
      </Accordion>
    </AccordionGroup>
  </Accordion>

  <Accordion title="Document Download & Access">
    Operations for downloading and accessing document content.

    <AccordionGroup>
      <Accordion title="GET /:documentId/download - Download Document">
        Get a signed URL to download a document or stream it for local storage.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/:documentId/download`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Query Parameters:**

            | Parameter               | Type   | Required | Description                                                                            |
            | ----------------------- | ------ | -------- | -------------------------------------------------------------------------------------- |
            | version                 | string | No       | Version number to download (transformed to number, must be > 0)                        |
            | expirationTimeInSeconds | string | No       | Expiration time for the signed URL (transformed to number, must be > 0, default: 3600) |

            **Validation:**

            * Version is transformed from string to number and must be greater than 0 if provided
            * expirationTimeInSeconds is transformed from string to number and must be greater than 0 if provided
            * Version must exist in document's version history
            * Only versioned documents can specify version parameter
            * Storage vendor must match current configuration
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            **For S3/Azure (JSON Response):**

            ```json theme={null}
            {
              "signedUrl": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/current/presentation.pptx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=..."
            }
            ```

            **For Local Storage:** File is streamed directly with appropriate headers:

            * `Content-Type`: Based on file extension via getMimeType()
            * `Content-Disposition`: `attachment; filename="documentName.extension"`
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /internal/:documentId/download - Internal Download">
        Internal endpoint for downloading documents.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/internal/:documentId/download`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Query Parameters:** Same as public download endpoint.
          </Tab>

          <Tab title="Response">
            Same response structure as public download endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /:documentId/buffer - Get Document Buffer">
        Get the document's content as a buffer.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/:documentId/buffer`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Query Parameters:**

            | Parameter | Type   | Required | Description                                                |
            | --------- | ------ | -------- | ---------------------------------------------------------- |
            | version   | string | No       | Version number to retrieve (transformed to number, min: 0) |

            **Validation:**

            * Version is transformed from string to number via `.pipe(z.number().min(0).optional())`
            * Version must exist if specified
            * Version cannot exceed available version history length
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            **Response:** Binary buffer data of the document content.

            **Error Response (when buffer retrieval fails):**

            ```json theme={null}
            "Failed to get buffer"
            ```
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /internal/:documentId/buffer - Internal Get Buffer">
        Internal endpoint for retrieving document buffers.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/internal/:documentId/buffer`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Query Parameters:** Same as public buffer endpoint.
          </Tab>

          <Tab title="Response">
            Same response as public buffer endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="PUT /:documentId/buffer - Update Document Buffer">
        Update a document's content.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `PUT /api/v1/document/:documentId/buffer`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Request Body Parameters:**

            * `file`: File to upload (multipart/form-data with field name 'file')

            **File Size Limit:** 100MB (1024 \* 1024 \* 100 bytes)

            **Validation:** Uses DocumentIdParams schema which includes fileBuffer in body

            **Behavior:**

            * Updates the document content in storage
            * Increments mutation count
            * Updates document size metadata
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/current/presentation.pptx"
            ```

            **Response:** Storage URL of the updated document.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="PUT /internal/:documentId/buffer - Internal Update Buffer">
        Internal endpoint for updating document buffers.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `PUT /api/v1/document/internal/:documentId/buffer`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Request Body:** Same as public buffer update endpoint.
          </Tab>

          <Tab title="Response">
            Same response as public buffer update endpoint.
          </Tab>
        </Tabs>
      </Accordion>
    </AccordionGroup>
  </Accordion>

  <Accordion title="Document Versioning">
    Version control operations for document lifecycle management.

    <AccordionGroup>
      <Accordion title="POST /:documentId/uploadNextVersion - Upload Next Version">
        Upload a new version of an existing document.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/:documentId/uploadNextVersion`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Request Body Parameters:**

            | Parameter          | Type   | Required | Description                                                       |
            | ------------------ | ------ | -------- | ----------------------------------------------------------------- |
            | file               | File   | Yes      | The new file content (multipart/form-data with field name 'file') |
            | currentVersionNote | string | No       | Note for the current version (if modified)                        |
            | nextVersionNote    | string | No       | Note for the next version                                         |

            **File Size Limit:** 100MB (1024 \* 1024 \* 100 bytes)

            **Validation:**

            * Document must support versioning (`isVersionedFile: true`)
            * File extension must match existing document extension
            * Document existence and access verification

            **Behavior:**

            * Checks if current document differs from last version
            * If changed, creates version entry for current state with currentVersionNote
            * Uploads new version with nextVersionNote
            * Updates current document content
            * Increments mutation count
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c12345",
              "documentName": "presentation",
              "isVersionedFile": true,
              "mutationCount": 2,
              "sizeInBytes": 2359296,
              "versionHistory": [
                {
                  "version": 0,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v0.pptx"
                  },
                  "createdAt": 1626955195000,
                  "size": 2048576,
                  "extension": ".pptx"
                },
                {
                  "version": 1,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v1.pptx"
                  },
                  "note": "Updated version",
                  "createdAt": 1626958895000,
                  "size": 2359296,
                  "extension": ".pptx",
                  "initiatedByUserId": "5fb3b7c8e1d8f32a94c98765",
                  "mutationCount": 2
                }
              ],
              "updatedAt": 1626958895000
            }
            ```
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /internal/:documentId/uploadNextVersion - Internal Upload Next Version">
        Internal endpoint for uploading next version.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/internal/:documentId/uploadNextVersion`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Request Body:** Same structure as public upload next version endpoint.
          </Tab>

          <Tab title="Response">
            Same response structure as public upload next version endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /:documentId/rollBack - Roll Back to Previous Version">
        Restore a document to a previous version.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/:documentId/rollBack`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Query Parameters:**

            | Parameter | Type   | Required | Description                                                    |
            | --------- | ------ | -------- | -------------------------------------------------------------- |
            | version   | string | Yes      | Version number to roll back to (transformed to number, min: 0) |

            **Request Body Parameters:**

            | Parameter | Type   | Required | Description                       |
            | --------- | ------ | -------- | --------------------------------- |
            | note      | string | Yes      | Note about the rollback operation |

            **Validation:**

            * Uses RollBackToPreviousVersionSchema which extends GetBufferSchema
            * Version comes from query parameter (inherited from GetBufferSchema)
            * Note comes from request body
            * Document must support versioning
            * Target version must exist and be less than current version count

            **Note:** There appears to be a discrepancy between the validator schema (which expects version in query) and the controller implementation (which reads version from body). The schema specification is documented here.
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "_id": "60f8a5b3e6d8f32a94c12345",
              "documentName": "presentation",
              "mutationCount": 3,
              "versionHistory": [
                {
                  "version": 0,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v0.pptx"
                  },
                  "createdAt": 1626955195000,
                  "size": 2048576,
                  "extension": ".pptx"
                },
                {
                  "version": 1,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v1.pptx"
                  },
                  "note": "Updated version",
                  "createdAt": 1626958895000,
                  "size": 2359296,
                  "extension": ".pptx",
                  "initiatedByUserId": "5fb3b7c8e1d8f32a94c98765"
                },
                {
                  "version": 2,
                  "s3": {
                    "url": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/marketing/campaigns/60f8a5b3e6d8f32a94c12345/versions/v2.pptx"
                  },
                  "note": "Rolled back due to errors",
                  "createdAt": 1626959195000,
                  "size": 2048576,
                  "extension": ".pptx",
                  "initiatedByUserId": "5fb3b7c8e1d8f32a94c98765",
                  "mutationCount": 3
                }
              ],
              "sizeInBytes": 2048576,
              "updatedAt": 1626959195000
            }
            ```
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /internal/:documentId/rollBack - Internal Roll Back">
        Internal endpoint for rolling back document versions.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/internal/:documentId/rollBack`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.

            **Request Body:** Same structure as public rollback endpoint.
          </Tab>

          <Tab title="Response">
            Same response structure as public rollback endpoint.
          </Tab>
        </Tabs>
      </Accordion>
    </AccordionGroup>
  </Accordion>

  <Accordion title="Direct Upload & Utilities">
    Direct upload capabilities and utility functions.

    <AccordionGroup>
      <Accordion title="POST /:documentId/directUpload - Get Direct Upload URL">
        Generate a presigned URL for directly uploading to storage.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/:documentId/directUpload`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Request Body:** Empty object (per DirectUploadSchema)

            **Use Case:** For large files that need to be uploaded directly to storage vendor without going through the API server.

            **Validation:**

            * Document and document path must exist
            * Storage vendor configuration must be valid
            * Uses DirectUploadSchema validation
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "signedUrl": "https://bucket.s3.region.amazonaws.com/org123/PipesHub/analytics/data/60f8a5b3e6d8f32a94c67890?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...",
              "documentId": "60f8a5b3e6d8f32a94c67890"
            }
            ```

            **Behavior:**

            * Updates document storage metadata with base URL
            * Presigned URL expires in 1 hour (3600 seconds)
            * Different implementations for S3 vs Azure Blob storage
            * For S3: Updates `document.s3` field
            * For Azure: Updates `document.azureBlob` field
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="POST /internal/:documentId/directUpload - Internal Direct Upload">
        Internal endpoint for generating direct upload URLs.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/internal/:documentId/directUpload`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.
          </Tab>

          <Tab title="Response">
            Same response structure as public direct upload endpoint.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /:documentId/isModified - Check If Document Is Modified">
        Check if a document has been modified since its last version.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/:documentId/isModified`

            **Path Parameters:**

            * `documentId`: MongoDB ObjectId (24-character hex string)

            **Validation:** Uses DocumentIdParams schema

            **Behavior:**

            * Compares current document buffer with latest version in history (or version 0)
            * Returns boolean indicating modification status
            * Uses buffer comparison for accurate change detection
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            true
            ```

            **Response:** Boolean value indicating if document has been modified.
          </Tab>
        </Tabs>
      </Accordion>

      <Accordion title="GET /internal/:documentId/isModified - Internal Check Modified">
        Internal endpoint for checking document modification status.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `GET /api/v1/document/internal/:documentId/isModified`

            **Authentication:** Requires scoped token with `STORAGE_TOKEN` scope.
          </Tab>

          <Tab title="Response">
            Same response as public modified check endpoint.
          </Tab>
        </Tabs>
      </Accordion>
    </AccordionGroup>
  </Accordion>

  <Accordion title="Configuration">
    Internal configuration management endpoints for service administration.

    <AccordionGroup>
      <Accordion title="POST /updateAppConfig - Update App Configuration">
        Internal endpoint for updating application configuration.

        <Tabs>
          <Tab title="Request">
            **Endpoint:** `POST /api/v1/document/updateAppConfig`

            **Authentication:** Requires scoped token with `FETCH_CONFIG` scope.

            **Behavior:**

            * Reloads application configuration from configuration manager
            * Updates storage configuration in dependency injection container
            * Recreates storage controller with new configuration
            * Allows dynamic reconfiguration without service restart
          </Tab>

          <Tab title="Response">
            **Status:** `200 OK`

            ```json theme={null}
            {
              "message": "Storage configuration updated successfully",
              "config": {
                "storage": {
                  "endpoint": "https://storage.example.com",
                  "maxFileSize": 1073741824
                }
              }
            }
            ```
          </Tab>
        </Tabs>
      </Accordion>
    </AccordionGroup>
  </Accordion>
</AccordionGroup>

## Schema Definitions

<Tabs>
  <Tab title="Document Schema">
    ```typescript theme={null}
    interface Document {
      // Basic document information
      documentName: string;
      alternateDocumentName?: string;
      documentPath?: string;
      isVersionedFile: boolean;
      orgId: mongoose.Types.ObjectId;
      
      // Document metrics
      mutationCount?: number;
      sizeInBytes?: number;
      mimeType?: string;
      extension: string;
      
      // Access control
      permissions?: 'owner' | 'editor' | 'commentator' | 'readonly';
      initiatorUserId: mongoose.Types.ObjectId | null;
      
      // Versioning
      versionHistory?: DocumentVersion[];
      currentVersion: number;
      
      // Metadata
      customMetadata?: CustomMetadata[];
      tags?: string[];
      
      // Timestamps
      createdAt?: number;
      updatedAt?: number;
      
      // Deletion status
      deletedByUserId?: mongoose.Types.ObjectId;
      isDeleted: boolean;
      
      // Storage information
      s3?: StorageInfo;
      azureBlob?: StorageInfo;
      local?: StorageInfo;
      storageVendor: StorageVendor;
    }
    ```
  </Tab>

  <Tab title="DocumentVersion Schema">
    ```typescript theme={null}
    interface DocumentVersion {
      version?: number;
      userAssignedVersionLabel?: string;
      s3?: StorageInfo;
      azureBlob?: StorageInfo;
      local?: StorageInfo;
      note?: string;
      extension?: string;
      currentVersion?: number;
      createdAt?: number;
      mutationCount?: number;
      initiatedByUserId?: mongoose.Schema.Types.ObjectId;
      size?: number;
    }
    ```
  </Tab>

  <Tab title="StorageInfo Schema">
    ```typescript theme={null}
    interface StorageInfo {
      url: string;
      localPath?: string;  // Only used for local storage
    }
    ```
  </Tab>

  <Tab title="CustomMetadata Schema">
    ```typescript theme={null}
    interface CustomMetadata {
      key: string;
      value: mongoose.Schema.Types.Mixed;
    }
    ```
  </Tab>

  <Tab title="Storage Vendor Types">
    ```typescript theme={null}
    enum StorageVendor {
      S3 = 's3',
      AzureBlob = 'azureBlob',
      Local = 'local'
    }

    interface FilePayload {
      documentPath: string;
      buffer: Buffer;
      mimeType: string;
      isVersioned: boolean;
    }

    interface StorageServiceResponse<T> {
      statusCode: number;
      data?: T;
      msg?: string;
    }
    ```
  </Tab>

  <Tab title="Validation Schemas">
    ```typescript theme={null}
    // Headers schema used across endpoints
    export const Headers = z.object({
      authorization: z.string(),
    });

    // Query parameter transformations
    const versionTransform = z.string()
      .optional()
      .transform((val) => (val ? Number(val) : undefined))
      .refine((num) => num === undefined || num > 0, {
        message: "version must be greater than zero",
      });

    const expirationTransform = z.string()
      .optional()
      .transform((val) => (val ? Number(val) : undefined))
      .refine((num) => num === undefined || num > 0, {
        message: "expirationTimeInSeconds must be greater than zero",
      });

    // Upload validation with fileBuffer constraints
    export const UploadNewSchema = z.object({
      body: z.object({
        documentName: z.string(),
        documentPath: z.string().optional(),
        alternateDocumentName: z.string().optional(),
        permissions: z.string().optional(),
        customMetadata: z.any().optional(),
        isVersionedFile: z.string(),
        fileBuffer: z.any().optional(),
        fileBuffers: z.array(z.any()).optional(),
      }).refine(
        (data) => data.fileBuffer || data.fileBuffers,
        { message: 'fileBuffer or fileBuffers must be present' }
      ),
      query: z.object({}),
      params: z.object({}),
      headers: Headers,
    });
    ```
  </Tab>
</Tabs>

## Supported MIME Types

The storage service supports an extensive list of MIME types including:

<Tabs>
  <Tab title="Document Formats">
    * **PDF**: `application/pdf`
    * **Microsoft Office**: Word (`.docx`, `.doc`), Excel (`.xlsx`, `.xls`), PowerPoint (`.pptx`, `.ppt`)
    * **OpenDocument**: `.odt`, `.ods`, `.odp`
    * **Text**: `.txt`, `.rtf`, `.csv`, `.md`, `.mdx`
    * **Web**: `.html`, `.css`, `.js`, `.json`, `.xml`
    * **eBooks**: `.epub`
    * **Google Workspace**: `.gdoc`, `.gsheet`, `.gslides`, `.gdraw`
  </Tab>

  <Tab title="Image Formats">
    * **Common**: `.jpg`, `.jpeg`, `.png`, `.gif`, `.svg`, `.webp`
    * **Raw/Professional**: `.tiff`, `.bmp`, `.ico`, `.psd`, `.icns`
    * **Modern**: `.heic`, `.heif`, `.avif`, `.jxl`, `.jp2`, `.jpx`
  </Tab>

  <Tab title="Archive Formats">
    * **Common**: `.zip`, `.rar`, `.7z`, `.tar`, `.gz`
    * **System**: `.cab`, `.rpm`, `.deb`, `.cpio`
    * **Compression**: `.bz2`, `.xz`, `.zst`, `.lz`
  </Tab>

  <Tab title="Media & Other Formats">
    * **Audio**: `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.ape`, `.amr`
    * **Video**: `.mp4`, `.avi`, `.mkv`, `.mov`, `.webm`, `.3gp`
    * **Fonts**: `.ttf`, `.otf`, `.woff`, `.woff2`, `.eot`
    * **CAD/3D**: `.dwg`, `.dxf`, `.stl`, `.glb`, `.gltf`
    * **Scientific**: `.fits`, `.dcm`, `.nii`
  </Tab>
</Tabs>

## Storage Provider Implementation

The Storage Service uses an adapter pattern to abstract the underlying storage provider implementation.

<Tabs>
  <Tab title="Amazon S3">
    **Features:**

    * Direct file uploads/downloads via AWS SDK
    * Signed URLs with configurable expiration (default: 1 hour)
    * Multipart uploads for large files
    * Object versioning support
    * Comprehensive error handling with specific S3 error types

    **Configuration:**

    ```typescript theme={null}
    interface S3StorageConfig {
      accessKeyId: string;
      secretAccessKey: string;
      region: string;
      bucketName: string;
    }
    ```

    **URL Structure:**

    ```
    https://{bucket}.s3.{region}.amazonaws.com/{orgId}/PipesHub/{path}/{documentId}/current/{name}.{ext}
    ```

    **Error Handling:**

    * `StorageConfigurationError` - Missing credentials
    * `StorageUploadError` - Upload failures
    * `StorageDownloadError` - Download failures
    * `PresignedUrlError` - URL generation failures
    * `MultipartUploadError` - Multipart operation failures
  </Tab>

  <Tab title="Azure Blob Storage">
    **Features:**

    * Blob storage operations via Azure SDK
    * SAS token generation for secure access
    * Container-based organization with automatic creation
    * Blob versioning support
    * Stream handling for large files

    **Configuration:**

    ```typescript theme={null}
    interface AzureBlobStorageConfig {
      azureBlobConnectionString?: string;  // Alternative 1
      accountName?: string;                // Alternative 2
      accountKey?: string;                 // Alternative 2
      containerName: string;               // Required
      endpointProtocol?: string;           // Default: https
      endpointSuffix?: string;             // Default: core.windows.net
    }
    ```

    **Limitations:**

    * Multipart uploads not implemented (throws `MultipartUploadError`)
    * Use direct upload instead for large files

    **SAS Token Features:**

    * Read permissions for downloads
    * Write permissions for uploads
    * Configurable expiration (default: 1 hour)
    * Content disposition support for downloads
  </Tab>

  <Tab title="Local Storage">
    **Features:**

    * Filesystem-based storage with OS-specific mount paths
    * File streaming for efficient delivery
    * Directory-based organization
    * Cross-platform path handling (Windows, macOS, Linux)
    * Direct file serving without intermediate storage

    **Configuration:**

    ```typescript theme={null}
    interface LocalStorageConfig {
      mountName: string;  // e.g., "PipesHub"
      baseUrl: string;    // API endpoint base URL
    }
    ```

    **Mount Paths:**

    * **macOS**: `~/Library/{mountName}`
    * **Windows**: `~/AppData/{mountName}`
    * **Linux**: `~/.local/{mountName}`

    **File URLs:**

    ```
    file:///path/to/mount/{orgId}/PipesHub/{path}/{documentId}/current/{name}.{ext}
    ```

    **URL Normalization for Downloads:**

    * Converts `file://` URLs to API download endpoints
    * Pattern: `{baseUrl}/api/v1/document/{documentId}/download`
    * Enables web access to local files through API proxy
  </Tab>
</Tabs>

## File Size Limits & Direct Upload

<Tabs>
  <Tab title="Size Limits">
    **Public Endpoints:**

    * Upload: 1GB (1024 \* 1024 \* 1000 bytes)
    * Buffer operations: 100MB (1024 \* 1024 \* 100 bytes)

    **Internal Endpoints:**

    * All operations: 100MB (1024 \* 1024 \* 100 bytes)

    **Direct Upload Threshold:**

    * Set via `maxFileSizeForPipesHubService` constant
    * Currently configured to 0MB (all files trigger direct upload evaluation)
    * Applied only to S3 and Azure Blob storage
    * Local storage always uses direct API upload
  </Tab>

  <Tab title="Direct Upload Flow">
    1. **Client** uploads file that exceeds threshold
    2. **API** creates document placeholder
    3. **API** returns `308 Permanent Redirect` with presigned URL
    4. **Client** uploads directly to storage vendor using presigned URL
    5. **Storage vendor** handles the upload
    6. **Document metadata** is updated with final storage location

    **Headers in redirect response:**

    * `Location`: Presigned upload URL (expires in 1 hour)
    * `x-document-id`: Document identifier for tracking
    * `x-document-name`: Document name for reference

    **Advantages:**

    * Reduces API server load for large files
    * Improves upload performance
    * Leverages storage vendor's optimized upload handling
  </Tab>
</Tabs>

## Error Handling

All endpoints return structured error responses:

```json theme={null}
{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid request parameters",
    "details": {
      "field": "documentName",
      "issue": "Document name cannot contain extensions"
    }
  },
  "timestamp": "2025-04-27T13:20:00.000Z"
}
```

**Common Error Codes:**

* `VALIDATION_ERROR` - Invalid request parameters
* `NOT_FOUND` - Document not found
* `BAD_REQUEST` - Invalid request format
* `FORBIDDEN` - File format mismatch or access denied
* `INTERNAL_SERVER_ERROR` - Storage service errors

**Storage-Specific Errors:**

* Extension validation failures
* File size limit exceeded
* Storage vendor configuration errors
* Version control constraint violations
* MIME type validation failures

**HTTP Status Codes:**

* `200` - Success
* `308` - Permanent Redirect (for direct uploads)
* `400` - Bad Request
* `401` - Unauthorized
* `403` - Forbidden
* `404` - Not Found
* `500` - Internal Server Error

## Configuration Management

<Tabs>
  <Tab title="Storage Configuration">
    Storage configuration is maintained in etcd under `/services/storage` path:

    ```json theme={null}
    {
      "storageType": "s3|azureBlob|local",
      "credentials": {
        // Provider-specific configuration
      }
    }
    ```

    **Dynamic Configuration:**

    * Service loads configuration at startup and via watch mechanism
    * Can switch storage vendors without service restart
    * Configuration changes trigger adapter reinitialization
    * Graceful handling of configuration errors
  </Tab>

  <Tab title="Service Integration">
    **Key-Value Store (etcd):**

    * Storage configuration management under `/services/storage`
    * Endpoint configuration under `/services/endpoints`
    * Dynamic configuration updates via watch mechanism
    * Service discovery for configuration manager endpoints

    **Configuration Manager:**

    * Centralized configuration storage and validation
    * Configuration update API for dynamic reconfiguration
    * Service-specific configuration routing
    * Configuration versioning and rollback capabilities

    **IAM Service:**

    * User authentication and authorization
    * Organization and user context extraction for multi-tenancy
    * Scoped token validation for internal APIs
    * User existence validation for shared operations
  </Tab>
</Tabs>

## Important Notes

1. **Schema Discrepancy**: There's a mismatch between the rollback validator schema (expects version in query) and controller implementation (reads version from body). Documentation follows schema specification.

2. **File Extensions**: Automatically detected from original filename and validated against comprehensive MIME type mappings.

3. **Version Control**: Optional per document via `isVersionedFile` flag, maintains complete history with metadata and notes.

4. **Storage Abstraction**: Seamless switching between storage vendors via configuration without code changes.

5. **Organization Isolation**: All documents are organization-scoped (`orgId`) for secure multi-tenancy.

6. **Soft Deletion**: Documents are marked as deleted (`isDeleted: true`) rather than physically removed.

7. **Direct Upload**: Large files bypass API server for optimal performance, using vendor-specific presigned URLs.

8. **Internal APIs**: Service-to-service communication with scoped authentication using specific token scopes.

9. **Mutation Tracking**: Documents track change count (`mutationCount`) for auditing and optimization.

10. **MIME Type Validation**: Extensive validation against supported file types with automatic extension detection.

11. **Path Structure**: Organized as `{orgId}/PipesHub/{documentPath}/{documentId}/current|versions/` for clear file organization.

12. **Cross-Platform Support**: Local storage handles Windows, macOS, and Linux filesystem differences transparently.