[Feature] Google Drive Folder-to-Bucket Mapping (External Mount Provider) #6

Summary

Add the ability to map Google Drive folders directly to MiddleOut buckets. This enables users to mount an existing Google Drive folder as an S3-compatible bucket (e.g., s3://gdrive-docs/), letting standard S3 tools (aws-cli, rclone, boto3, SDKs) and the MiddleOut web console transparently browse, read, and write files backed by Google Drive storage.

Problem Statement & Motivation

• Local Quota Constraints: MiddleOut currently stores objects in PostgreSQL with strict limits (8 MB max object, 64 MB account quota). Users wanting to store large assets, backups, or existing datasets hit these limits quickly.
• S3 Gateway for Google Drive: Google Drive is widely used for personal and team storage, but it lacks an S3-compatible interface. By allowing folder-to-bucket mapping:
  • Teams can point existing S3-compatible pipelines directly at Google Drive folders.
  • Users can generate MiddleOut public URLs or share links for files in Google Drive without managing messy GDrive link permissions.
  • Files modified in Google Drive reflect in MiddleOut and vice versa.

Proposed Solution & Architecture

1. Storage Backend Abstraction

Refactor MiddleOut's storage layer (src/lib/storage/) to support Storage Drivers:

• pg (current default: Base64/byte chunks in PostgreSQL)
• gdrive (Google Drive API v3 bridge)

2. Google Drive OAuth2 & Account Connection

• Add an Integrations or Cloud Accounts tab in the MiddleOut Console.
• Users authenticate via Google OAuth2 with scopes:
  • https://www.googleapis.com/auth/drive.file (access to app-created files) or drive.readonly / drive depending on desired mount permissions.
  • Support for Google Service Account JSON keys for headless/server-to-server deployments.

3. Folder-to-Bucket Mapping

When creating a bucket:

• Bucket Type Selection: Standard (Database) vs Google Drive Mount.
• If Google Drive is chosen:
  • Interactive Google Drive folder picker (or manual folder_id input).
  • Read-Only vs Read-Write mount modes.
  • Path translation: S3 keys like folder/subfolder/file.pdf map hierarchically to Drive folder IDs and file items.

4. S3 & Console Operation Mapping

UI / UX Changes

• New Bucket Modal: Add a toggle for "Storage Provider" (Internal Storage vs Google Drive).
• Bucket List & Detail Views: Display a distinct GDrive badge next to mapped buckets.
• Settings / Integrations: Connect/disconnect Google Account, display token status and remaining Google Drive quota.

Implementation Checklist

• Schema Migration: Add provider (enum: 'local' | 'gdrive'), provider_config (jsonb), and external_credentials table to store encrypted OAuth tokens/service account credentials.
• Google Drive Client Service: Implement token refresh, rate-limit backoff, and caching of folder-to-ID mappings.
• S3 Bridge Adapter: Implement S3 operations (list, get, put, delete, head) against the Google Drive v3 API.
• Console UI: Add Google OAuth setup workflow and folder picker to bucket creation dialog.
• Tests: Unit tests with mocked Google Drive API responses; integration test verifying S3 client (aws s3 cp) writing to a mapped GDrive bucket.