ShipAI Docs
Codebase

Data Flow

Request lifecycle and cross-package interactions for chat, files, billing, and background jobs.

This page documents the main runtime paths in ship-ai.

Read this with code

Use this page together with route handlers in apps/app/src/app/api/* and corresponding package modules to trace real execution paths.

1) Chat Request Flow

Route entrypoint

POST /api/chat in apps/app/src/app/api/chat/route.ts authenticates and delegates to buildChatStream(...).

Pipeline execution

Context assembly, memory, search/deep-search branching, and stream generation are coordinated inside chat builder modules.

Persistence and accounting

Post-finish handlers persist state and usage/billing telemetry for subsequent limits and analytics.

Execution Stages Inside Chat Builder

Core behavior in apps/app/src/lib/ai/chat/build-chat-stream.ts:

  1. preflight and guards
    • derive chat operation (chat or deep_search)
    • call checkUsageLimits(...) from @ai/billing
    • load or create chat in DB (@ai/db)
  2. context assembly
    • fetch prior messages
    • optimize history for context window
    • build memory context
    • compose system instruction + selected context
  3. attachment and document processing
  4. routing and handler execution
    • deep-search branch or standard branch
  5. streaming response
  6. post-finish persistence and accounting

2) Resumable Stream Flow

  • Active stream id is persisted on chat record.
  • GET /api/chat/[id]/stream validates ownership and resumes via resumable-stream.
  • Pub/sub backing uses Redis through @ai/kv/client.

3) File Upload and Storage Flow

Typical upload URL path:

  1. POST /api/files/upload-url
  2. validate auth + mime + filename + size
  3. enforce upload and file-size limits via @ai/billing
  4. generate presigned URL using @ai/storage
  5. client uploads object to S3-compatible storage

4) Instruments and Jobs Flow

Transcription

  1. POST /api/instruments/transcription
  2. validate file ownership/type/size + billing limits
  3. create transcription DB record and mark processing
  4. enqueue BullMQ job via @ai/jobs
  5. worker runs transcription and writes results back to DB

5) Billing and Subscription Flow

Checkout path:

  1. POST /api/billing/checkout
  2. verify plan + existing subscription state
  3. create/reuse Stripe customer (@ai/stripe)
  4. create checkout session

Webhook path:

  1. POST /api/billing/webhook
  2. verify Stripe signature
  3. dedupe event (Redis key)
  4. apply subscription/payment state changes in DB

6) Feature and Service Gating

Feature behavior is environment-driven through @ai/features:

  • product flags (FEATURE_*) gate capabilities like search, deep search, memory, artifacts, and transcription
  • service flags (SERVICE_*) represent deployed infra availability
  • development mode can bypass billing enforcement where configured

7) Data Ownership Summary

  • Database: canonical state (@ai/db)
  • Redis: cache/rate-limit/queue/stream coordination (@ai/kv, BullMQ)
  • Object storage: uploaded file blobs (@ai/storage)
  • Workers: asynchronous processing (@ai/jobs)
  • Tracing/logging: observability across request and workflow boundaries (@ai/tracing, @ai/logger)

Packages

Package-level ownership map for shared platform modules.

Features

Architecture-first reference for core product features in ShipAI.

On this page

1) Chat Request FlowRoute entrypointPipeline executionPersistence and accountingExecution Stages Inside Chat Builder2) Resumable Stream Flow3) File Upload and Storage Flow4) Instruments and Jobs FlowTranscription5) Billing and Subscription Flow6) Feature and Service Gating7) Data Ownership Summary