# SignSync Web App - Project Specifications

## 1. Product Summary

SignSync is a browser-based practice and comparison tool for sign language interpreters.
Users watch a YouTube source video, record their own interpretation via webcam, save that recording, and share a link so a second interpreter can record against the same source. The app then plays the source plus both interpretation videos in sync for side-by-side review.

## 2. Core Objectives

1. Let interpreters practice against any YouTube video.
2. Capture webcam + microphone recordings directly in-browser.
3. Persist recordings with lightweight user metadata.
4. Generate a shareable link for collaboration.
5. Compare two interpreters in synchronized playback.

## 3. User Roles

1. First interpreter:
Creates an initial recording and share link.
2. Second interpreter:
Opens shared link and adds a second recording.
3. Viewer/reviewer:
Uses playback controls to compare source + interpretations.

## 4. Primary User Flows

### Flow A: Create first recording

1. User opens app.
2. User pastes YouTube URL (or raw video ID) and clicks `Load Video`.
3. User enters name/email.
4. User records interpretation from webcam.
5. User stops recording and previews it.
6. User clicks `Save Recording`.
7. Backend stores metadata + video file and returns unique link ID.
8. Frontend shows full share URL (`?share=<id>`) and copy button.
9. Saved recording appears in "First Recording" panel.

### Flow B: Add second recording from shared link

1. User opens shared link with `?share=<unique_link>`.
2. App loads existing record and cues stored YouTube video.
3. If second recording does not exist:
   - First recording video is shown blurred with overlay text: "Recording done, waiting for other person..."
   - Playback controls (play/pause, timeline seek) are disabled.
   - Form + recorder are shown for the second user.
4. User records and submits second video.
5. Backend updates same row with `name_2`, `email_2`, `recorded_video_path_2`.
6. Frontend refreshes shared data and shows both videos immediately (no reload needed).
7. First video blur is removed and playback controls are enabled.
8. Once both videos exist, form + recorder are hidden.

### Flow C: Synchronized comparison

1. User presses global play/pause button.
2. App controls YouTube + local videos together.
3. Timeline slider seeks all videos simultaneously.
4. First interpretation is always offset +2 seconds (headstart).
5. User can swap left/right interpreter panels.

## 5. Functional Requirements

### 5.1 YouTube Loading

1. Must support:
- Full URLs (`youtube.com/watch?v=...`)
- Short URLs (`youtu.be/...`)
- Embed URLs
- Raw 11-char video IDs
2. Invalid input must show alert: `Please enter a valid YouTube URL`.
3. Default loaded video ID: `wLM5bzt1xks`.
4. Use YouTube IFrame API (`https://www.youtube.com/iframe_api`).
5. Show title + duration once available.

### 5.2 Webcam Recording

1. Request `getUserMedia` with:
- Video: 640x480
- Audio: true
2. Use `MediaRecorder` with preferred mime:
- `video/webm;codecs=vp9` if supported
- fallback `video/webm`
3. Record in 1-second chunks.
4. Show recording timer (`mm:ss`) and active indicator.
5. After stop, create preview player using `URL.createObjectURL`.
6. Provide `Discard` and `Re-record` actions.
7. On camera failure, show:
`Unable to access webcam. Please ensure camera permissions are granted.`

### 5.3 Form Validation

1. Name required (non-empty).
2. Email required and regex-validated in UI.
3. Submission blocked if no recorded blob.
4. Show inline validation errors in form.

### 5.4 Recording Submission Logic

1. New recording (no share context):
- `POST /api/recordings`
- Include `name`, `email`, `video`, `youtubeVideoUrl`
2. Second recording (share context, second not yet present):
- `POST /api/share/:uniqueLink/second-video`
- Include `name`, `email`, `video`
3. On success for first recording:
- Show success message
- Generate and display share URL
- Load saved recording into first video panel immediately (no reload needed)
4. On success for second recording:
- Show success message
- Refresh shared recording data and show second video immediately (no reload needed)
5. On server/network failure:
- Show friendly error message.

### 5.5 Shared Recording Behavior

1. If URL has `share` query param, app must load recording via `GET /api/share/:uniqueLink`.
2. If record includes YouTube URL, cue corresponding source video.
3. If first/second local video exists, load into respective HTML5 video elements.
4. If second recording exists, disable additional recording UI.
5. If second recording does not yet exist (`isSecondUserPending`):
   - First video is blurred (`filter: blur(15px)`) with pointer events disabled.
   - Overlay text "Recording done, waiting for other person..." is shown centered on the blurred video.
   - Play/pause button and timeline slider are disabled.

### 5.6 Playback + Sync

1. Global play button label toggles:
- `▶ Play`
- `⏸ Pause`
2. Pressing play should reset all media to start state:
- YouTube at `0`
- First local video at `2` seconds
- Second local video at `0`
3. Timeline reflects YouTube current time (poll every 250ms while playing).
4. Seek operation:
- YouTube seek to `t`
- First local video seek to `min(t + 2, duration1)`
- Second local video seek to `min(t, duration2)`
5. If user presses play/pause on local videos, YouTube should sync play/pause.

### 5.7 Video Panels and Swap

1. Show two local video panels in one row on desktop.
2. Left panel is "First Recording" and marked with `This video has 2 seconds headstart`.
3. Right panel is "Second Recording".
4. If content missing, show placeholders:
- `No recording yet`
- `Waiting for second recording...`
- `Record your interpretation above` (in shared state with missing second video)
5. Swap button (`⇄`) exchanges displayed first/second recording content.
6. Swap disabled until second recording exists.

## 6. UI/UX Requirements

### 6.1 Page Layout (top to bottom)

1. YouTube URL input + Load button
2. YouTube title + duration
3. Embedded YouTube player
4. User form + webcam recorder (when allowed)
5. Timeline slider with current/duration text
6. Two local recording panels + swap button
7. Global play/pause control

### 6.2 Responsive Behavior

1. Mobile breakpoint around `600px`.
2. URL input stack vertically on mobile.
3. Local video panels stack vertically on mobile.
4. Swap button rotates 90 degrees on mobile.

### 6.3 Visual Style Baseline

1. Dark UI theme.
2. Blue primary actions (`#2563eb`).
3. Red recording actions (`#e63946`).
4. Rounded cards, slider, and buttons.

## 7. Technical Architecture

### 7.1 Frontend

1. Stack: React + Vite.
2. Main components:
- `YouTubePlayer` (orchestrator + sync + state)
- `WebcamRecorder` (capture and preview)
- `UserForm` (metadata input + validation)
3. Frontend API base URL: `http://localhost:3001`.
4. Share state is URL-driven (`window.location.search`).

### 7.2 Backend

1. Stack: Node.js + Express + SQLite.
2. Middleware:
- `cors()`
- `express.json()`
- `multer` for multipart uploads
3. Upload constraints:
- max file size: 100MB
- accept only `video/*` mimetypes
4. File naming pattern:
- `recording_<timestamp>.webm`
5. Storage path:
- `public/media/` (local filesystem)

### 7.3 Data Persistence Model

Single table: `recordings`

- `id` INTEGER PK AUTOINCREMENT
- `unique_link` TEXT UNIQUE NOT NULL
- `name` TEXT NOT NULL
- `email` TEXT NOT NULL
- `recorded_video_path` TEXT NOT NULL
- `youtube_video_url` TEXT NOT NULL
- `name_2` TEXT NULL
- `email_2` TEXT NULL
- `recorded_video_path_2` TEXT NULL
- `created_at` DATETIME DEFAULT CURRENT_TIMESTAMP

Unique link generation: 8 random bytes hex string.

## 8. Backend API Contract

### `POST /api/recordings`

Creates first recording.

Multipart fields:
- `name` (required)
- `email` (required)
- `youtubeVideoUrl` (optional, stored as empty string if missing)
- `video` file (required)

Success `200`:
- `success`
- `id`
- `uniqueLink`
- `recordedVideoPath`
- `message`

Validation failure `400`:
- `error: Name, email, and video file are required`

### `GET /api/recordings`

Returns all recordings ordered by newest first.

### `GET /api/recordings/:id`

Returns one recording by numeric ID.

`404` if not found.

### `GET /api/share/:uniqueLink`

Returns one recording by share token.

`404` if not found.

### `POST /api/share/:uniqueLink/second-video`

Adds second interpreter video to existing row.

Multipart fields:
- `name` (required)
- `email` (required)
- `video` file (required)

Errors:
- `404` recording not found
- `400` second video already recorded
- `400` missing required fields

Success `200`:
- `success`
- `recordedVideoPath2`
- `message`

## 9. Error Handling and Edge Cases

1. Invalid YouTube URL input blocks load.
2. Camera permission denied shows inline error.
3. Missing recording on submit blocked client-side.
4. Backend returns generic `500` on server exceptions.
5. Shared link with invalid token should show not-found behavior (current app logs and remains mostly empty).
6. If local video shorter than seek target, clamp to video duration.
7. Newly uploaded video files may not be immediately available from Vite's dev server. A `loadWithRetry` mechanism retries `.load()` up to 5 times at 500ms intervals using native `addEventListener('error')` on the video element.
8. WebM files from `MediaRecorder` lack seek indices. Setting `currentTime` before metadata loads crashes the demuxer. Only set `currentTime` in `onLoadedMetadata` handlers.
9. React's synthetic `onError` prop on `<video>` elements is unreliable for media errors. Use native event listeners instead.

## 10. Browser/Platform Requirements

1. Modern Chromium/Safari/Firefox with support for:
- `navigator.mediaDevices.getUserMedia`
- `MediaRecorder`
- `URL.createObjectURL`
- Clipboard API (`navigator.clipboard.writeText`)
2. Desktop-first, mobile supported via responsive CSS.

## 11. Security and Privacy Baseline

1. No authentication.
2. No authorization; share link is access mechanism.
3. No encryption-at-rest beyond host defaults.
4. User email stored in plaintext SQLite.
5. CORS enabled broadly (default permissive).
6. Uploaded files are stored locally and directly web-accessible via `/media/...`.

## 12. Local Development and Run Commands

1. Install dependencies: `npm install`
2. Run frontend + backend: `npm run dev:all`
3. Frontend only: `npm run dev` (Vite default port `5173`)
4. Backend only: `npm run server` (port `3001`)

## 13. Implementation Notes for Another AI Builder

1. Keep functionality equivalent to this baseline rather than redesigning feature behavior.
2. Preserve the 2-second headstart offset for first recording in all sync/seek/play reset paths.
3. Keep shared-link mode logic URL-driven.
4. Ensure form+recorder are hidden once both recordings exist.
5. Use multipart uploads and SQLite schema exactly unless explicitly changing architecture.
6. If deploying production, ensure `/media` static files are served from the same origin expected by frontend paths.
7. Video loading after upload uses a belt-and-suspenders approach: callback refs trigger `loadWithRetry` when video elements mount, and backup `useEffect` hooks watching `sharedRecording.recorded_video_path` / `recorded_video_path_2` catch edge cases where the callback ref alone is insufficient.
8. Always clean up previous native error listeners before adding new ones on the same video element to prevent duplicate retry loops.
9. Never set `currentTime` on a video element before its metadata has loaded — use `onLoadedMetadata` handlers instead.

## 14. Acceptance Criteria

1. User can load a YouTube video and see title/duration.
2. User can record webcam video with audio and preview it.
3. User can submit first recording and receive usable share link.
4. Shared link opens same source video and first recording.
5. Second user can add second recording exactly once.
6. App can play/pause/seek source + local videos in sync.
7. First recording consistently plays with +2s offset.
8. User can swap left/right recording panels.
9. UI works on desktop and mobile widths.
10. After saving first recording, video appears immediately in the first video panel without page reload.
11. After saving second recording, video appears immediately in the second video panel without page reload.
12. Second user sees first video blurred with overlay text before recording, with playback controls disabled.