diff --git a/project-specifications.md b/project-specifications.md new file mode 100644 index 0000000..362c822 --- /dev/null +++ b/project-specifications.md @@ -0,0 +1,343 @@ +# SignSync Web App - Project Specifications + +## 1. Product Summary + +SignSync is a browser-based practice and comparison tool for sign language interpreters. +Users watch a YouTube source video, record their own interpretation via webcam, save that recording, and share a link so a second interpreter can record against the same source. The app then plays the source plus both interpretation videos in sync for side-by-side review. + +## 2. Core Objectives + +1. Let interpreters practice against any YouTube video. +2. Capture webcam + microphone recordings directly in-browser. +3. Persist recordings with lightweight user metadata. +4. Generate a shareable link for collaboration. +5. Compare two interpreters in synchronized playback. + +## 3. User Roles + +1. First interpreter: +Creates an initial recording and share link. +2. Second interpreter: +Opens shared link and adds a second recording. +3. Viewer/reviewer: +Uses playback controls to compare source + interpretations. + +## 4. Primary User Flows + +### Flow A: Create first recording + +1. User opens app. +2. User pastes YouTube URL (or raw video ID) and clicks `Load Video`. +3. User enters name/email. +4. User records interpretation from webcam. +5. User stops recording and previews it. +6. User clicks `Save Recording`. +7. Backend stores metadata + video file and returns unique link ID. +8. Frontend shows full share URL (`?share=`) and copy button. +9. Saved recording appears in "First Recording" panel. + +### Flow B: Add second recording from shared link + +1. User opens shared link with `?share=`. +2. App loads existing record and cues stored YouTube video. +3. If second recording does not exist: + - First recording video is shown blurred with overlay text: "Recording done, waiting for other person..." + - Playback controls (play/pause, timeline seek) are disabled. + - Form + recorder are shown for the second user. +4. User records and submits second video. +5. Backend updates same row with `name_2`, `email_2`, `recorded_video_path_2`. +6. Frontend refreshes shared data and shows both videos immediately (no reload needed). +7. First video blur is removed and playback controls are enabled. +8. Once both videos exist, form + recorder are hidden. + +### Flow C: Synchronized comparison + +1. User presses global play/pause button. +2. App controls YouTube + local videos together. +3. Timeline slider seeks all videos simultaneously. +4. First interpretation is always offset +2 seconds (headstart). +5. User can swap left/right interpreter panels. + +## 5. Functional Requirements + +### 5.1 YouTube Loading + +1. Must support: +- Full URLs (`youtube.com/watch?v=...`) +- Short URLs (`youtu.be/...`) +- Embed URLs +- Raw 11-char video IDs +2. Invalid input must show alert: `Please enter a valid YouTube URL`. +3. Default loaded video ID: `wLM5bzt1xks`. +4. Use YouTube IFrame API (`https://www.youtube.com/iframe_api`). +5. Show title + duration once available. + +### 5.2 Webcam Recording + +1. Request `getUserMedia` with: +- Video: 640x480 +- Audio: true +2. Use `MediaRecorder` with preferred mime: +- `video/webm;codecs=vp9` if supported +- fallback `video/webm` +3. Record in 1-second chunks. +4. Show recording timer (`mm:ss`) and active indicator. +5. After stop, create preview player using `URL.createObjectURL`. +6. Provide `Discard` and `Re-record` actions. +7. On camera failure, show: +`Unable to access webcam. Please ensure camera permissions are granted.` + +### 5.3 Form Validation + +1. Name required (non-empty). +2. Email required and regex-validated in UI. +3. Submission blocked if no recorded blob. +4. Show inline validation errors in form. + +### 5.4 Recording Submission Logic + +1. New recording (no share context): +- `POST /api/recordings` +- Include `name`, `email`, `video`, `youtubeVideoUrl` +2. Second recording (share context, second not yet present): +- `POST /api/share/:uniqueLink/second-video` +- Include `name`, `email`, `video` +3. On success for first recording: +- Show success message +- Generate and display share URL +- Load saved recording into first video panel immediately (no reload needed) +4. On success for second recording: +- Show success message +- Refresh shared recording data and show second video immediately (no reload needed) +5. On server/network failure: +- Show friendly error message. + +### 5.5 Shared Recording Behavior + +1. If URL has `share` query param, app must load recording via `GET /api/share/:uniqueLink`. +2. If record includes YouTube URL, cue corresponding source video. +3. If first/second local video exists, load into respective HTML5 video elements. +4. If second recording exists, disable additional recording UI. +5. If second recording does not yet exist (`isSecondUserPending`): + - First video is blurred (`filter: blur(15px)`) with pointer events disabled. + - Overlay text "Recording done, waiting for other person..." is shown centered on the blurred video. + - Play/pause button and timeline slider are disabled. + +### 5.6 Playback + Sync + +1. Global play button label toggles: +- `▶ Play` +- `⏸ Pause` +2. Pressing play should reset all media to start state: +- YouTube at `0` +- First local video at `2` seconds +- Second local video at `0` +3. Timeline reflects YouTube current time (poll every 250ms while playing). +4. Seek operation: +- YouTube seek to `t` +- First local video seek to `min(t + 2, duration1)` +- Second local video seek to `min(t, duration2)` +5. If user presses play/pause on local videos, YouTube should sync play/pause. + +### 5.7 Video Panels and Swap + +1. Show two local video panels in one row on desktop. +2. Left panel is "First Recording" and marked with `This video has 2 seconds headstart`. +3. Right panel is "Second Recording". +4. If content missing, show placeholders: +- `No recording yet` +- `Waiting for second recording...` +- `Record your interpretation above` (in shared state with missing second video) +5. Swap button (`⇄`) exchanges displayed first/second recording content. +6. Swap disabled until second recording exists. + +## 6. UI/UX Requirements + +### 6.1 Page Layout (top to bottom) + +1. YouTube URL input + Load button +2. YouTube title + duration +3. Embedded YouTube player +4. User form + webcam recorder (when allowed) +5. Timeline slider with current/duration text +6. Two local recording panels + swap button +7. Global play/pause control + +### 6.2 Responsive Behavior + +1. Mobile breakpoint around `600px`. +2. URL input stack vertically on mobile. +3. Local video panels stack vertically on mobile. +4. Swap button rotates 90 degrees on mobile. + +### 6.3 Visual Style Baseline + +1. Dark UI theme. +2. Blue primary actions (`#2563eb`). +3. Red recording actions (`#e63946`). +4. Rounded cards, slider, and buttons. + +## 7. Technical Architecture + +### 7.1 Frontend + +1. Stack: React + Vite. +2. Main components: +- `YouTubePlayer` (orchestrator + sync + state) +- `WebcamRecorder` (capture and preview) +- `UserForm` (metadata input + validation) +3. Frontend API base URL: `http://localhost:3001`. +4. Share state is URL-driven (`window.location.search`). + +### 7.2 Backend + +1. Stack: Node.js + Express + SQLite. +2. Middleware: +- `cors()` +- `express.json()` +- `multer` for multipart uploads +3. Upload constraints: +- max file size: 100MB +- accept only `video/*` mimetypes +4. File naming pattern: +- `recording_.webm` +5. Storage path: +- `public/media/` (local filesystem) + +### 7.3 Data Persistence Model + +Single table: `recordings` + +- `id` INTEGER PK AUTOINCREMENT +- `unique_link` TEXT UNIQUE NOT NULL +- `name` TEXT NOT NULL +- `email` TEXT NOT NULL +- `recorded_video_path` TEXT NOT NULL +- `youtube_video_url` TEXT NOT NULL +- `name_2` TEXT NULL +- `email_2` TEXT NULL +- `recorded_video_path_2` TEXT NULL +- `created_at` DATETIME DEFAULT CURRENT_TIMESTAMP + +Unique link generation: 8 random bytes hex string. + +## 8. Backend API Contract + +### `POST /api/recordings` + +Creates first recording. + +Multipart fields: +- `name` (required) +- `email` (required) +- `youtubeVideoUrl` (optional, stored as empty string if missing) +- `video` file (required) + +Success `200`: +- `success` +- `id` +- `uniqueLink` +- `recordedVideoPath` +- `message` + +Validation failure `400`: +- `error: Name, email, and video file are required` + +### `GET /api/recordings` + +Returns all recordings ordered by newest first. + +### `GET /api/recordings/:id` + +Returns one recording by numeric ID. + +`404` if not found. + +### `GET /api/share/:uniqueLink` + +Returns one recording by share token. + +`404` if not found. + +### `POST /api/share/:uniqueLink/second-video` + +Adds second interpreter video to existing row. + +Multipart fields: +- `name` (required) +- `email` (required) +- `video` file (required) + +Errors: +- `404` recording not found +- `400` second video already recorded +- `400` missing required fields + +Success `200`: +- `success` +- `recordedVideoPath2` +- `message` + +## 9. Error Handling and Edge Cases + +1. Invalid YouTube URL input blocks load. +2. Camera permission denied shows inline error. +3. Missing recording on submit blocked client-side. +4. Backend returns generic `500` on server exceptions. +5. Shared link with invalid token should show not-found behavior (current app logs and remains mostly empty). +6. If local video shorter than seek target, clamp to video duration. +7. Newly uploaded video files may not be immediately available from Vite's dev server. A `loadWithRetry` mechanism retries `.load()` up to 5 times at 500ms intervals using native `addEventListener('error')` on the video element. +8. WebM files from `MediaRecorder` lack seek indices. Setting `currentTime` before metadata loads crashes the demuxer. Only set `currentTime` in `onLoadedMetadata` handlers. +9. React's synthetic `onError` prop on `