# babylonjs-effect

**Repository Path**: hejiuzui/babylonjs-effect

## Basic Information

- **Project Name**: babylonjs-effect
- **Description**: 使用babylonjs，创造3d粒子效果
- **Primary Language**: TypeScript
- **License**: LGPL-3.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-06-27
- **Last Updated**: 2026-07-22

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Video Cut — GPU-Accelerated Game Highlight Editor

**Electron + PixiJS + WebCodecs** — render cinematic game highlight reels entirely in-process with GPU-accelerated effects. No ffmpeg dependency, no external transcoders.

> 专注于王者荣耀击杀高光视频的自动剪辑与特效渲染，生成抖音/剪映风格卡点短视频。

## Features

### 🎬 Headless Render Engine
- **Zero ffmpeg** — decode via `<video>`, encode via WebCodecs H.264, mix audio via Web Audio API (`OfflineAudioContext`)
- **PixiJS v8 scene graph** — layered rendering: background → video → overlays (particles, text, bloom, mist, lens flares) → UI
- **Slow-motion time-remapping** — dynamic LUT-based speed ramping with hero profile: `normal → whip → stutter → crawl → recover`
- **Frame interpolation** — alpha-blended cross-fade between frames during slow-motion for smooth playback
- **Per-kill highlight broadcast** — 60-frame highlight window with cascading particle waves, shockwaves, speed lines, and camera shake

### 🎨 Visual Effects (Douyin/CapCut Grade)

| Effect | Description |
|---|---|
| **Radial Chromatic Aberration** | Combo-scaled RGB offset via center-anchored sprite scaling |
| **Multi-layer Bloom** | Dual blur layers for rich glow (combo boosts intensity) |
| **Dynamic Color Grading** | 3-phase warm/cold/intense color washes via multiply blend |
| **Particle Burst System** | 6 texture types: glow dots, stars, diamonds, speed lines, ring fragments, light columns |
| **Vortex & Ring Fragments** | Orbits for multi-kill moments |
| **Mist / Cloud Drift** | 3-layer parallax mist with organic multi-frequency sine drift |
| **Lens Flares** | Horizontal colored streaks (pink, cyan, gold) with slow drift |
| **Light Sweep** | Diagonal bright beam at highlight entry |
| **Film Grain** | 16-frame seeded noise pool, overlay-blended |
| **Hex Bokeh** | Screen-blended hexagonal bokeh texture |
| **Vignette Breathing** | Subtle dark edge pulse during highlights |
| **Dreamy Haze** | Radial soft haze that clears progressively |
| **Time-Freeze Rings** | 3 concentric cyan rings during slow-motion |
| **Floating Embers** | Warm golden particles during deep slow-motion crawl phase |
| **Shutter Flash** | Subliminal full-white single-frame flash |
| **Color Wash** | Transient magenta/purple impact overlay |
| **Dutch Angle & Camera Shake** | 2° tilt with exponential decay, shake scaled by combo |
| **Ghost Trail (灵魂出窍)** | 4-frame ring buffer for delayed composite trails |
| **Speed-Effect Coupling** | Whip/stutter/crawl/recover phases each gate unique visual treatments |
| **Beat Synchronization** | RMS energy envelope + onset detection syncs effects to BGM peaks |

### 📐 Template System
8+ curated templates with per-effect configuration:

| Template | Style | Use Case |
|---|---|---|
| `cinematic-gold` | 抖音黄金时刻 | Golden high-contrast, warm, vignette-heavy |
| `neon-cyber` | 霓虹赛博 | Cyberpunk cyan/magenta, glitch effects |
| `minimal-clean` | 简洁干净 | Clean white/cyan, minimal particles |
| `extreme-speed` | 极速节奏 | Fast cuts, aggressive shake, speed lines |
| `cinematic-epic` | 史诗电影 | Widescreen bars, slow-motion dramatic |
| `fps-highlights` | FPS高光 | First-person shooter specific |
| `moba-clutch` | MOBA团战 | Multi-kill combos, team-fight focus |
| `huya-live` | 直播间高光 | Live streaming overlay style |

### 🧠 AI-Assisted Features
- **Auto kill detection** — ONNX-based frame analysis for highlight extraction
- **Rhythm analysis** — BGM energy envelope → music structure segments (intro/build-up/drop/outro)
- **Smart transition** — beat-synced segment transitions with tension computation
- **OCRS detection** — Tesseract.js for in-game kill feed text recognition

### 🖥️ Interactive Editor (React + PixiJS)
- Timeline-based clip editing with GSAP animations
- Live PixiJS preview with effect layering
- Frame scheduler for precise per-frame effect sequencing
- Babylon.js optional 3D effect overlay layer
- Danmaku (弹幕) overlay system
- WebRTC screen/window capture for live recording

## Architecture

```
src/
├── headless/           # Server-side render engine (Electron main process)
│   ├── main.ts         # Render loop: decode → effect → encode pipeline
│   ├── rhythm.ts       # Rhythm state, speed phases, filter layer system
│   ├── headless-api.ts # IPC API types
│   └── gen-fixtures.ts # Test fixture generator
├── renderer/           # Browser-side editor (React + PixiJS)
│   ├── babylon/        # 3D effect overlay (Babylon.js)
│   ├── danmaku/        # Bullet chat overlay
│   ├── effects/        # Effect layer system
│   ├── gsap/           # GSAP animation integrations
│   ├── layers/         # Composition layers
│   ├── live/           # Live recording (WebRTC)
│   ├── particles/      # Particle system
│   ├── FrameScheduler.ts  # Per-frame effect scheduling
│   ├── PixiPreview.ts     # PixiJS preview canvas
│   ├── RenderPipeline.ts  # Full render pipeline orchestration
│   └── TransitionManager.ts # Segment transitions
├── config/
│   └── templates.ts    # Effect template definitions
├── codecs/             # Video/audio codec wrappers
│   ├── encoder.ts      # WebCodecs H.264 encoder
│   ├── audio-extractor.ts  # MP4 audio track extraction
│   └── audio-mixer.ts  # Web Audio API mixing
├── ai/                 # AI inference (ONNX runtime)
├── components/         # React UI components
├── hooks/              # React hooks
├── types/              # TypeScript type definitions
├── test/               # E2E and integration tests
└── workers/            # Web Workers (AI, effects)

electron/               # Electron main process setup
├── main.js             # App entry, IPC, file dialogs
└── preload.js          # Context bridge

dist-electron/          # Compiled Electron output
scripts/                # Build/dev/batch scripts
tests/                  # Playwright E2E tests
```

## Render Pipeline

```
Input MP4 → <video> decode → PixiJS scene (effects + particles) → Canvas2D capture → WebCodecs encode → H.264 MP4
                                                                       ↕
                                                            Frame cache (optional)
```

### Slow-Motion Speed Profile
```
hlDist 0:   normal (1.0×)    — let flash/stutter happen
hlDist 1-3: whip (1.5×)      — speed UP into slowmo (sucks viewer in)
hlDist 4:   stutter (0.02×)  — near-freeze, frame hold
hlDist 5-22: crawl (0.02→1×) — ease-out cubic with overshoot bounce, ethereal effects
hlDist 22+: normal (1.0×)    — recovered
```

## Quick Start

```bash
# Install dependencies
npm install

# Run in development mode (Electron + Vite HMR)
npm run electron:dev

# Build for production
npm run build

# Run batch processing
npm run batch -- --input ./clips --template moba-clutch --bgm ./music/beat.mp3
```

## Scripts

| Script | Description |
|---|---|
| `dev` | Vite dev server (browser) |
| `build` | TypeScript + Vite build |
| `electron:dev` | Electron dev with HMR |
| `lint` | TypeScript type-check |
| `batch` | Batch process videos with template profiles |
| `test` | Vitest unit tests |
| `test:e2e` | Playwright E2E tests |
| `test:auto-edit` | End-to-end auto-edit pipeline smoke test |
| `test:gen-fixtures` | Generate test video fixtures |

## Batch Processing

```bash
# Process all MP4s in a directory with a specific template
npm run batch -- --input ./clips --output ./output --template moba-clutch

# With background music
npm run batch -- --input ./clips --template cinematic-gold --bgm ./music/epic.mp3

# Using presets
npm run batch:neon      # neon-cyber template
npm run batch:speed     # extreme-speed template
npm run batch:fps       # fps-highlights template
```

## Cross-Platform

- **Windows** — primary target (tested)
- **macOS / Linux** — Electron-compatible, build scripts available
- **GPU acceleration** — WebGL via PixiJS, WebCodecs video encode

## Tech Stack

| Layer | Technology |
|---|---|
| Desktop Shell | Electron 35 |
| Render | PixiJS v8 (WebGL), optional Babylon.js 3D |
| UI | React 19, TypeScript |
| Video Codec | WebCodecs (mediabunny VideoEncoderEngine) |
| Audio | Web Audio API (OfflineAudioContext) |
| Visual Effects | Canvas2D, PixiJS filters (blur, blend modes) |
| AI Inference | ONNX Runtime Web (Tesseract.js, custom models) |
| Build | Vite 6, TypeScript 5.9 |
| Testing | Vitest, Playwright |

## License

MIT