MogensR commited on
Commit
d4f305e
Β·
1 Parent(s): f05e4da

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +286 -31
README.md CHANGED
@@ -1,33 +1,288 @@
1
- ---
2
- title: Video Background Replacement
3
- emoji: 🎬
4
- colorFrom: blue
5
- colorTo: green
6
- sdk: gradio
7
- sdk_version: 4.44.1
8
- app_file: app.py
9
- pinned: false
10
- license: mit
11
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
- # 🎬 Video Background Replacement
14
- Advanced video background replacement using **SAM2 + MatAnyone** for professional-quality results.
15
-
16
- ## ✨ Features
17
- - **High-Quality Segmentation**: Precise foreground/background separation
18
- - **Two Processing Modes**:
19
- - **Single-Stage**: Fast, direct background replacement
20
- - **Two-Stage**: Green screen intermediate for **cinema-quality edges**
21
- - **Multiple Background Options**:
22
- - Upload custom images
23
- - Use professional presets (gradients, solid colors)
24
- - Generate AI backgrounds (experimental)
25
- - **GPU Optimized**: Efficient processing with **CUDA support** (NVIDIA GPUs)
26
- - **User-Friendly Interface**: Simple Gradio UI with real-time progress tracking
27
-
28
- ## πŸš€ How to Use
29
-
30
- ### **Option 1: Run with Docker (Recommended)**
31
- 1. **Build the Docker image**:
32
  ```bash
33
- docker build -t video-bg-replacement .
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Video Background Replacement
2
+
3
+ Professional-quality video background replacement using SAM2 + MatAnyone for precise segmentation and cinema-grade compositing.
4
+
5
+ ## Features
6
+
7
+ ### Core Capabilities
8
+ - **High-Quality Segmentation**: SAM2-powered person detection with sub-pixel accuracy
9
+ - **Advanced Matting**: MatAnyone integration for professional edge refinement
10
+ - **Dual Processing Modes**:
11
+ - **Single-Stage**: Direct background replacement (fast)
12
+ - **Two-Stage**: Green screen intermediate for broadcast-quality results
13
+ - **Multiple Background Options**: Custom uploads, professional presets, procedural generation
14
+ - **GPU Accelerated**: CUDA optimization for NVIDIA GPUs with CPU fallback
15
+
16
+ ### Technical Highlights
17
+ - Keyframe-based processing with temporal consistency
18
+ - Automatic memory management and error recovery
19
+ - Real-time progress tracking with ETA estimation
20
+ - Audio preservation throughout processing
21
+ - Robust codec fallback system
22
+
23
+ ## Quick Start
24
+
25
+ ### Option 1: Docker (Recommended)
26
+
27
+ ```bash
28
+ # Clone repository
29
+ git clone <your-repo-url>
30
+ cd video-background-replacement
31
+
32
+ # Build and run with GPU support
33
+ docker build -t video-bg-replacement .
34
+ docker run --gpus all -p 7860:7860 video-bg-replacement
35
+ ```
36
+
37
+ ### Option 2: Local Installation
38
+
39
+ ```bash
40
+ # Clone repository
41
+ git clone <your-repo-url>
42
+ cd video-background-replacement
43
+
44
+ # Create virtual environment
45
+ python -m venv venv
46
+ source venv/bin/activate # On Windows: venv\Scripts\activate
47
+
48
+ # Install dependencies
49
+ pip install -r requirements.txt
50
+
51
+ # Run application
52
+ python app.py
53
+ ```
54
+
55
+ ## System Requirements
56
+
57
+ ### Minimum Requirements
58
+ - Python 3.10+
59
+ - 8GB RAM
60
+ - 4GB storage space
61
+ - FFmpeg installed
62
+
63
+ ### Recommended for Best Performance
64
+ - NVIDIA GPU with 6GB+ VRAM
65
+ - 16GB+ RAM
66
+ - CUDA 12.1+ support
67
+ - Fast SSD storage
68
 
69
+ ### Supported Platforms
70
+ - Linux (Ubuntu 20.04+, tested)
71
+ - Windows 10/11 with WSL2
72
+ - macOS (CPU-only, limited testing)
73
+
74
+ ## Usage Guide
75
+
76
+ ### Basic Workflow
77
+
78
+ 1. **Launch Application**
 
 
 
 
 
 
 
 
 
79
  ```bash
80
+ python app.py
81
+ ```
82
+ Access web interface at `http://localhost:7860`
83
+
84
+ 2. **Load Models** (first-time setup)
85
+ - Click "Load Models" button
86
+ - Wait for SAM2 and MatAnyone to download and initialize
87
+ - Status will show "Models loaded and validated"
88
+
89
+ 3. **Process Video**
90
+ - Upload your video file (MP4, AVI, MOV supported)
91
+ - Choose background method:
92
+ - **Professional Presets**: Studio-quality backgrounds
93
+ - **Custom Upload**: Your own background image
94
+ - Select processing options:
95
+ - **Two-Stage Mode**: Better quality, slower processing
96
+ - **Quality Preset**: Fast/Balanced/High
97
+ - Click "Process Video"
98
+
99
+ ### Processing Modes
100
+
101
+ #### Single-Stage Mode (Default)
102
+ - Direct background replacement
103
+ - Faster processing (2-5x speed)
104
+ - Good quality for most use cases
105
+ - Recommended for: Social media, quick edits, testing
106
+
107
+ #### Two-Stage Mode (Premium)
108
+ - Green screen intermediate step
109
+ - Cinema-quality edge compositing
110
+ - Advanced chroma key algorithms
111
+ - Recommended for: Professional content, broadcast, film
112
+
113
+ ### Background Options
114
+
115
+ #### Professional Presets
116
+ - `office_modern`: Clean contemporary office
117
+ - `studio_blue`: Broadcast-quality blue background
118
+ - `studio_green`: Professional green screen replacement
119
+ - `minimalist`: Clean white gradient
120
+ - `warm_gradient`: Warm sunset atmosphere
121
+ - `tech_dark`: Modern tech/gaming setup
122
+
123
+ #### Custom Backgrounds
124
+ - Upload any image (JPG, PNG supported)
125
+ - Automatically resized to match video resolution
126
+ - Best results with high-resolution images (1920x1080+)
127
+
128
+ ## Configuration
129
+
130
+ ### Environment Variables
131
+
132
+ ```bash
133
+ # Model settings
134
+ export MODEL_CACHE_DIR="/path/to/model/cache"
135
+ export FORCE_CPU="false"
136
+ export DISABLE_MATANYONE="false"
137
+
138
+ # Processing settings
139
+ export KEYFRAME_INTERVAL="5"
140
+ export FRAME_SKIP="1"
141
+ export QUALITY_PRESET="balanced"
142
+
143
+ # Video encoding
144
+ export OUTPUT_CODEC="mp4v"
145
+ export CRF="18"
146
+ ```
147
+
148
+ ### Quality Presets
149
+
150
+ | Preset | Speed | Quality | Use Case |
151
+ |--------|-------|---------|----------|
152
+ | `fast` | 3x faster | Good | Social media, previews |
153
+ | `balanced` | Normal | High | General use |
154
+ | `high` | 2x slower | Excellent | Professional content |
155
+
156
+ ## API Reference
157
+
158
+ ### Core Functions
159
+
160
+ ```python
161
+ from app import process_video_fixed, load_models_with_validation
162
+
163
+ # Load models
164
+ status = load_models_with_validation()
165
+
166
+ # Process video
167
+ result_path, message = process_video_fixed(
168
+ video_path="input.mp4",
169
+ background_choice="studio_blue",
170
+ custom_background_path=None,
171
+ use_two_stage=False,
172
+ chroma_preset="standard"
173
+ )
174
+ ```
175
+
176
+ ### Two-Stage Processing
177
+
178
+ ```python
179
+ from two_stage_processor import TwoStageProcessor
180
+
181
+ processor = TwoStageProcessor(sam2_predictor, matanyone_model)
182
+
183
+ # Full pipeline
184
+ result_path, message = processor.process_full_pipeline(
185
+ video_path="input.mp4",
186
+ background=background_image,
187
+ final_output="output.mp4",
188
+ chroma_settings={"tolerance": 40, "edge_softness": 2}
189
+ )
190
+ ```
191
+
192
+ ## Troubleshooting
193
+
194
+ ### Common Issues
195
+
196
+ **CUDA Out of Memory**
197
+ ```bash
198
+ # Reduce processing quality
199
+ export QUALITY_PRESET="fast"
200
+ export KEYFRAME_INTERVAL="8"
201
+ ```
202
+
203
+ **Models Not Loading**
204
+ ```bash
205
+ # Clear cache and retry
206
+ rm -rf /tmp/model_cache
207
+ python app.py
208
+ ```
209
+
210
+ **Video Processing Fails**
211
+ - Check video format (MP4 recommended)
212
+ - Ensure video is not corrupted
213
+ - Try shorter clips first (under 30 seconds)
214
+
215
+ **Audio Missing**
216
+ - FFmpeg must be installed and in PATH
217
+ - Check input video has audio track
218
+ - Try different output format
219
+
220
+ ### Performance Optimization
221
+
222
+ **For Large Videos**
223
+ - Use "fast" quality preset
224
+ - Increase `KEYFRAME_INTERVAL` to 8-10
225
+ - Process in shorter segments
226
+
227
+ **For High Resolution**
228
+ - Ensure sufficient VRAM (6GB+ recommended)
229
+ - Use two-stage mode for best quality
230
+ - Consider downscaling input video
231
+
232
+ ## Development
233
+
234
+ ### Project Structure
235
+ ```
236
+ β”œβ”€β”€ app.py # Main application entry point
237
+ β”œβ”€β”€ utilities.py # Core CV functions (segmentation, compositing)
238
+ β”œβ”€β”€ two_stage_processor.py # Green screen pipeline
239
+ β”œβ”€β”€ ui_components.py # Gradio interface
240
+ β”œβ”€β”€ requirements.txt # Python dependencies
241
+ β”œβ”€β”€ Dockerfile # Container configuration
242
+ └── README.md # This file
243
+ ```
244
+
245
+ ### Contributing
246
+
247
+ 1. Fork the repository
248
+ 2. Create feature branch (`git checkout -b feature/amazing-feature`)
249
+ 3. Commit changes (`git commit -m 'Add amazing feature'`)
250
+ 4. Push to branch (`git push origin feature/amazing-feature`)
251
+ 5. Open Pull Request
252
+
253
+ ### Testing
254
+
255
+ ```bash
256
+ # Run with test video
257
+ python app.py --test-mode
258
+
259
+ # Process sample
260
+ python -c "
261
+ from app import process_video_fixed, load_models_with_validation
262
+ load_models_with_validation()
263
+ result = process_video_fixed('test_video.mp4', 'office_modern', None)
264
+ print(f'Result: {result}')
265
+ "
266
+ ```
267
+
268
+ ## License
269
+
270
+ MIT License - see LICENSE file for details.
271
+
272
+ ## Acknowledgments
273
+
274
+ - **SAM2**: Meta's Segment Anything 2 for segmentation
275
+ - **MatAnyone**: High-quality image matting
276
+ - **Gradio**: Web interface framework
277
+ - **OpenCV**: Computer vision processing
278
+ - **FFmpeg**: Video encoding/decoding
279
+
280
+ ## Support
281
+
282
+ - **Issues**: Report bugs via GitHub Issues
283
+ - **Discussions**: Feature requests and questions
284
+ - **Documentation**: Check troubleshooting section first
285
+
286
+ ---
287
+
288
+ *For deployment on Hugging Face Spaces, see the space configuration in the app header.*