MogensR commited on
Commit
e35e5ca
·
1 Parent(s): e74e423

Create scripts/README.md

Browse files
Files changed (1) hide show
  1. scripts/README.md +340 -0
scripts/README.md ADDED
@@ -0,0 +1,340 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # BackgroundFX Pro Scripts
2
+
3
+ Utility scripts for setup, deployment, maintenance, and monitoring of BackgroundFX Pro.
4
+
5
+ ## Available Scripts
6
+
7
+ ### setup.sh
8
+ Complete environment setup script that handles:
9
+ - System requirements checking
10
+ - Virtual environment creation
11
+ - Dependency installation
12
+ - Model downloading
13
+ - Configuration creation
14
+ - CLI installation
15
+
16
+ **Usage:**
17
+ ```bash
18
+ chmod +x scripts/setup.sh
19
+ ./scripts/setup.sh
20
+ ```
21
+
22
+ ### deploy.sh
23
+ Deployment automation script supporting multiple environments and methods:
24
+ - Docker deployment
25
+ - Kubernetes deployment
26
+ - Server deployment
27
+ - Health checks and rollback
28
+
29
+ **Usage:**
30
+ ```bash
31
+ # Deploy to production with Docker
32
+ ./scripts/deploy.sh --env production --method docker
33
+
34
+ # Deploy to staging with specific version
35
+ ./scripts/deploy.sh --env staging --version 1.2.0 --registry myregistry.com
36
+
37
+ # Deploy to Kubernetes
38
+ ./scripts/deploy.sh --env production --method kubernetes
39
+
40
+ # Deploy to server (requires DEPLOY_HOST and DEPLOY_USER env vars)
41
+ export DEPLOY_HOST=server.example.com
42
+ export DEPLOY_USER=deploy
43
+ ./scripts/deploy.sh --method server
44
+ ```
45
+
46
+ ### benchmark.py
47
+ Performance benchmarking script for testing various aspects:
48
+ - Image processing speed
49
+ - Model loading times
50
+ - Video processing performance
51
+ - Batch processing throughput
52
+
53
+ **Usage:**
54
+ ```bash
55
+ # Run all benchmarks
56
+ python scripts/benchmark.py
57
+
58
+ # Run specific benchmarks
59
+ python scripts/benchmark.py --tests image model
60
+
61
+ # Save results to specific file
62
+ python scripts/benchmark.py --output results.json --iterations 10
63
+ ```
64
+
65
+ ### monitor.py
66
+ System monitoring script for production environments:
67
+ - Resource usage tracking
68
+ - Processing statistics
69
+ - Error monitoring
70
+ - Performance metrics
71
+
72
+ **Usage:**
73
+ ```bash
74
+ # Start monitoring
75
+ python scripts/monitor.py
76
+
77
+ # Monitor with custom interval
78
+ python scripts/monitor.py --interval 30
79
+
80
+ # Export metrics to Prometheus
81
+ python scripts/monitor.py --export prometheus
82
+ ```
83
+
84
+ ### backup.sh
85
+ Backup and restore script for data and configurations:
86
+ - Model backups
87
+ - Configuration backups
88
+ - Database backups
89
+ - Volume backups (Docker)
90
+
91
+ **Usage:**
92
+ ```bash
93
+ # Create backup
94
+ ./scripts/backup.sh create
95
+
96
+ # Restore from backup
97
+ ./scripts/backup.sh restore --file backup_20240101_120000.tar.gz
98
+
99
+ # Schedule automatic backups
100
+ ./scripts/backup.sh schedule --cron "0 2 * * *"
101
+ ```
102
+
103
+ ### update.sh
104
+ Update script for upgrading BackgroundFX Pro:
105
+ - Git pull latest changes
106
+ - Update dependencies
107
+ - Download new models
108
+ - Database migrations
109
+ - Rolling restart
110
+
111
+ **Usage:**
112
+ ```bash
113
+ # Update to latest version
114
+ ./scripts/update.sh
115
+
116
+ # Update to specific version
117
+ ./scripts/update.sh --version 2.0.0
118
+
119
+ # Update without restart
120
+ ./scripts/update.sh --no-restart
121
+ ```
122
+
123
+ ### clean.sh
124
+ Cleanup script for maintenance:
125
+ - Remove temporary files
126
+ - Clear cache
127
+ - Clean old logs
128
+ - Remove unused Docker images
129
+ - Free disk space
130
+
131
+ **Usage:**
132
+ ```bash
133
+ # Basic cleanup
134
+ ./scripts/clean.sh
135
+
136
+ # Aggressive cleanup
137
+ ./scripts/clean.sh --aggressive
138
+
139
+ # Dry run (show what would be deleted)
140
+ ./scripts/clean.sh --dry-run
141
+ ```
142
+
143
+ ## Environment Variables
144
+
145
+ Many scripts use environment variables for configuration:
146
+
147
+ ```bash
148
+ # Deployment
149
+ export DEPLOY_HOST=server.example.com
150
+ export DEPLOY_USER=deploy
151
+ export DEPLOY_KEY=/path/to/key.pem
152
+
153
+ # Docker Registry
154
+ export REGISTRY=myregistry.com
155
+ export REGISTRY_USER=username
156
+ export REGISTRY_PASS=password
157
+
158
+ # Backup
159
+ export BACKUP_DIR=/backups
160
+ export S3_BUCKET=my-backups
161
+ export AWS_ACCESS_KEY_ID=xxx
162
+ export AWS_SECRET_ACCESS_KEY=xxx
163
+
164
+ # Monitoring
165
+ export PROMETHEUS_PUSHGATEWAY=http://prometheus:9091
166
+ export SLACK_WEBHOOK=https://hooks.slack.com/xxx
167
+ ```
168
+
169
+ ## Cron Jobs
170
+
171
+ Example crontab entries for automated tasks:
172
+
173
+ ```cron
174
+ # Daily backup at 2 AM
175
+ 0 2 * * * /opt/backgroundfx/scripts/backup.sh create
176
+
177
+ # Hourly monitoring export
178
+ 0 * * * * /opt/backgroundfx/scripts/monitor.py --export prometheus
179
+
180
+ # Weekly cleanup on Sunday at 3 AM
181
+ 0 3 * * 0 /opt/backgroundfx/scripts/clean.sh
182
+
183
+ # Daily model updates at 4 AM
184
+ 0 4 * * * /opt/backgroundfx/scripts/update_models.py
185
+ ```
186
+
187
+ ## Docker Integration
188
+
189
+ Running scripts in Docker containers:
190
+
191
+ ```bash
192
+ # Run setup in container
193
+ docker run --rm -v $(pwd):/app backgroundfx-pro:latest ./scripts/setup.sh
194
+
195
+ # Run benchmark
196
+ docker run --rm --gpus all backgroundfx-pro:latest python scripts/benchmark.py
197
+
198
+ # Backup Docker volumes
199
+ docker run --rm \
200
+ -v backgroundfx_data:/data \
201
+ -v $(pwd)/backups:/backups \
202
+ backgroundfx-pro:latest ./scripts/backup.sh create
203
+ ```
204
+
205
+ ## CI/CD Integration
206
+
207
+ ### GitHub Actions
208
+
209
+ ```yaml
210
+ name: Deploy
211
+ on:
212
+ push:
213
+ branches: [main]
214
+
215
+ jobs:
216
+ deploy:
217
+ runs-on: ubuntu-latest
218
+ steps:
219
+ - uses: actions/checkout@v2
220
+ - name: Deploy to production
221
+ env:
222
+ DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
223
+ DEPLOY_USER: ${{ secrets.DEPLOY_USER }}
224
+ run: |
225
+ ./scripts/deploy.sh --env production --method server
226
+ ```
227
+
228
+ ### GitLab CI
229
+
230
+ ```yaml
231
+ deploy:
232
+ stage: deploy
233
+ script:
234
+ - ./scripts/deploy.sh --env production --method kubernetes
235
+ only:
236
+ - main
237
+ ```
238
+
239
+ ## Troubleshooting
240
+
241
+ ### Permission Issues
242
+ ```bash
243
+ # Make scripts executable
244
+ chmod +x scripts/*.sh
245
+
246
+ # Fix ownership
247
+ sudo chown -R $USER:$USER scripts/
248
+ ```
249
+
250
+ ### Path Issues
251
+ ```bash
252
+ # Add scripts to PATH
253
+ export PATH=$PATH:/opt/backgroundfx/scripts
254
+
255
+ # Or create symlinks
256
+ sudo ln -s /opt/backgroundfx/scripts/bgfx-deploy /usr/local/bin/
257
+ ```
258
+
259
+ ### Python Scripts Not Found
260
+ ```bash
261
+ # Ensure virtual environment is activated
262
+ source venv/bin/activate
263
+
264
+ # Or use full Python path
265
+ /opt/backgroundfx/venv/bin/python scripts/benchmark.py
266
+ ```
267
+
268
+ ## Development
269
+
270
+ ### Adding New Scripts
271
+
272
+ 1. Create script in `scripts/` directory
273
+ 2. Add shebang line (`#!/bin/bash` or `#!/usr/bin/env python3`)
274
+ 3. Make executable: `chmod +x scripts/your_script.sh`
275
+ 4. Add documentation to this README
276
+ 5. Test in Docker environment
277
+
278
+ ### Script Template
279
+
280
+ ```bash
281
+ #!/bin/bash
282
+ # Script description
283
+ # Usage: script.sh [options]
284
+
285
+ set -e # Exit on error
286
+
287
+ # Configuration
288
+ SCRIPT_DIR=$(dirname $(realpath $0))
289
+ PROJECT_ROOT=$(dirname $SCRIPT_DIR)
290
+
291
+ # Colors
292
+ RED='\033[0;31m'
293
+ GREEN='\033[0;32m'
294
+ NC='\033[0m'
295
+
296
+ # Functions
297
+ show_help() {
298
+ cat << EOF
299
+ Usage: $0 [OPTIONS]
300
+ Description of what the script does
301
+
302
+ Options:
303
+ --help Show this help message
304
+ EOF
305
+ }
306
+
307
+ # Main logic
308
+ main() {
309
+ echo "Running script..."
310
+ # Your code here
311
+ }
312
+
313
+ # Parse arguments
314
+ while [[ $# -gt 0 ]]; do
315
+ case $1 in
316
+ --help)
317
+ show_help
318
+ exit 0
319
+ ;;
320
+ *)
321
+ echo "Unknown option: $1"
322
+ show_help
323
+ exit 1
324
+ ;;
325
+ esac
326
+ done
327
+
328
+ # Run main function
329
+ main
330
+ ```
331
+
332
+ ## Security Considerations
333
+
334
+ - Never commit sensitive data in scripts
335
+ - Use environment variables for secrets
336
+ - Validate all user inputs
337
+ - Use `set -e` to exit on errors
338
+ - Implement proper error handling
339
+ - Log actions for audit trail
340
+ - Restrict script permissions appropriately