Spaces:
				
			
			
	
			
			
		Running
		
			on 
			
			Zero
	
	
	
			
			
	
	
	
	
		
		
		Running
		
			on 
			
			Zero
	update
Browse files
    	
        README.md
    CHANGED
    
    | @@ -1,111 +1,13 @@ | |
| 1 | 
            -
             | 
| 2 | 
            -
             | 
| 3 | 
            -
             | 
| 4 | 
            -
             | 
| 5 | 
            -
             | 
| 6 | 
            -
             | 
| 7 | 
            -
              | 
| 8 | 
            -
              | 
| 9 | 
            -
             | 
| 10 | 
            -
             | 
| 11 | 
            -
             | 
| 12 | 
            -
             | 
| 13 | 
            -
             | 
| 14 | 
            -
            [Sijie Zhao<sup>1</sup>](https://scholar.google.com/citations?user=tZ3dS3MAAAAJ&hl=en), 
         | 
| 15 | 
            -
            [Xiaodong Cun<sup>1</sup>](https://vinthony.github.io/academic), <br>
         | 
| 16 | 
            -
            [Yong Zhang<sup>1</sup>](https://yzhang2016.github.io), 
         | 
| 17 | 
            -
            [Long Quan<sup>2</sup>](https://home.cse.ust.hk/~quan), 
         | 
| 18 | 
            -
            [Ying Shan<sup>3, 1</sup>](https://scholar.google.com/citations?user=4oXBp9UAAAAJ&hl=en)**_
         | 
| 19 | 
            -
            <br><br>
         | 
| 20 | 
            -
            <sup>1</sup>Tencent AI Lab
         | 
| 21 | 
            -
            <sup>2</sup>The Hong Kong University of Science and Technology
         | 
| 22 | 
            -
            <sup>3</sup>ARC Lab, Tencent PCG
         | 
| 23 | 
            -
             | 
| 24 | 
            -
            arXiv preprint, 2024
         | 
| 25 | 
            -
             | 
| 26 | 
            -
            </div>
         | 
| 27 | 
            -
             | 
| 28 | 
            -
            ## π Introduction
         | 
| 29 | 
            -
             | 
| 30 | 
            -
            - [24-9-19] Add scripts for preparing benchmark datasets. 
         | 
| 31 | 
            -
            - [24-9-18] Add point cloud sequence visualization.
         | 
| 32 | 
            -
            - [24-9-14] π₯π₯π₯ **DepthCrafter** is released now, have fun!
         | 
| 33 | 
            -
             | 
| 34 | 
            -
             | 
| 35 | 
            -
            π€ DepthCrafter can generate temporally consistent long depth sequences with fine-grained details for open-world videos, 
         | 
| 36 | 
            -
            without requiring additional information such as camera poses or optical flow.
         | 
| 37 | 
            -
             | 
| 38 | 
            -
            ## π₯ Visualization
         | 
| 39 | 
            -
            We provide some demos of unprojected point cloud sequences, with reference RGB and estimated depth videos. 
         | 
| 40 | 
            -
            Please refer to our [project page](https://depthcrafter.github.io) for more details.
         | 
| 41 | 
            -
             | 
| 42 | 
            -
             | 
| 43 | 
            -
            https://github.com/user-attachments/assets/62141cc8-04d0-458f-9558-fe50bc04cc21
         | 
| 44 | 
            -
             | 
| 45 | 
            -
             | 
| 46 | 
            -
             | 
| 47 | 
            -
             | 
| 48 | 
            -
            ## π Quick Start
         | 
| 49 | 
            -
             | 
| 50 | 
            -
            ### π οΈ Installation
         | 
| 51 | 
            -
            1. Clone this repo:
         | 
| 52 | 
            -
            ```bash
         | 
| 53 | 
            -
            git clone https://github.com/Tencent/DepthCrafter.git
         | 
| 54 | 
            -
            ```
         | 
| 55 | 
            -
            2. Install dependencies (please refer to [requirements.txt](requirements.txt)):
         | 
| 56 | 
            -
            ```bash
         | 
| 57 | 
            -
            pip install -r requirements.txt
         | 
| 58 | 
            -
            ```
         | 
| 59 | 
            -
             | 
| 60 | 
            -
            ## π€ Model Zoo
         | 
| 61 | 
            -
            [DepthCrafter](https://huggingface.co/tencent/DepthCrafter) is available in the Hugging Face Model Hub.
         | 
| 62 | 
            -
             | 
| 63 | 
            -
            ### πββοΈ Inference
         | 
| 64 | 
            -
            #### 1. High-resolution inference, requires a GPU with ~26GB memory for 1024x576 resolution:
         | 
| 65 | 
            -
            - Full inference (~0.6 fps on A100, recommended for high-quality results):
         | 
| 66 | 
            -
             | 
| 67 | 
            -
                ```bash
         | 
| 68 | 
            -
                python run.py  --video-path examples/example_01.mp4
         | 
| 69 | 
            -
                ```
         | 
| 70 | 
            -
             | 
| 71 | 
            -
             | 
| 72 | 
            -
            - Fast inference through 4-step denoising and without classifier-free guidance οΌ~2.3 fps on A100οΌ:
         | 
| 73 | 
            -
             | 
| 74 | 
            -
                ```bash
         | 
| 75 | 
            -
                python run.py  --video-path examples/example_01.mp4 --num-inference-steps 4 --guidance-scale 1.0
         | 
| 76 | 
            -
                ```
         | 
| 77 | 
            -
             | 
| 78 | 
            -
             | 
| 79 | 
            -
            #### 2. Low-resolution inference, requires a GPU with ~9GB memory for 512x256 resolution:
         | 
| 80 | 
            -
             | 
| 81 | 
            -
            - Full inference (~2.3 fps on A100):
         | 
| 82 | 
            -
             | 
| 83 | 
            -
                ```bash
         | 
| 84 | 
            -
                python run.py  --video-path examples/example_01.mp4 --max-res 512
         | 
| 85 | 
            -
                ```
         | 
| 86 | 
            -
             | 
| 87 | 
            -
            - Fast inference through 4-step denoising and without classifier-free guidance (~9.4 fps on A100):
         | 
| 88 | 
            -
                ```bash
         | 
| 89 | 
            -
                python run.py  --video-path examples/example_01.mp4  --max-res 512 --num-inference-steps 4 --guidance-scale 1.0
         | 
| 90 | 
            -
                ```
         | 
| 91 | 
            -
             | 
| 92 | 
            -
            ## π€ Gradio Demo
         | 
| 93 | 
            -
            We provide a local Gradio demo for DepthCrafter, which can be launched by running:
         | 
| 94 | 
            -
            ```bash
         | 
| 95 | 
            -
            gradio app.py
         | 
| 96 | 
            -
            ``` 
         | 
| 97 | 
            -
             | 
| 98 | 
            -
            ## π€ Contributing
         | 
| 99 | 
            -
            - Welcome to open issues and pull requests.
         | 
| 100 | 
            -
            - Welcome to optimize the inference speed and memory usage, e.g., through model quantization, distillation, or other acceleration techniques.
         | 
| 101 | 
            -
             | 
| 102 | 
            -
            ## π Citation
         | 
| 103 | 
            -
            If you find this work helpful, please consider citing:
         | 
| 104 | 
            -
            ```bibtex
         | 
| 105 | 
            -
            @article{hu2024-DepthCrafter,
         | 
| 106 | 
            -
                        author      = {Hu, Wenbo and Gao, Xiangjun and Li, Xiaoyu and Zhao, Sijie and Cun, Xiaodong and Zhang, Yong and Quan, Long and Shan, Ying},
         | 
| 107 | 
            -
                        title       = {DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos},
         | 
| 108 | 
            -
                        journal     = {arXiv preprint arXiv:2409.02095},
         | 
| 109 | 
            -
                        year        = {2024}
         | 
| 110 | 
            -
                }
         | 
| 111 | 
            -
            ```
         | 
|  | |
| 1 | 
            +
            ---
         | 
| 2 | 
            +
            title: DepthCrafter
         | 
| 3 | 
            +
            emoji: π¦
         | 
| 4 | 
            +
            colorFrom: purple
         | 
| 5 | 
            +
            colorTo: pink
         | 
| 6 | 
            +
            sdk: gradio
         | 
| 7 | 
            +
            sdk_version: 4.44.0
         | 
| 8 | 
            +
            app_file: app.py
         | 
| 9 | 
            +
            pinned: false
         | 
| 10 | 
            +
            license: other
         | 
| 11 | 
            +
            ---
         | 
| 12 | 
            +
             | 
| 13 | 
            +
            Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
         | 
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | 
    	
        app.py
    CHANGED
    
    | @@ -138,4 +138,5 @@ demo = construct_demo() | |
| 138 |  | 
| 139 | 
             
            if __name__ == "__main__":
         | 
| 140 | 
             
                demo.queue()
         | 
| 141 | 
            -
                demo.launch(server_name="0.0.0.0", server_port=80, debug=True)
         | 
|  | 
|  | |
| 138 |  | 
| 139 | 
             
            if __name__ == "__main__":
         | 
| 140 | 
             
                demo.queue()
         | 
| 141 | 
            +
                # demo.launch(server_name="0.0.0.0", server_port=80, debug=True)
         | 
| 142 | 
            +
                demo.launch(share=True)
         | 

