Dockerfile and compatible GPUs

[lucas-vitrus]

Hi,
first of all amazing model! (I've been tirelessly playing with it for hours, great results when combined with FLUX)!

Is it possible to get a Dockerfile to easily setup and replicate the environment for the model?

And which GPUs do you recommend? I saw that in the Huggingface demo you used a ZeroGPU, have you tested in other GPUs?
is 40Gb of vRAM the minimum?

ZeroGPU uses Nvidia A100 GPU devices under the hood (40GB of vRAM are available for each workloads)

Huggingface

[JeffreyXiang]

Hi! Thank you for your kind words and enthusiasm!

Good idea for adding a Dockerfile. I will try to include one in the repo. But I am not quite familier with it, so it may take sometime.

About the GPU, we tested with A100 and A6000, it takes about 16GB to run (See prerequisites). I also noticed successful deployment of TRELLIS on RTX3090 and RTX4090.

Hope this is helpful to you.

[realisticdreamer114514]

Is the 16GB requirement for the I2I model, the T2I model, or both? (Asking this to see if it is usable in Free Colab)
Any VRAM optimization in the works?

[Rothfeld]

only tested on linux with a 4090

note: diso & rembg need to be installed from within the running container

FROM nvidia/cuda:12.2.2-cudnn8-devel-ubuntu22.04

# 30xx  8.0 8.6 8.7 
# ENV TORCH_CUDA_ARCH_LIST="8.0;8.6;8.7"
# 40xx  8.9
# ENV TORCH_CUDA_ARCH_LIST="8.9"

ENV TORCH_CUDA_ARCH_LIST=8.0;8.6;8.7;8.9

RUN apt update \
 && apt-get install -yq --no-install-recommends \
    python3 \
    python3-dev \
    git \
    curl \
    ninja-build

RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py \
    && python3 get-pip.py \
    && rm get-pip.py


RUN pip install \
    torch==2.4.0 \
    torchvision==0.19

# BASIC
RUN pip install \
    pillow imageio imageio-ffmpeg tqdm easydict opencv-python-headless scipy ninja  onnxruntime trimesh xatlas pyvista pymeshfix igraph transformers
RUN pip install git+https://github.com/EasternJournalist/utils3d.git@9a4eb15e4021b67b12c460c7057d642626897ec8

# XFORMERS
RUN pip install xformers==0.0.27.post2 --index-url https://download.pytorch.org/whl/cu121
# FLASHATTN
RUN pip install flash-attn
# KAOLIN
RUN pip install kaolin -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.4.0_cu121.html 
# NVDIFFRAST 729261d
RUN pip install git+https://github.com/NVlabs/nvdiffrast.git

# DIFFOCTREERAST b09c20b
RUN git clone --recurse-submodules https://github.com/JeffreyXiang/diffoctreerast.git /tmp/extensions/diffoctreerast \
 && pip install /tmp/extensions/diffoctreerast

# MIPGAUSSIAN e3a68d0
RUN git clone https://github.com/autonomousvision/mip-splatting.git /tmp/extensions/mip-splatting \
 && pip install /tmp/extensions/mip-splatting/submodules/diff-gaussian-rasterization/

# VOX2SEQ
COPY extensions/vox2seq /tmp/extensions/vox2seq
RUN pip install /tmp/extensions/vox2seq

# SPCONV 2.3.6
RUN pip install spconv-cu120
# DEMO
RUN pip install gradio==4.44.1 gradio_litmodel3d==0.0.1

RUN useradd -ms /bin/bash user 
USER user
RUN mkdir /home/user/.cache

WORKDIR /workdir
EXPOSE 7860

ENV GRADIO_SERVER_NAME="0.0.0.0"
ENV GRADIO_SERVER_PORT="7860"

CMD ["bash"]

# docker build -t trellis .
# docker run --runtime=nvidia --rm -it -u $(id -u):$(id -g) -p 7860:7860 -v $(realpath ~)/.cache/huggingface:/home/user/.cache/huggingface -v $(pwd):/workdir trellis

# has to be run from within the container (needs the nvidia container runtime):
# pip install diso rembg

[AlexNolasco]

thank you, almost worked

podman run --rm -it --runtime=nvidia --gpus all -u $(id -u):$(id -g) -p 7860:7860 \
    -v $(realpath ~)/.cache/huggingface:/home/user/.cache/huggingface \
    -v $(pwd):/workdir trellis

did run pip install diso rembg inside the container as well as the setup, but..

[SPARSE][CONV] spconv algo: auto
[ATTENTION] Using backend: flash_attn
Traceback (most recent call last):
  File "/workdir/app.py", line 246, in <module>
    pipeline = TrellisImageTo3DPipeline.from_pretrained("JeffreyXiang/TRELLIS-image-large")
  File "/workdir/trellis/pipelines/trellis_image_to_3d.py", line 55, in from_pretrained
    pipeline = super(TrellisImageTo3DPipeline, TrellisImageTo3DPipeline).from_pretrained(path)
  File "/workdir/trellis/pipelines/base.py", line 39, in from_pretrained
    _models = {
  File "/workdir/trellis/pipelines/base.py", line 40, in <dictcomp>
    k: models.from_pretrained(f"{path}/{v}")
  File "/workdir/trellis/models/__init__.py", line 59, in from_pretrained
    model = __getattr__(config['name'])(**config['args'], **kwargs)
  File "/workdir/trellis/models/structured_latent_vae/decoder_mesh.py", line 105, in __init__
    self.mesh_extractor = SparseFeatures2Mesh(res=self.resolution*4, use_color=self.rep_config.get('use_color', False))
  File "/workdir/trellis/representations/mesh/cube2mesh.py", line 69, in __init__
    self.mesh_extractor = FlexiCubes(device=device)
NameError: name 'FlexiCubes' is not defined
user@7fd4b0aad2ff:/workdir$ 

[bitplane]

I've been fighting against nvidia and conda all weekend and finally managed to get it building. Opened a pull request #30 for review.

https://hub.docker.com/repository/docker/bitplane1/trellis/tags

There were a few things stopping the build:

• nvidia container tools actually mount the /bin directory when you give the container GPU access, which means you're running in a subtly different environment to when you're building.
• Docker builders (and buildx builders) don't have GPU capability, so I've added a post_build.sh script that installs the last couple of things and then runs example.py to cache the models in the image. (After this step I manually run an image through in the web UI and do a docker commit, so the above image on Docker Hub shouldn't download anything at all)
• JIT needs nvidia's compiler on there and g++. Conda seems to add them as system paths rather than user ones, so I had to stick a wrapper in for the compiler. Conda wanted to override CXX and I couldn't figure out how to make it stick, so I just use an export 🤷
• Gradio bound to loopback, so you can't expose the webui when its running.

[Rothfeld]

thank you, almost worked

podman run --rm -it --runtime=nvidia --gpus all -u $(id -u):$(id -g) -p 7860:7860 \
    -v $(realpath ~)/.cache/huggingface:/home/user/.cache/huggingface \
    -v $(pwd):/workdir trellis

did run pip install diso rembg inside the container as well as the setup, but..

[SPARSE][CONV] spconv algo: auto
[ATTENTION] Using backend: flash_attn
Traceback (most recent call last):
  File "/workdir/app.py", line 246, in <module>
    pipeline = TrellisImageTo3DPipeline.from_pretrained("JeffreyXiang/TRELLIS-image-large")
  File "/workdir/trellis/pipelines/trellis_image_to_3d.py", line 55, in from_pretrained
    pipeline = super(TrellisImageTo3DPipeline, TrellisImageTo3DPipeline).from_pretrained(path)
  File "/workdir/trellis/pipelines/base.py", line 39, in from_pretrained
    _models = {
  File "/workdir/trellis/pipelines/base.py", line 40, in <dictcomp>
    k: models.from_pretrained(f"{path}/{v}")
  File "/workdir/trellis/models/__init__.py", line 59, in from_pretrained
    model = __getattr__(config['name'])(**config['args'], **kwargs)
  File "/workdir/trellis/models/structured_latent_vae/decoder_mesh.py", line 105, in __init__
    self.mesh_extractor = SparseFeatures2Mesh(res=self.resolution*4, use_color=self.rep_config.get('use_color', False))
  File "/workdir/trellis/representations/mesh/cube2mesh.py", line 69, in __init__
    self.mesh_extractor = FlexiCubes(device=device)
NameError: name 'FlexiCubes' is not defined
user@7fd4b0aad2ff:/workdir$ 

Make sure you "clone --recursive" the repo rather than just "clone"

[AlexNolasco]

Make sure you "clone --recursive" the repo rather than just "clone"

Thank you, that did it, your Dockerfile created a 35.6GB image now running nicely on Ubuntu 24.04.1 LTS NVIDIA 3090.

docker run --rm -it --gpus all -p 8080:8080 trellis

[duylddev]

@Rothfeld why don't you run setup.sh inside docker?
is this shorter than manual install libs?

[Rothfeld]

@Rothfeld why don't you run setup.sh inside docker? is this shorter than manual install libs?

i couldn't get it to work so i gave up and made a simpler file that worked for my use case.

[MotazAM]

> only tested on linux with a 4090
> 
> note: diso & rembg need to be installed from within the running container
> 
> FROM nvidia/cuda:12.2.2-cudnn8-devel-ubuntu22.04
> 
> # 30xx  8.0 8.6 8.7 
> # ENV TORCH_CUDA_ARCH_LIST="8.0;8.6;8.7"
> # 40xx  8.9
> # ENV TORCH_CUDA_ARCH_LIST="8.9"
> 
> ENV TORCH_CUDA_ARCH_LIST=8.0;8.6;8.7;8.9
> 
> RUN apt update \
>  && apt-get install -yq --no-install-recommends \
>     python3 \
>     python3-dev \
>     git \
>     curl \
>     ninja-build
> 
> RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py \
>     && python3 get-pip.py \
>     && rm get-pip.py
> 
> 
> RUN pip install \
>     torch==2.4.0 \
>     torchvision==0.19
> 
> # BASIC
> RUN pip install \
>     pillow imageio imageio-ffmpeg tqdm easydict opencv-python-headless scipy ninja  onnxruntime trimesh xatlas pyvista pymeshfix igraph transformers
> RUN pip install git+https://github.com/EasternJournalist/utils3d.git@9a4eb15e4021b67b12c460c7057d642626897ec8
> 
> # XFORMERS
> RUN pip install xformers==0.0.27.post2 --index-url https://download.pytorch.org/whl/cu121
> # FLASHATTN
> RUN pip install flash-attn
> # KAOLIN
> RUN pip install kaolin -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.4.0_cu121.html 
> # NVDIFFRAST 729261d
> RUN pip install git+https://github.com/NVlabs/nvdiffrast.git
> 
> # DIFFOCTREERAST b09c20b
> RUN git clone --recurse-submodules https://github.com/JeffreyXiang/diffoctreerast.git /tmp/extensions/diffoctreerast \
>  && pip install /tmp/extensions/diffoctreerast
> 
> # MIPGAUSSIAN e3a68d0
> RUN git clone https://github.com/autonomousvision/mip-splatting.git /tmp/extensions/mip-splatting \
>  && pip install /tmp/extensions/mip-splatting/submodules/diff-gaussian-rasterization/
> 
> # VOX2SEQ
> COPY extensions/vox2seq /tmp/extensions/vox2seq
> RUN pip install /tmp/extensions/vox2seq
> 
> # SPCONV 2.3.6
> RUN pip install spconv-cu120
> # DEMO
> RUN pip install gradio==4.44.1 gradio_litmodel3d==0.0.1
> 
> RUN useradd -ms /bin/bash user 
> USER user
> RUN mkdir /home/user/.cache
> 
> WORKDIR /workdir
> EXPOSE 7860
> 
> ENV GRADIO_SERVER_NAME="0.0.0.0"
> ENV GRADIO_SERVER_PORT="7860"
> 
> CMD ["bash"]
> 
> # docker build -t trellis .
> # docker run --runtime=nvidia --rm -it -u $(id -u):$(id -g) -p 7860:7860 -v $(realpath ~)/.cache/huggingface:/home/user/.cache/huggingface -v $(pwd):/workdir trellis
> 
> # has to be run from within the container (needs the nvidia container runtime):
> # pip install diso rembg

I built the image and ran it, together with all listed commands in the shell.
When running app.py, I get the following error:

Traceback (most recent call last):
  File "/workdir/app.py", line 401, in <module>
    pipeline = TrellisImageTo3DPipeline.from_pretrained("JeffreyXiang/TRELLIS-image-large")
  File "/workdir/trellis/pipelines/trellis_image_to_3d.py", line 56, in from_pretrained
    pipeline = super(TrellisImageTo3DPipeline, TrellisImageTo3DPipeline).from_pretrained(path)
  File "/workdir/trellis/pipelines/base.py", line 34, in from_pretrained
    config_file = hf_hub_download(path, "pipeline.json")
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 860, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 977, in _hf_hub_download_to_cache_dir
    os.makedirs(os.path.dirname(blob_path), exist_ok=True)
  File "/usr/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/user/.cache/huggingface/hub'

the file/directory "/home/user/.cache/huggingface/hub" doesn't exist in my container

[Mutaz-Datatactics]

@Rothfeld I'm having the same issue as the comment above. Did I forget to download anything? where does "/home/user/.cache/huggingface/hub" come from?

[Rothfeld]

@Rothfeld I'm having the same issue as the comment above. Did I forget to download anything? where does "/home/user/.cache/huggingface/hub" come from?

if the volume mount it is giving you trouble you can run without it.
(it is just there to prevent redownloading models from hugginface that were downloaded before)

docker run --runtime=nvidia --rm -it -u $(id -u):$(id -g) -p 7860:7860 -v $(pwd):/workdir trellis

[MotazAM]

@Rothfeld I'm having the same issue as the comment above. Did I forget to download anything? where does "/home/user/.cache/huggingface/hub" come from?

if the volume mount it is giving you trouble you can run without it. (it is just there to prevent redownloading models from hugginface that were downloaded before)

docker run --runtime=nvidia --rm -it -u $(id -u):$(id -g) -p 7860:7860 -v $(pwd):/workdir trellis

That fixed it. Thanks!