3D Gaussian Splatting

3D Gaussian Splatting is a revolutionary real-time 3D scene reconstruction technique with over 15,000 GitHub stars. Unlike NeRF-based methods, Gaussian Splatting represents scenes as millions of tiny 3D Gaussians that can be rendered at real-time frame rates (100+ FPS) while achieving photorealistic quality. Deploy it on Clore.ai's GPU cloud to reconstruct and explore 3D scenes from your own photos.

---

What is 3D Gaussian Splatting?

Traditional NeRF methods implicitly encode a scene in a neural network, requiring per-pixel ray marching at render time. Gaussian Splatting takes a fundamentally different approach:

1. Initialization: Start from a sparse point cloud (from COLMAP)

2. Representation: Expand each point into a 3D Gaussian with position, scale, rotation, opacity, and spherical harmonics color

3. Optimization: Differentiably render Gaussians and optimize against training images

4. Rendering: Project Gaussians onto the image plane via alpha-compositing (extremely fast)

Key advantages over NeRF:

• Real-time rendering (100+ FPS at 1080p)

• Better fine detail reconstruction

• Explicit 3D representation (editable, exportable)

• Faster training (30–60 min vs hours)

• Works on consumer GPUs

---

Prerequisites

Requirement	Minimum	Recommended
GPU VRAM	12 GB	24 GB
GPU	RTX 3080 12GB	RTX 4090 / A100
RAM	16 GB	32 GB
Storage	30 GB	60 GB
CUDA	11.7+	12.1+

Gaussian Splatting has strict CUDA requirements. The CUDA version must match the diff-gaussian-rasterization compiled extension. Using the provided Dockerfile eliminates compatibility issues.

---

Step 1 — Rent a GPU on Clore.ai

1. Log in to clore.ai.

2. Click Marketplace and filter by VRAM ≥ 16 GB.

3. Select a server — RTX 4090 offers the best price/performance.

4. Set Docker image to your custom image (see Step 2).

5. Set open ports: 22 (SSH) and 8080 (web viewer).

6. Click Rent.

---

Step 2 — Dockerfile

Build a custom Docker image with all dependencies:

FROM pytorch/pytorch:2.1.2-cuda12.1-cudnn8-devel

ENV DEBIAN_FRONTEND=noninteractive
ENV TORCH_CUDA_ARCH_LIST="6.0;6.1;7.0;7.5;8.0;8.6;8.9;9.0+PTX"

RUN apt-get update && apt-get install -y \
    git wget curl cmake build-essential \
    libboost-program-options-dev libboost-filesystem-dev \
    libboost-graph-dev libboost-system-dev libboost-test-dev \
    libeigen3-dev libflann-dev libfreeimage-dev \
    libmetis-dev libgoogle-glog-dev libgflags-dev \
    libsqlite3-dev libglew-dev qtbase5-dev libqt5opengl5-dev \
    libcgal-dev libceres-dev \
    ffmpeg libgl1 libglib2.0-0 \
    openssh-server \
    python3-pip python3-dev \
    && rm -rf /var/lib/apt/lists/*

# Install COLMAP
RUN apt-get update && apt-get install -y colmap && rm -rf /var/lib/apt/lists/*

# Configure SSH
RUN mkdir /var/run/sshd && \
    echo 'root:clore123' | chpasswd && \
    sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

WORKDIR /workspace

# Clone original 3DGS repo
RUN git clone https://github.com/graphdeco-inria/gaussian-splatting /workspace/gaussian-splatting \
    --recursive

# Install Python dependencies
RUN cd /workspace/gaussian-splatting && \
    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 && \
    pip install -r requirements.txt

# Build CUDA extensions
RUN cd /workspace/gaussian-splatting && \
    pip install submodules/diff-gaussian-rasterization && \
    pip install submodules/simple-knn

# Install web viewer dependencies
RUN pip install viser==0.1.29 nerfview==0.0.4 trimesh

EXPOSE 22 8080

CMD service ssh start && tail -f /dev/null

Build and Push

Build the image and push it to your own Docker Hub account (replace YOUR_DOCKERHUB_USERNAME with your actual username):

docker build -t YOUR_DOCKERHUB_USERNAME/gaussian-splatting:latest .
docker push YOUR_DOCKERHUB_USERNAME/gaussian-splatting:latest

There is no official pre-built Docker image for 3D Gaussian Splatting on Docker Hub. The official repository at graphdeco-inria/gaussian-splatting does not provide one — build from the Dockerfile above. The image must be built with the correct CUDA architecture flags matching your target GPU.

Use YOUR_DOCKERHUB_USERNAME/gaussian-splatting:latest in your Clore.ai configuration.

---

Step 3 — Connect via SSH

ssh root@<clore-host> -p <assigned-ssh-port>

Verify the build:

cd /workspace/gaussian-splatting
python -c "from diff_gaussian_rasterization import GaussianRasterizationSettings; print('CUDA extension OK')"

---

Step 4 — Prepare Your Dataset

Option A: Use Tandt (Tanks and Temples) Dataset

Classic benchmark dataset for quick testing:

mkdir -p /workspace/data && cd /workspace/data

# Download small test scene
wget https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/input/tandt.zip
unzip tandt.zip

Option B: Process Your Own Photos

# Upload photos
scp -P <port> -r ./my_photos/ root@<clore-host>:/workspace/data/

# Run COLMAP processing script (provided with 3DGS)
cd /workspace/gaussian-splatting

python convert.py \
    -s /workspace/data/my_photos \
    --no_gpu   # optional: if COLMAP GPU solver conflicts

The convert.py script runs the full COLMAP pipeline: feature extraction, matching, sparse reconstruction, and undistortion. This takes 5–30 minutes depending on image count.

Option C: Process from Video

# Extract frames from video at 2fps
ffmpeg -i /workspace/data/my_video.mp4 \
    -vf fps=2 \
    /workspace/data/frames/frame_%04d.jpg

# Then run COLMAP processing on the frames
python convert.py -s /workspace/data/frames

---

Step 5 — Train a Gaussian Splat

Standard Training

cd /workspace/gaussian-splatting

python train.py \
    -s /workspace/data/my_photos \
    -m /workspace/output/my_scene \
    --iterations 30000 \
    --eval

Training on Tandt Dataset

python train.py \
    -s /workspace/data/tandt/truck \
    -m /workspace/output/truck \
    --iterations 30000 \
    --eval

Fast Training (Quick Preview)

python train.py \
    -s /workspace/data/my_photos \
    -m /workspace/output/my_scene_fast \
    --iterations 7000

Training to 7,000 iterations takes ~10 minutes on an RTX 4090 and gives a good quality preview. Full 30,000 iterations takes ~30–40 minutes and produces final quality.

Training Progress

Monitor training output — you'll see metrics like:

[ITER 1000] Evaluating train: L1 0.04, PSNR 26.12 dB
[ITER 7000] Evaluating train: L1 0.02, PSNR 29.45 dB
[ITER 30000] Evaluating train: L1 0.01, PSNR 32.80 dB

PSNR above 30 dB indicates high-quality reconstruction.

---

Step 6 — Render and Visualize

Render from Trained Model

python render.py \
    -m /workspace/output/my_scene \
    --skip_train

Renders are saved to /workspace/output/my_scene/test/ours_30000/renders/.

Create a Flythrough Video

# Convert rendered frames to video
ffmpeg -framerate 24 \
    -pattern_type glob \
    -i '/workspace/output/my_scene/test/ours_30000/renders/*.png' \
    -c:v libx264 \
    -pix_fmt yuv420p \
    /workspace/output/flythrough.mp4

Evaluate Metrics

python metrics.py -m /workspace/output/my_scene

Expected output:

SSIM : 0.8324
PSNR : 32.81
LPIPS: 0.1893

---

Step 7 — Interactive Web Viewer

To explore the trained scene interactively:

Using nerfview/viser

# /workspace/view_splat.py
import viser
import numpy as np
from plyfile import PlyData
import torch

server = viser.ViserServer(host="0.0.0.0", port=8080)
print("Viewer running at http://0.0.0.0:8080")

# Load PLY file
ply_path = "/workspace/output/my_scene/point_cloud/iteration_30000/point_cloud.ply"
plydata = PlyData.read(ply_path)

xyz = np.stack([
    plydata['vertex']['x'],
    plydata['vertex']['y'],
    plydata['vertex']['z'],
], axis=-1)

# Add point cloud to viewer
server.add_point_cloud(
    name="/splat",
    points=xyz,
    colors=np.ones((len(xyz), 3)) * 0.7,
    point_size=0.003,
)

import time
while True:
    time.sleep(0.01)

python /workspace/view_splat.py &

Then open: http://<clore-host>:<public-port-8080>

Alternative: Use SuperSplat (Browser-Based Viewer)

Download the .ply file and open it in SuperSplat:

# Download from your local machine
scp -P <port> root@<clore-host>:/workspace/output/my_scene/point_cloud/iteration_30000/point_cloud.ply ./

Then drag-and-drop the .ply into the SuperSplat browser at: https://playcanvas.com/super-splat

---

Advanced Options

Control Number of Gaussians

# Higher densification for more detailed scenes
python train.py \
    -s /workspace/data/my_photos \
    -m /workspace/output/my_scene \
    --densify_until_iter 15000 \
    --densify_grad_threshold 0.0002

White Background (for Objects)

python train.py \
    -s /workspace/data/my_object \
    -m /workspace/output/my_object \
    --white_background

Large Scale Scenes

# Increase opacity reset interval for outdoor scenes
python train.py \
    -s /workspace/data/outdoor \
    -m /workspace/output/outdoor \
    --opacity_reset_interval 5000 \
    --iterations 50000

---

Alternative: Gaussian Splatting with gsplat

gsplat is a faster, memory-efficient implementation:

pip install gsplat

# Training with gsplat
python examples/simple_trainer.py \
    --data_dir /workspace/data/my_photos \
    --result_dir /workspace/gsplat_output

---

Troubleshooting

CUDA Extension Build Fails

error: no kernel image is available for execution on the device

Solution: Rebuild for your specific GPU architecture:

export TORCH_CUDA_ARCH_LIST="8.6"  # For RTX 3090/4090
cd /workspace/gaussian-splatting
pip install submodules/diff-gaussian-rasterization --force-reinstall

COLMAP Fails to Reconstruct

Solutions:

• Ensure ≥ 50% image overlap

• Use more photos (100+ recommended)

• Try sequential matching for video frames: add --match sequential to convert.py

Out of Memory During Training

# Reduce max number of Gaussians
python train.py \
    -s /workspace/data/my_photos \
    -m /workspace/output/my_scene \
    --max_num_splats 2000000  # default is ~6M

Floaters in the Scene

Floating artifacts from Gaussian initialization:

• Increase --densify_grad_threshold to be more selective

• Use --prune_opacity_threshold 0.005 to remove low-opacity Gaussians earlier

---

Clore.ai GPU Recommendations

Gaussian Splatting training is GPU-compute intensive with frequent CUDA kernel calls. VRAM determines max scene complexity (number of Gaussians); compute determines training speed.

GPU	VRAM	Clore.ai Price	30K iter training	Max Gaussians
RTX 3090	24 GB	~$0.12/hr	~45–55 min	~6M
RTX 4090	24 GB	~$0.70/hr	~30–35 min	~6M
A100 40GB	40 GB	~$1.20/hr	~12–18 min	~10M+
RTX 3080 12GB	12 GB	~$0.08/hr	~70 min	~3M (limited)

RTX 3090 at ~$0.12/hr is the best choice for Gaussian Splatting. A full 30K iteration training run costs ~$0.09–0.11 in GPU time. For multiple scenes in a session, the cost is negligible.

For quick experiments: train to 7,000 iterations first (~15 min on RTX 3090, ~$0.03). Check quality in the web viewer. Only run full 30K iterations for final output.

COLMAP preprocessing note: COLMAP (Structure from Motion) runs on CPU/GPU but the heavy compute is on CPU. Most Clore.ai servers have adequate CPUs for scenes under 200 images. For 500+ image datasets, look for servers with 16+ CPU cores.

---

Useful Resources

• 3D Gaussian Splatting GitHub

• Original Paper (SIGGRAPH 2023)

• gsplat — Fast Implementation

• SuperSplat — Browser Viewer

• Gaussian Splatting Community (Reddit)

• Awesome Gaussian Splatting