# Nerfstudio

**Nerfstudio** is a modular, researcher-friendly framework for training and rendering **Neural Radiance Fields (NeRF)** — a technique that reconstructs photorealistic 3D scenes from 2D images. With over 10,000 GitHub stars, it is the de facto standard for NeRF research and production applications. Run it on Clore.ai's GPU cloud to reconstruct 3D scenes from your own photos or videos.

***

## What is Nerfstudio?

NeRF (Neural Radiance Field) represents a 3D scene as a neural network that, given a camera position and direction, outputs the color and density at that point. By training on dozens of photographs taken from different angles, NeRF learns a complete 3D representation that can be rendered from any viewpoint.

**Nerfstudio provides:**

* Multiple NeRF methods: Nerfacto, Instant-NGP, Splatfacto, TensoRF, and more
* CLI and Python API
* Interactive web viewer (Viser) at port 7007
* Export to point clouds, meshes, and video flythroughs
* Support for custom datasets via COLMAP integration

**Use cases:**

* 3D scene reconstruction from drone footage
* Product visualization from photos
* Virtual tours from smartphone captures
* Research into novel view synthesis

***

## Prerequisites

| Requirement | Minimum  | Recommended     |
| ----------- | -------- | --------------- |
| GPU VRAM    | 8 GB     | 16–24 GB        |
| GPU         | RTX 3080 | RTX 4090 / A100 |
| RAM         | 16 GB    | 32 GB           |
| Storage     | 20 GB    | 50+ GB          |
| CUDA        | 11.8+    | 12.1+           |

{% hint style="info" %}
Training time scales with scene complexity. A typical outdoor scene from 100 photos trains in 10–30 minutes on an RTX 4090. The interactive viewer updates in real-time during training.
{% endhint %}

***

## Step 1 — Rent a GPU on Clore.ai

1. Log in to [clore.ai](https://clore.ai).
2. Click **Marketplace** and filter by VRAM ≥ 16 GB.
3. Select a server — RTX 4090 is ideal for Nerfstudio.
4. Set Docker image: **`dromni/nerfstudio:latest`**
5. Set open ports: `22` (SSH) and `7007` (Viser web viewer).
6. Click **Rent** and wait for the instance to initialize.

{% hint style="info" %}
The `dromni/nerfstudio` image is the community-maintained official image and includes all dependencies pre-installed (CUDA, tiny-cuda-nn, colmap, ffmpeg).
{% endhint %}

***

## Step 2 — Connect via SSH

```bash
ssh user@<clore-host> -p <assigned-ssh-port>
```

{% hint style="info" %}
The `dromni/nerfstudio` image uses `user` (not `root`) by default. Use `sudo` for administrative tasks.
{% endhint %}

Verify the installation:

```bash
ns-train --help
ns-render --help
ns-viewer --help
```

***

## Step 3 — Prepare Your Dataset

### Option A: Use the Provided Example Dataset

Nerfstudio includes built-in datasets to test immediately:

```bash
# Download the poster dataset (small, great for testing)
ns-download-data nerfstudio --capture-name=poster

# Other available datasets:
# ns-download-data nerfstudio --capture-name=egypt
# ns-download-data nerfstudio --capture-name=floating-tree
# ns-download-data nerfstudio --capture-name=stump
```

### Option B: Process Your Own Images

If you have photos or video of your scene:

#### From images (COLMAP pipeline):

```bash
# Upload your images via SCP
scp -P <port> -r ./my_photos/ user@<clore-host>:/home/user/data/

# Process with COLMAP to get camera poses
ns-process-data images \
    --data /home/user/data/my_photos \
    --output-dir /home/user/data/my_scene
```

#### From video:

```bash
# Upload video
scp -P <port> my_video.mp4 user@<clore-host>:/home/user/data/

# Extract frames and process
ns-process-data video \
    --data /home/user/data/my_video.mp4 \
    --output-dir /home/user/data/my_scene \
    --num-frames-target 200
```

{% hint style="info" %}
For best results, use **100–300 photos** with significant overlap (>60% between adjacent frames). Walk around the object/scene in a systematic pattern — circles, grids, or figure-eights work well.
{% endhint %}

***

## Step 4 — Train a NeRF

### Basic Training with Nerfacto (Recommended)

Nerfacto is Nerfstudio's flagship method, balancing quality and speed:

```bash
ns-train nerfacto \
    --data /home/user/data/my_scene \
    --viewer.websocket-port 7007 \
    --output-dir /home/user/outputs
```

### Training with Instant-NGP (Fastest)

```bash
ns-train instant-ngp \
    --data /home/user/data/my_scene \
    --viewer.websocket-port 7007 \
    --output-dir /home/user/outputs
```

### Training with the Provided Poster Dataset

```bash
ns-train nerfacto \
    --data /home/user/data/nerfstudio/poster \
    --viewer.websocket-port 7007
```

***

## Step 5 — Access the Interactive Viewer

Open your browser and navigate to:

```
http://<clore-host>:<public-port-7007>
```

You will see a 3D viewer powered by **Viser** that shows:

* Live training progress
* Current NeRF rendering quality
* Interactive camera controls
* Training loss curves

{% hint style="info" %}
The viewer updates every few seconds during training. You can rotate, pan, and zoom to inspect the scene quality as training progresses.
{% endhint %}

***

## Available Training Methods

| Method         | Speed     | Quality   | VRAM  | Notes              |
| -------------- | --------- | --------- | ----- | ------------------ |
| `nerfacto`     | Medium    | High      | 8 GB  | Best all-around    |
| `instant-ngp`  | Fast      | Medium    | 6 GB  | Fastest training   |
| `splatfacto`   | Fast      | High      | 8 GB  | Gaussian splatting |
| `tensorf`      | Medium    | High      | 12 GB | Good for objects   |
| `mipnerf360`   | Slow      | Very High | 24 GB | Best quality       |
| `vanilla-nerf` | Very Slow | High      | 16 GB | Research baseline  |

### Training with Splatfacto (Gaussian Splatting)

```bash
ns-train splatfacto \
    --data /home/user/data/my_scene \
    --viewer.websocket-port 7007 \
    --pipeline.model.num-random 50000
```

***

## Step 6 — Evaluate and Render

### Check Training Metrics

```bash
# View the training summary
ls /home/user/outputs/my_scene/nerfacto/

# Latest checkpoint
ls /home/user/outputs/my_scene/nerfacto/*/nerfstudio_models/
```

### Render a Video Flythrough

```bash
# Interactive: set camera path in the viewer, then export
# Command-line rendering:

ns-render camera-path \
    --load-config /home/user/outputs/my_scene/nerfacto/<timestamp>/config.yml \
    --camera-path-filename /home/user/data/my_scene/camera_paths/my_path.json \
    --output-path /home/user/renders/output.mp4
```

### Render Interpolated Spiral

```bash
ns-render interpolate \
    --load-config /home/user/outputs/my_scene/nerfacto/<timestamp>/config.yml \
    --output-path /home/user/renders/spiral.mp4 \
    --render-nearest-camera True \
    --order-poses True
```

***

## Step 7 — Export 3D Geometry

### Export Point Cloud

```bash
ns-export pointcloud \
    --load-config /home/user/outputs/my_scene/nerfacto/<timestamp>/config.yml \
    --output-dir /home/user/exports/ \
    --num-points 1000000
```

### Export Mesh

```bash
ns-export marching-cubes \
    --load-config /home/user/outputs/my_scene/nerfacto/<timestamp>/config.yml \
    --output-dir /home/user/exports/ \
    --resolution 1024
```

### Export Gaussian Splats (PLY)

```bash
ns-export gaussian-splat \
    --load-config /home/user/outputs/my_scene/splatfacto/<timestamp>/config.yml \
    --output-dir /home/user/exports/
```

***

## Python API

For programmatic training and evaluation:

```python
from nerfstudio.scripts.train import main
from nerfstudio.configs.base_config import ViewerConfig
from nerfstudio.engine.trainer import TrainerConfig

# Python API training example
from nerfstudio.configs.method_configs import method_configs
from pathlib import Path

config = method_configs["nerfacto"]
config.pipeline.datamanager.data = Path("/home/user/data/my_scene")
config.viewer = ViewerConfig(websocket_port=7007, num_rays_per_chunk=1<<15)
config.max_num_iterations = 30000

trainer = config.setup()
trainer.train()
```

***

## Custom Dataset Tips

### Camera Capture Best Practices

| Setting  | Recommendation                       |
| -------- | ------------------------------------ |
| Overlap  | ≥ 60% between frames                 |
| Images   | 100–300 (outdoors), 50–150 (objects) |
| Motion   | Slow, steady movement                |
| Lighting | Consistent, avoid harsh shadows      |
| Focus    | Sharp throughout                     |

### Improving COLMAP Results

```bash
# For difficult scenes, use exhaustive matching (slower but more accurate)
ns-process-data images \
    --data /home/user/data/my_photos \
    --output-dir /home/user/data/my_scene \
    --sfm-tool colmap \
    --matching-method exhaustive
```

***

## Troubleshooting

### COLMAP Fails to Find Camera Poses

**Solutions:**

* Ensure images have sufficient overlap
* Verify images are sharp (no motion blur)
* Try exhaustive matching: `--matching-method exhaustive`
* Reduce `--num-frames-target` for video to select better frames

### Viewer Not Accessible

**Solution:** Ensure port 7007 is forwarded in Clore.ai. Test connectivity:

```bash
ss -tlnp | grep 7007
```

### Training Loss Not Decreasing

**Solutions:**

* Check COLMAP succeeded (look for `transforms.json` in output dir)
* Reduce learning rate: `--pipeline.model.field-implementation hash`
* Check for dominant sky (use `--pipeline.model.background-color white`)

### Out of Memory

```bash
# Reduce batch size
ns-train nerfacto \
    --data /home/user/data/my_scene \
    --pipeline.datamanager.train-num-rays-per-batch 2048 \
    --pipeline.datamanager.eval-num-rays-per-batch 1024
```

***

## Download Outputs

After training, download your renders and exports:

```bash
# From your local machine
scp -P <port> -r user@<clore-host>:/home/user/renders/ ./local-renders/
scp -P <port> -r user@<clore-host>:/home/user/exports/ ./local-exports/
```

***

## Cost Estimation

| GPU       | VRAM  | Est. Price | 100-image scene |
| --------- | ----- | ---------- | --------------- |
| RTX 3080  | 10 GB | \~$0.10/hr | \~30–45 min     |
| RTX 4090  | 24 GB | \~$0.35/hr | \~10–15 min     |
| A100 40GB | 40 GB | \~$0.80/hr | \~5–8 min       |

{% hint style="info" %}
Start with Instant-NGP for fast previews, then switch to Nerfacto or MipNeRF360 for final quality. This workflow saves significant compute cost.
{% endhint %}

***

## Useful Resources

* [Nerfstudio GitHub](https://github.com/nerfstudio-project/nerfstudio)
* [Nerfstudio Documentation](https://docs.nerf.studio)
* [dromni/nerfstudio Docker Hub](https://hub.docker.com/r/dromni/nerfstudio)
* [Nerfstudio Method Comparison](https://docs.nerf.studio/nerfology/methods/index.html)
* [COLMAP Documentation](https://colmap.github.io/)

***

## Clore.ai GPU Recommendations

| Use Case                      | Recommended GPU | Est. Cost on Clore.ai |
| ----------------------------- | --------------- | --------------------- |
| Development/Testing           | RTX 3090 (24GB) | \~$0.12/gpu/hr        |
| Production                    | RTX 4090 (24GB) | \~$0.70/gpu/hr        |
| Large Scale / High-res Scenes | A100 80GB       | \~$1.20/gpu/hr        |

> 💡 All examples in this guide can be deployed on [Clore.ai](https://clore.ai/marketplace) GPU servers. Browse available GPUs and rent by the hour — no commitments, full root access.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.clore.ai/guides/3d-generation/nerfstudio.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.