Documentation

Everything you need to install Open4D, understand its data model, and start working with time-varying 4D geometry.

Introduction

Open4D is an open, research-driven platform for the representation, compression, processing, evaluation, and streaming of time-varying 4D geometry data. It serves as shared infrastructure for application domains such as XR systems, robotics, teleoperation, digital twins, and autonomous systems, where geometry evolves over time and must be processed under tight latency, bandwidth, and accuracy constraints.

The platform is organized into five layers:

• Core (open4d/) — canonical 4D data models, IO, metrics
• Modules — research algorithms (e.g., TVMC, ARAP-based tracking)
• Benchmarks — paper-reproducible evaluation code, baselines, configs, and scripts
• Apps — end-to-end pipelines and demos
• Bindings (cpp/, python/) — performance-critical and research-friendly APIs

Installation

Requirements

• Python 3.9 or later
• Git (with submodule support, for the N4MC and TSMC compression modules)
• Unity 2022 LTS or later (only for the XR decoder plugin)

Development install

git clone https://github.com/open4dfoundation/Open4D.git
cd Open4D
pip install -e .

With compression submodules

N4MC and TSMC are integrated as git submodules. To fetch them along with the core:

git clone --recurse-submodules https://github.com/open4dfoundation/Open4D.git

# or, in an existing clone
git submodule update --init --recursive

Quickstart

The examples/ directory contains minimal, runnable scripts for the most common tasks:

# play a time-varying mesh sequence
python examples/play_mesh_o4d.py

# play a point-cloud sequence
python examples/play_poincloud_o4d.py

# play a Draco-compressed point-cloud sequence
python examples/play_draco_pointcloud_o4d.py

A typical workflow — convert raw per-frame data into an .o4d sequence, then play it back:

# 1. package raw mesh frames into an .o4d sequence
python open4d/tools/create_o4d_mesh.py

# 2. play it back
python examples/play_mesh_o4d.py

Repository Structure

Open4D/
├── open4d/          # Core public API (stable)
│   ├── core/        # 4D data structures + core codecs (N4MC, TSMC, TVMC)
│   ├── io/          # Readers / writers
│   ├── metrics/     # Quality + temporal metrics
│   ├── modules/     # Research modules (tvmc, unity_decoder)
│   ├── player/      # Playback for meshes and point clouds
│   └── tools/       # .o4d creation utilities
├── benchmarks/      # Reproducible experiments + baselines
├── apps/            # End-to-end pipelines
├── examples/        # Minimal usage examples
├── tests/           # Unit + integration tests
├── docs/            # Documentation and figures
├── scripts/         # Helper scripts
└── docker/          # Reproducible environments

The .o4d Data Model

Open4D treats time as a first-class signal. Instead of a directory of unrelated per-frame files, an .o4d sequence is a single logical object: an ordered set of geometry frames plus timing metadata, so downstream code — codecs, metrics, players — can reason about temporal structure directly.

Supported content types in the current core:

Design Principles

• 4D-first — time is a first-class signal, not an afterthought
• Separation of concerns — core abstractions vs. research modules
• Reproducibility — benchmarks are explicit and scriptable
• Systems-aware — metrics include bitrate, latency, and temporal stability
• Cross-domain — XR, robotics, and autonomy share the same foundations

IO — Readers & Writers

The open4d.io package provides a unified interface for reading and writing 4D sequences. Each content type has a dedicated IO module with a symmetric read/write API, so codecs and players are decoupled from on-disk formats.

from open4d import io

# read a time-varying mesh sequence
seq = io.o4d_mesh_io.read("longdress.o4d")

# iterate frames over time
for frame in seq:
    process(frame)

Player

open4d.player renders sequences interactively. It supports mesh sequences, raw point-cloud sequences, and Draco-compressed point-cloud streams — decode happens frame-by-frame, matching how a streaming client would consume the data.

Conversion Tools

open4d.tools packages raw per-frame assets (e.g., a folder of PLY/OBJ frames) into .o4d sequences, including Draco-compressed variants for bandwidth-constrained playback.

Metrics

open4d.metrics hosts quality and temporal metrics used across benchmarks. The design goal is systems-aware evaluation: not just geometric distortion, but the quantities that decide whether a pipeline is deployable —

• Geometric quality — point-to-point / point-to-plane distortion, mesh distance
• Rate — bitrate and compression ratio
• Latency — encode/decode time budgets for real-time pipelines
• Temporal stability — consistency of geometry across frames, beyond per-frame error

Docker Environments

The docker/ directory provides containerized environments so that experiments run the same way on any machine — a prerequisite for reproducible benchmarks.

FAQ

How is Open4D different from Open3D?

Open3D focuses on static 3D data processing. Open4D targets the time-varying case: sequences of meshes and point clouds whose geometry (and even connectivity) changes every frame, with compression, streaming, and temporal metrics as core concerns rather than extensions.

Can I use only the codecs?

Yes. N4MC, TSMC, and TVMC live in their own repositories/submodules and can be used standalone; Open4D adds the shared IO, metrics, and benchmark scaffolding around them.

How do I contribute?

New modules, benchmarks, datasets, metrics, and documentation are all welcome — see Contributing.