Conference on Robot Learning · 2026

VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detection

Haowen Sun, Shaolong Zhang, Mingyang Li, Chengzhong Ma,
Xinzhe Chen, Qiongjie Cui, Xingyu Chen, Zeyang Liu*, Xuguang Lan*

Xi'an Jiaotong University

National Key Laboratory of Human-Machine Hybrid Augmented Intelligence
Institute of Artificial Intelligence and Robotics

Paper PDF BibTeX
A point-cloud observation and manipulation pipeline comparing language tokens with voxel-enhanced affordance tokens.
VoxAfford combines language-model semantics with multi-scale voxel geometry to locate interaction regions in 3D.

Abstract

Open-vocabulary 3D affordance detection localizes the regions of an object that support a language-described interaction. Language-model output tokens provide affordance semantics, while accurate masks also depend on spatial structure.

VoxAfford retrieves complementary geometric features from a frozen 3D VQVAE at three resolutions. Cross-attention and a learned scalar compatibility gate enhance each generated affordance token. Query-conditioned aggregation forms a spatially aware prompt, which works alongside point features to predict the final mask.

Read the full abstract in the paper.

Method

The point cloud and language query enter a LoRA-adapted LLM, which generates three affordance tokens. Each token retrieves geometry from its paired VQVAE scale (163, 323, or 643). A scalar compatibility gate controls the geometric injection. A semantic query then aggregates the enhanced tokens into a prompt for mask prediction.

VoxAfford's three components: multimodal input encoding, hierarchical voxel-token spatial fusion, and affordance-aware spatial propagation.
The frozen voxel encoder supplies geometry; the LLM supplies affordance semantics.

Results

OpenAD open-set evaluation with full-object point clouds and partial observations. All values below are instance-level metrics in percent; higher is better.

Full-view instance-level metrics, percentages
MethodmIoU ↑mAcc ↑mAP50 ↑
3D-ADLLM30.2870.6627.80
3D-ADLLM†32.6073.2135.13
VoxAfford40.5777.6840.67

† Matched pretraining: PartNet segmentation pretraining and ShapeLLM-Omni VQVAE features at the LLM input. Values are from Table 1 of the paper.

Qualitative comparisons

Point-cloud affordance predictions from OpenAD, LASO, 3D-ADLLM, VoxAfford, and ground-truth annotations across multiple objects.
Open-set examples from Figure 3. Click any figure to enlarge it.

Real robot experiments

VoxAfford succeeds in 33 of 50 trials (66%) across five novel objects. A Franka Research 3 arm and three Orbbec Femto Bolt RGB-D cameras provide the manipulation platform and scene point clouds.

Two real-robot sequences: grasping and lifting a banana while avoiding a carrot, and grasping a ladle and transferring it into a red bowl.
Representative trials from Figure 5: banana grasping and ladle manipulation.

Camera viewpoints and extrinsics remain fixed. Object pose, robot initial state, and clutter vary across trials. Both methods use paired scenes and the same execution pipeline.

Successful trials
Object3D-ADLLMVoxAfford
Cup with handle5 / 108 / 10
Cup without handle0 / 105 / 10
Fruits6 / 106 / 10
Ladle5 / 105 / 10
Pan2 / 109 / 10
Overall18 / 5033 / 50
Evaluation scope and limitations

VoxAfford relies on a frozen pre-trained VQVAE and fixed token–scale retrieval pairing. Evaluation focuses on isolated objects and one robot platform. Noisy or partial observations, segmentation, calibration, and reconstruction errors can affect localization and execution. See the paper for the full protocol and failure analysis.

Citation

@inproceedings{sun2026voxafford,
  title     = {{VoxAfford}: Multi-Scale Voxel-Token Fusion
               for Open-Vocabulary {3D} Affordance Detection},
  author    = {Sun, Haowen and Zhang, Shaolong and Li, Mingyang
               and Ma, Chengzhong and Chen, Xinzhe and Cui, Qiongjie
               and Chen, Xingyu and Liu, Zeyang and Lan, Xuguang},
  booktitle = {Conference on Robot Learning},
  year      = {2026}
}

Figure