Camera-Lidar Fusion for Road Detection: A Vision Transformer-Based Study

Autonomous vehicles require accurate road detection for safe navigation, and often fuse Camera and LiDAR sensors to enhance perception capabilities. While previous studies suggested that fusing Camera with LiDAR yields enhanced performance over unimodal approaches, recent studies in Vision Transformers (ViTs) are challenging this notion. This study compares fusion models with a Camera baseline using SegFormer-B2 on KITTI-360 with 8-fold cross-validation. Results show a trade-off: Camera achieves the highest observed Road IoU of 0.9600 and the fewest false positives. Early and late fusion performed similarly; late fusion reduced variance and improved worst-case Road IoU in the challenging sequence. Cross-attention fusion performed poorly, indicating complex architecture doesn't guarantee better results. The study concludes that, using the KITTI-360 dataset of clear weather, it is advisable to utilize efficient Camera detection rather than robust late fusion, which requires twice the resources. A 'Fusion Justification Test' was also proposed to guide future multimodal decisions based on robustness, recovery, and superiority over unimodal methods baselines.