An investigation for RGBL tracking. Hopefully, it can help other researchers become familiar with multi-modal tracking as soon as possible. This repository is started on 27/12/2023, and will keep on updating.
This repository will give a detail investigation of the RGBL tracking community, including the Datasets, Results, and the Methods.
- Datasets
- Results
- Methods
- ...
🫵Find our survey work at another repo
| Dataset | Publish | GitHub | Introduction |
|---|---|---|---|
| UniMod1K | IJCV'2024 | UniMod1K | RGB+D+L: UniMod1K: Towards a More Universal Large-Scale Dataset and Benchmark for Multi-modal Learning |
| WebUAV-3M | TPAMI'2023 | WebUAV-3M | RGB+L+Audio: WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking |
| Dataset | Publish | GitHub | Introduction |
|---|---|---|---|
| DTVLT | Arxiv'2024 | DTVLT | DTVLT: A MULTI-MODAL DIVERSE TEXT BENCHMARK FOR VISUAL LANGUAGE TRACKING BASED ON LLM |
| VLT-MI | Arxiv'2024 | VLT-MI | Visual Language Tracking with Multi-modal Interaction: A Robust Benchmark |
| ElysiumTrack-1M | Arxiv'2024 | ElysiumTrack-1M | Elysium: Exploring Object-level Perception in Videos via MLLM |
| WebUOT-1M | Arxiv'2024 | WebUOT-1M | WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark |
| VastTrack | Arxiv'2024 | VastTrack | VastTrack: Vast Category Visual Object Tracking |
| MGIT | NIPS'2023 | MGIT | A Multi-modal Global Instance Tracking Benchmark (MGIT): Better Locating Target in Complex Spatio-temporal and Causal Relationship |
| TNL2K | CVPR'2021 | TNL2K | Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark |
| LaSOT_EXT | IJCV'2021 | LaSOT_EXT | LaSOT: A High-quality Large-scale Single Object Tracking Benchmark |
| LaSOT | CVPR'2019 | LaSOT | LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking |
| OTB99-L | CVPR'2017 | OTB99-L | Tracking by Natural Language Specification |
| OTB99_L | TNL2K | |||||||
|---|---|---|---|---|---|---|---|---|
| Methods | Venue | Speed | PR | NPR | SR | PR | NPR | SR |
| MAVLT | TSCVT'2025 | 42.6/3090 | 93.5 | 71.6 | 58.1 | 57.0 | ||
| A-CLIP | EAAI'2025 | 31/2080Super | 89.2 | 69.0 | 53.8 | 49.5 | ||
| ProVLT | TCSVT'2025 | 64/3090 | 94.5 | 72.4 | 61.8 | 59.8 | ||
| SUTrack | AAAI'2025 | 93.1 | 71.2 | 72.1 | 67.9 | |||
| MugTracker | TCSVT'2024 | 92.4 | 69.8 | 58.1 | 56.5 | |||
| PGG | TIST'2024 | 90.8 | 69.3 | 57.3 | 56.3 | |||
| MambaVLT | Arxiv'2024 | 94.4 | 88.1 | 72.2 | 69.9 | 90.9 | 66.5 | |
| ChatTracker | NIPS'2024 | 70.2 | 76.5 | 65.4 | ||||
| ATTracker | ACMMM'2024 | 88.6 | 67.6 | 56.1 | 69.9 | 52.2 | ||
| UnTrack | CVPR'2024 | 91.5 | 69.7 | 59.1 | 58.0 | |||
| SeqTrackv2 | Arxiv'2024 | 93.4 | 71.8 | 66.0 | 62.7 | |||
| QueryNLT | CVPR'2024 | 88.2 | 82.4 | 66.7 | 58.7 | 75.6 | 57.8 | |
| MFAVLT | Remote Sensing'2024 | 87.2 | 65.4 | 51.3 | 44.7 | |||
| OSDT | TCSVT'2024 | 67 | 86.7 | 78.0 | 66.2 | 61.5 | 76.2 | 59.3 |
| TTCTrack | TCSVT'2024 | 52 | 70.1 | 58.1 | ||||
| VLT_OST | TPAMI'2024 | 95.0 | 77.7 | 54.5 | 55.7 | |||
| MMTrack | TCSVT'2024 | 91.8 | 70.5 | 59.4 | 75.2 | 58.6 | ||
| UVLTrack | AAAI'2024 | 92.0 | 71.1 | 69.3 | 64.9 | |||
| All in One | ICCV'2023 | 60 | 93.0 | 71.0 | 57.2 | 55.3 | ||
| SATracker | Arxiv'2023 | 94.3 | 74.2 | 64.4 | 78.4 | 61.6 | ||
| CiteTracker | ICCV'2023 | 92.2 | 85.1 | 69.6 | 59.6 | 57.7 | ||
| JointNLT | CVPR'2023 | 85.6 | 65.3 | 58.1 | 56.9 | |||
| VLATrack | RICAI'2023 | 93.3 | 72.0 | 65.9 | 79.7 | 62.6 | ||
| OVLM | TMM'2023 | 69.3 | 82.6 | 64.7 | ||||
| TransNLT | TCSVT'2023 | 87.0 | 67.0 | 57.0 | 75.0 | 57.0 | ||
| DecoupleNLT | ICCV'2023 | 94.8 | 73.8 | 56.0 | 56.7 | |||
| PTG | PRL'2023 | 91.5 | 70.6 | 61.7 | 56.0 | |||
| AdaRS | CVPRW'2023 | 91.0 | 69.0 | 45.0 | 52.0 | 44.0 | ||
| VLT_TT | NIPS'2022 | 93.1 | 76.4 | 53.3 | 53.1 | |||
| CapsuleNLT | ACMMM'2021 | 92.4 | 71.1 | |||||
| SNLT | CVPR'2021 | 84.8 | 80.4 | 66.6 | ||||
| RTTNLD | WACV'2020 | 79.0 | 73.0 | 61.0 | ||||
| LaSOT | LaSOT_EXT | |||||||
|---|---|---|---|---|---|---|---|---|
| MAVLT | TSCVT'2025 | 42.6/3090 | 81.3 | 86.6 | 73.4 | 65.8 | 69.3 | 57.5 |
| A-CLIP | EAAI'2025 | 31/2080Super | 71.3 | 66.4 | ||||
| ProVLT | TCSVT'2025 | 64/3090 | 77.3 | 70.7 | 58.8 | 51.4 | ||
| MugTracker | TCSVT'2024 | 75.9 | 70.4 | |||||
| PGG | TIST'2024 | 75.3 | 70.1 | |||||
| ChatTracker | NIPS'2024 | 81.2 | 83.8 | 74.1 | ||||
| MambaVLT | Arxiv'2024 | 71.0 | 77.3 | 66.6 | ||||
| ATTracker | ACMMM'2024 | 60.5 | 60.4 | |||||
| QueryNLT | CVPR'2024 | 63.5 | 69.6 | 59.9 | ||||
| MFAVLT | Remote Sensing'2024 | 55.6 | 55.3 | |||||
| OSDT | TCSVT'2024 | 67 | 68.6 | 73.4 | 64.3 | |||
| TTCTrack | TCSVT'2024 | 52 | 67.6 | 48.8 | ||||
| VLT_OST | TPAMI'2024 | 77.0 | 70.5 | 54.8 | 48.6 | |||
| MMTrack | TCSVT'2024 | 75.7 | 82.3 | 70.0 | 55.3 | 59.9 | 49.4 | |
| UVLTrack | AAAI'2024 | 78.7 | 71.4 | |||||
| All in One | ICCV'2023 | 60 | 78.5 | 82.4 | 71.7 | 66.0 | 54.5 | |
| SATracker | Arxiv'2023 | 79.6 | 72.4 | |||||
| CiteTracker | ICCV'2023 | 75.7 | 78.6 | 69.7 | ||||
| JointNLT | CVPR'2023 | 63.6 | 60.4 | |||||
| VLATrack | RICAI'2023 | 73.8 | 77.7 | 68.2 | ||||
| OVLM | TMM'2023 | 74.2 | 77.6 | 67.7 | ||||
| DecoupleNLT | ICCV'2023 | 75.3 | 71.2 | |||||
| PTG | PRL'2023 | 70.8 | 66.4 | |||||
| AdaRS | CVPRW'2023 | 56.0 | 53.0 | |||||
| VLT_TT | NIPS'2022 | 72.1 | 67.3 | 55.9 | 48.4 | |||
| CapsuleNLT | ACMMM'2021 | 63.3 | 71.1 | 61.5 | ||||
| SNLT | CVPR'2021 | 57.4 | 63.6 | 54.0 | ||||
| RTTNLD | WACV'2020 | 35.0 | 43.0 | 35.0 | ||||
| TransNLT | TCSVT'2023 | 63.0 | 60.0 | |||||
| MGIT | VastTrack | WebUOT-1M | WevUAV-3M | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Methods | Venue | Speed | PR | NPR | SR | PR | NPR | SR | PR | NPR | SR | PR | NPR | SR |
| MAVLT | TSCVT'2025 | 42.6/3090 | 74.6 | 59.0 | ||||||||||
| MambaVLT | Arxiv'2024 | 58.9 | 78.0 | 69.9 | ||||||||||
| UVLTrack | AAAI'2024 | 52.5 | 60.0 | 55.8 | ||||||||||
| All in One | ICCV'2023 | 60 | 53.1 | 61.5 | 57.1 | 71.5 | 60.5 | 56.8 | ||||||
| JointNLT | CVPR'2023 | 25.5 | 34.9 | 32.7 | ||||||||||
| CiteTracker | ICCV'2023 | 49.3 | 58.4 | 54.6 | ||||||||||
| VLT_TT | NIPS'2022 | 41.7 | 52.1 | 48.3 | ||||||||||
Questions
If you have any questions, please contact [email protected], and wechat: Tzy18861871359 is also welcomed.