|
--- |
|
license: cc-by-4.0 |
|
datasets: |
|
- RussRobin/SpatialQA |
|
language: |
|
- en |
|
tags: |
|
- Embodied AI |
|
- MLLM |
|
- VLM |
|
- Spatial Understanding |
|
- Phi-2 |
|
pipeline_tag: visual-question-answering |
|
--- |
|
|
|
SpatialBot is a VLM with spatial understanding and reasoning abilties, by precisely understanding depth maps and using them to do high-level tasks. |
|
|
|
In this HF repo, we provide ckpts of SpatialBot-3B with LoRA, which is based on Phi-2 and SigLIP. It can perform well on general VLM tasks and spatial understanding benchmarks like SpatialBench. |
|
|
|
You will also need to download [pretrained CKPT](https://huggingface.co/RussRobin/SpatialBot-3B-pretrain). |
|
### Paper: |
|
https://arxiv.org/abs/2406.13642 |
|
|
|
### GitHub repo: |
|
https://github.com/BAAI-DCAI/SpatialBot |
|
|
|
<!-- ### SpatialQA, the training set: |
|
https://huggingface.co/datasets/RussRobin/SpatialQA --> |
|
|
|
### SpatialBench, the benchmark: |
|
https://huggingface.co/datasets/RussRobin/SpatialBench |
|
|
|
### Merged SpatialBot-3B: |
|
https://huggingface.co/RussRobin/SpatialBot-3B |
