Collections

9

DocLLM: A layout-aware generative language model for multimodal document understanding

Paper • 2401.00908 • Published Dec 31, 2023 • 181
COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

Paper • 2401.00849 • Published Jan 1, 2024 • 16
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents

Paper • 2311.05437 • Published Nov 9, 2023 • 48
LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing

Paper • 2311.00571 • Published Nov 1, 2023 • 41

DocLLM: A layout-aware generative language model for multimodal document understanding

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents

LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing

Octo-planner: On-device Language Model for Planner-Action Agents

Adaptable Logical Control for Large Language Models

SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models

MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding

Transformers meet Neural Algorithmic Reasoners

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models

The Prompt Report: A Systematic Survey of Prompting Techniques

CRAG -- Comprehensive RAG Benchmark

Transformers meet Neural Algorithmic Reasoners

MotionLLM: Understanding Human Behaviors from Human Motions and Videos

Spectrally Pruned Gaussian Fields with Neural Compensation

Paint by Inpaint: Learning to Add Image Objects by Removing Them First

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report

iVideoGPT: Interactive VideoGPTs are Scalable World Models

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

An Introduction to Vision-Language Modeling

Matryoshka Multimodal Models

Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation

How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

MoAI: Mixture of All Intelligence for Large Language and Vision Models

MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training

BLINK: Multimodal Large Language Models Can See but Not Perceive

TextSquare: Scaling up Text-Centric Visual Instruction Tuning

Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models

InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD

EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters

Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

EfficientViT-SAM: Accelerated Segment Anything Model Without Performance Loss

TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers

From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations