Create README.md

README.md

@@ -0,0 +1,145 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- meta-llama/Llama-3.2-11B-Vision-Instruct
+pipeline_tag: visual-question-answering
+
+tags:
+
+- indox
+- phoenix
+- osllm.ai
+- language
+---
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+Llama-3.2V-11B-cot is the first version of [LLaVA-o1](https://github.com/PKU-YuanGroup/LLaVA-o1), which is a visual language model capable of spontaneous, systematic reasoning.
+
+## Model Details
+
+<!-- Provide a longer summary of what this model is. -->
+
+- **License:** apache-2.0
+- **Finetuned from model:** meta-llama/Llama-3.2-11B-Vision-Instruct
+
+## Benchmark Results
+
+| MMStar | MMBench | MMVet | MathVista | AI2D | Hallusion | Average |
+|--------|---------|-------|-----------|------|-----------|---------|
+| 57.6   | 75.0    | 60.3  | 54.8      | 85.7 | 47.8      | 63.5    |
+
+## Reproduction
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+To reproduce our results, you should use [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) and the following settings.
+
+| Parameter         | Value   |
+|-------------------|---------|
+| do_sample         | True    |
+| temperature       | 0.6     |
+| top_p             | 0.9     |
+| max_new_tokens    | 2048    |
+
+You may change them in [this file](https://github.com/open-compass/VLMEvalKit/blob/main/vlmeval/vlm/llama_vision.py), line 80-83, and modify the max_new_tokens throughout the file.
+
+Note: We follow the same settings as Llama-3.2-11B-Vision-Instruct, except that we extend the max_new_tokens to 2048.
+
+After you get the results, you should filter the model output and only **keep the outputs between \<CONCLUSION\> and \</CONCLUSION\>**.
+
+This shouldn't have any difference in theory, but empirically we observe some performance difference because the jugder GPT-4o can be inaccurate sometimes.
+
+By keeping the outputs between \<CONCLUSION\> and \</CONCLUSION\>, most answers can be direclty extracted using VLMEvalKit system, which can be much less biased.
+
+## How to Get Started with the Model
+
+You can use the inference code for Llama-3.2-11B-Vision-Instruct.
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+The model is trained on the LLaVA-o1-100k dataset (to be released).
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+The model is finetuned on [llama-recipes](https://github.com/Meta-Llama/llama-recipes) with the following settings.
+Using the same setting should accurately reproduce our results.
+
+| Parameter                     | Value                                             |
+|-------------------------------|---------------------------------------------------|
+| FSDP                          | enabled                                           |
+| lr                            | 1e-5                                              |
+| num_epochs                    | 3                                                 |
+| batch_size_training           | 4                                                 |
+| use_fast_kernels              | True                                              |
+| run_validation                | False                                             |
+| batching_strategy             | padding                                           |
+| context_length                | 4096                                              |
+| gradient_accumulation_steps   | 1                                                 |
+| gradient_clipping             | False                                             |
+| gradient_clipping_threshold   | 1.0                                               |
+| weight_decay                  | 0.0                                               |
+| gamma                         | 0.85                                              |
+| seed                          | 42                                                |
+| use_fp16                      | False                                             |
+| mixed_precision               | True                                              |
+
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+The model may generate biased or offensive content, similar to other VLMs, due to limitations in the training data. 
+Technically, the model's performance in aspects like instruction following still falls short of leading industry models.
+
+
+
+
+**About [osllm.ai](https://osllm.ai)**:
+
+[osllm.ai](https://osllm.ai) is a community-driven platform that provides access to a wide range of open-source language models.
+
+1. **[IndoxJudge](https://github.com/indoxJudge)**: A free, open-source tool for evaluating large language models (LLMs).  
+It provides key metrics to assess performance, reliability, and risks like bias and toxicity, helping ensure model safety.
+
+1. **[inDox](https://github.com/inDox)**: An open-source retrieval augmentation tool for extracting data from various  
+document formats (text, PDFs, HTML, Markdown, LaTeX). It handles structured and unstructured data and supports both  
+online and offline LLMs.
+
+1. **[IndoxGen](https://github.com/IndoxGen)**: A framework for generating high-fidelity synthetic data using LLMs and  
+human feedback, designed for enterprise use with high flexibility and precision.
+
+1. **[Phoenix](https://github.com/Phoenix)**: A multi-platform, open-source chatbot that interacts with documents  
+locally, without internet or GPU. It integrates inDox and IndoxJudge to improve accuracy and prevent hallucinations,  
+ideal for sensitive fields like healthcare.
+
+1. **[Phoenix_cli](https://github.com/Phoenix_cli)**: A multi-platform command-line tool that runs LLaMA models locally,  
+supporting up to eight concurrent tasks through multithreading, eliminating the need for cloud-based services.
+
+
+
+**Disclaimers**
+
+[osllm.ai](https://osllm.ai) is not the creator, originator, or owner of any Model featured in the Community Model Program.  
+Each Community Model is created and provided by third parties. osllm.ai does not endorse, support, represent,  
+or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model. You understand  
+that Community Models can produce content that might be offensive, harmful, inaccurate, or otherwise  
+inappropriate, or deceptive. Each Community Model is the sole responsibility of the person or entity who  
+originated such Model. osllm.ai may not monitor or control the Community Models and cannot, and does not, take  
+responsibility for any such Model. osllm.ai disclaims all warranties or guarantees about the accuracy,  
+reliability, or benefits of the Community Models. osllm.ai further disclaims any warranty that the Community  
+Model will meet your requirements, be secure, uninterrupted, or available at any time or location, or  
+error-free, virus-free, or that any errors will be corrected, or otherwise. You will be solely responsible for  
+any damage resulting from your use of or access to the Community Models, your downloading of any Community  
+Model, or use of any other Community Model provided by or through [osllm.ai](https://osllm.ai).
+
+