region:us
# MythoOrca Model Card

## Overview

**Model Name:** MythoOrca  
**Type:** Language Model  
**Version:** 4-bit Quantization  
**Mixture:** 50% MythoMax and 50% Orca 2  
**Developer:** Combination of two distinct AI research initiatives  

## Intended Use

- **Research:** MythoOrca is primarily intended for research purposes, combining the reasoning abilities of Orca 2 with the storytelling and roleplaying capabilities of MythoMax.
- **Roleplaying and Storywriting:** Given its heritage from MythoMax, it excels in these areas.
- **Reasoning and Comprehension:** Inherits Orca 2's strengths in reasoning, reading comprehension, and problem-solving.

## Model Architecture

- **Base Models:** Combines LLAMA-2 (Orca 2 base) and Huginn (from MythoMax).
- **Layer Composition:** Features a unique tensor type merge technique, with each of its 363 tensors having a unique ratio applied to it.

## Quantization

- **Bit-Depth:** 4-bit, optimizing the model for more efficient computation while maintaining performance.
- **Source:** Quantized models facilitated by TheBloke's GPTQ (GGML - GPTQ).

## Performance

- **Capabilities:** Enhanced coherence across the model structure, proficient in roleplaying, storywriting, and reasoning tasks.
- **Limitations:** As a quantized model, some nuances in performance compared to its full-bit counterparts might be observed.

## Data and Training

- **Training Data:** Synthetic dataset (Orca 2) and a blend of MythoLogic-L2 and Huginn data (MythoMax).
- **Training Method:** A combination of fine-tuning and an experimental tensor merge technique.

## Bias, Risks, and Limitations

- **Data Biases:** Inherits biases from both parent models, potentially reflecting biases in source data.
- **Contextual Understanding:** Limited real-world understanding, potentially leading to inaccuracies.
- **Transparency:** Complex architecture makes it difficult to understand specific output rationales.
- **Content Harms:** Potential to generate biased or harmful content. Usage of content moderation services recommended.
- **Hallucination:** Risk of fabricating content, especially in complex or ungrounded scenarios.
- **Misuse Potential:** Risk of being used to generate disinformation or harmful content.

## Usage Guidelines

- **Intended for Research:** Not optimized for downstream applications without further analysis.
- **Content Moderation:** Integration with content safety tools like Azure AI Content Safety recommended.
- **Prompt Format:** Best used with Alpaca formatting for optimal performance.

## Licensing

- **Orca 2 License:** Microsoft Research License.
- **LLaMA 2 License:** LLAMA 2 Community License, Copyright © Meta Platforms, Inc.

## Additional Information

- **Model Availability:** Publicly accessible for research purposes.
- **Documentation and Support:** Refer to the original papers and technical reports of MythoMax and Orca 2 for detailed information.

## Getting Started Example

Refer to Orca 2's implementation with Hugging Face's transformers library for a basic usage example, adapted for MythoOrca's unique capabilities.


---
license: llama2
---
