# Aalap: A 32K context length Indian legal LLM

Aalap (Assistant for Legal and Paralegal functions in India) is an instructions fine-tuned version of Mistral 7B that can perform specific legal tasks in the Indian context. The details about which specific legal tasks Aalap can perform and training dataset can be found [here](https://huggingface.co/datasets/opennyaiorg/aalap_instruction_dataset).

This research model intends to show that we can develop tasks for the legal domain and teach LLMs to do them at an affordable cost.
The details about the dataset, model training, and evaluations can be found in the [paper](https://arxiv.org/abs/2402.01758). Aalap training & evaluation code can be found on the [repo](https://github.com/OpenNyAI/aalap_legal_llm).

<!-- Provide a quick summary of what the model is/does. -->

<!--Aalap(Assistant for Legal and Paralegal functions in India) is built for research purposes only and provides a single-turn response in tasks such as generating legal explanation and perform legal tasks over user given details and question. For more details about the task model is capable of checkout the [dataset](https://huggingface.co/datasets/opennyaiorg/aalap_instruction_dataset) used for training the model. The model is designed to excel particularly in generating explanations.



We make Aalap weights publicly available to support further research on the development. -->



## What is Aalap’s intended use(s)?
From the evaluation results, we can conclude that, for the tasks that are present in the training data, Aalap is performing comparably to ‘gpt-3.5-turbo’. But for the AIBE exam and Legalbench data, Aalap is not doing any better than the Mistral 7B base model. Hence Aalap is not a general-purpose India legal LLM but can do well under the constraints of the specific legal tasks.


## Model Details

Aalap is a finetuned version of Mistral 7b. Aalap’s training data is a synthetic dataset that was created to develop explanations and solve legal task capabilites.

Please refer to the [Aalap paper]() for details on the model architecture.

## License
Apache 2.0

## Getting started with Aalap

**Inference with Hugging Face library**

```python
import torch
import transformers
if torch.cuda.is_available():
    torch.set_default_device("cuda")
else:
    torch.set_default_device("cpu")
    
model = transformers.AutoModelForCausalLM.from_pretrained("opennyaiorg/Aalap-Mistral-7B-v0.1-bf16", device_map='auto')
tokenizer = transformers.AutoTokenizer.from_pretrained("opennyaiorg/Aalap-Mistral-7B-v0.1-bf16")


## load the dataset to test on samples
dataset_pandas = load_dataset("opennyaiorg/aalap_instruction_dataset", split='test').to_pandas()
def get_sample(task=None, skip_keyword=None):
    if task==None:
        task_list = list(set(dataset_pandas['task'].to_list()))
        if skip_keyword:
            task_list = [i for i in task_list if skip_keyword not in i]
        task=random.choice(task_list)
    print('Random sample selected from ',task)
    filtered = dataset_pandas[dataset_pandas.task==task]['combined_input_prompt'].to_list()
    choice = random.choice(filtered);prompt = choice.split('[/INST]')[0]+'[/INST]';response = choice.split('[/INST]')[-1]
    return prompt,response


## Provide your custom prompt
user_prompt = 'Who are you and what are your capabilites?'
system_prompt = '''You are helpful assistant.'''
prompt = f'<s> [INST] <<SYS>> {system_prompt} <</SYS>> {user_prompt} [/INST]'

inputs = tokenizer(prompt, return_tensors='pt')
output_ids = model.generate(inputs["input_ids"],)
answer = tokenizer.batch_decode(output_ids)[0]

print("Custom prompt answer:\n"answer)

## Test on a random data point from dataset
prompt, response = get_sample(task=None,skip_keyword=None)

inputs = tokenizer(prompt, return_tensors='pt')
output_ids = model.generate(inputs["input_ids"],)
answer = tokenizer.batch_decode(output_ids)[0]

print("Random prompt answer:\n"answer)
```



## Limitations:

1. We have synthetically generated data for various legal tasks and used a portion of the ORCA dataset relevant to law for teach LLM do explanation and certain tasks.
2. The model is not optimized for chat and supports single-turn conversations.
3. The model has not been trained with RLHF or DPO. It can sometimes generate wrong information, and hence, it is recommended to review the answers by an expert before using them.
4. Beyond explanation generation, the model inherits the capabilities and limitations of its base (Mistral 7B).

## Bias, Risks, and Limitations

Aalap, built upon the Mistral 7B, retains many of its limitations, as well as the
common limitations of other large language models or limitation caused by its training process,
including:

**Data Biases**: Large language models, trained on extensive data, can inadvertently carry
biases present in the source data. Consequently, the models may generate outputs that could
be potentially biased or unfair.

**Lack of Contextual Understanding**: Despite their impressive capabilities in language understanding and generation, these models exhibit limited real-world understanding, resulting
in potential inaccuracies or nonsensical responses.

**Lack of Transparency**: Due to the complexity and size, large language models can act
as “black boxes”, making it difficult to comprehend the rationale behind specific outputs or
decisions. We have tried to generated explanations to solve this but there are some limitations still present.

**Content Harms**: There are various types of content harms that large language models
can cause. It is important to be aware of them when using these models, and to take
actions to prevent them. It is recommended to leverage various content moderation services
provided by different companies and institutions. On an important note, we hope for better
regulations and standards from government and technology leaders around content harms
for AI technologies in future. We value and acknowledge the important role that research
and open source community can play in this direction.

**Hallucination**: It is important to be aware and cautious not to entirely rely on a given
language model for critical decisions or information that might have deep impact as it is
not obvious how to prevent these models from fabricating content. Moreover, it is not clear
whether small models may be more susceptible to hallucination in ungrounded generation
use cases due to their smaller sizes and hence reduced memorization capacities. This is an
active research topic and we hope there will be more rigorous measurement, understanding
and mitigations around this topic.

**Potential for Misuse**: Without suitable safeguards, there is a risk that these models could
be maliciously used for generating disinformation or harmful content.

**Data Distribution**: Aalap’s performance is likely to correlate strongly with the distribution
of the tuning data. This correlation might limit its accuracy in areas underrepresented in
the training dataset such as summary generation, reasoning, in-context learning, etc...

**System messages**: Aalap demonstrates variance in performance depending on the system
instructions.

**Zero-Shot Settings**: Aalap was trained on data that mostly simulate zero-shot settings.
While the model demonstrate good performance in zero-shot settings, it does not show
the same gains of using few-shot learning compared to other.

**Synthetic data**: As Aalap is trained on synthetic data, it could inherit both the advantages
and shortcomings of the models and methods used for data generation.





## Citation
```bibtex
@misc{tiwari2024aalap,
      title={Aalap: AI Assistant for Legal & Paralegal Functions in India}, 
      author={Aman Tiwari and Prathamesh Kalamkar and Atreyo Banerjee and Saurabh Karn and Varun Hemachandran and Smita Gupta},
      year={2024},
      eprint={2402.01758},
      archivePrefix={arXiv},
      primaryClass={cs.CY}
}
```
{"language": ["en"], "license": "apache-2.0", "library_name": "transformers", "tags": ["legal", "aalap", "opennyai"], "datasets": ["opennyaiorg/aalap_instruction_dataset"], "pipeline_tag": "text-generation", "license_name": "other", "license_link": "LICENSE"}
transformers safetensors mistral text-generation legal aalap opennyai en dataset:opennyaiorg/aalap_instruction_dataset arxiv:2402.01758 license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us