|
--- |
|
tags: |
|
- autotrain |
|
- text-generation-inference |
|
- text-generation |
|
- peft |
|
library_name: transformers |
|
base_model: allenai/Llama-3.1-Tulu-3-8B |
|
widget: |
|
- messages: |
|
- role: user |
|
content: What are the requirements for cross-examination according to Indian law? |
|
license: other |
|
--- |
|
# InLawMate-peft: Indian Legal Domain PEFT Model |
|
|
|
## Model Description |
|
InLawMate-peft is a Parameter-Efficient Fine-Tuned (PEFT) language model specifically optimized for understanding and reasoning about Indian legal documentation. The model was trained on a carefully curated dataset of nearly 7,000 question-answer pairs derived from Indian criminal law documentation, making it particularly adept at legal comprehension and explanation tasks. |
|
|
|
## Training Data |
|
The training data consists of nearly 7,000 high-quality legal Q&A pairs that were systematically generated using a sophisticated two-stage process: |
|
1. **Question Generation**: Questions were extracted to cover key legal concepts, definitions, procedures, and roles, ensuring comprehensive coverage of: |
|
- Legal terminology and definitions |
|
- Procedural rules and steps |
|
- Rights and penalties |
|
- Jurisdictional aspects |
|
- Roles of legal entities (judges, lawyers, law enforcement) |
|
|
|
2. **Answer Generation**: Answers were crafted following a structured legal reasoning approach, ensuring: |
|
- Legal precision and accuracy |
|
- Comprehensive coverage of relevant points |
|
- Clear explanation of legal concepts |
|
- Professional legal discourse style |
|
|
|
## Training Details |
|
- **Base Model**: allenai/Llama-3.1-Tulu-3-8B |
|
- **Architecture**: PEFT (Parameter-Efficient Fine-Tuning) |
|
- **Training Epochs**: 3 |
|
- **Batch Size**: 2 (with gradient accumulation steps of 4) |
|
- **Learning Rate**: 3e-05 with cosine scheduler |
|
- **Sequence Length**: 1024 tokens |
|
- **Mixed Precision**: BF16 |
|
- **Optimization**: AdamW with β1=0.9, β2=0.999 |
|
|
|
## Use Cases |
|
This model is particularly suited for: |
|
- Legal document analysis and comprehension |
|
- Answering questions about Indian criminal law |
|
- Understanding legal procedures and requirements |
|
- Explaining legal concepts and terminology |
|
- Assisting in legal research and education |
|
|
|
## Limitations |
|
- The model is specifically trained on Indian legal documentation |
|
- Responses should be verified by legal professionals for critical applications |
|
- The model should not be used as a substitute for professional legal advice |
|
|
|
## Usage |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
"aryaman/legalpara-lm", |
|
device_map="auto", |
|
torch_dtype='auto' |
|
).eval() |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("Aryaman02/InLawMate-peft") |
|
|
|
# Example legal query |
|
messages = [ |
|
{"role": "user", "content": "What are the requirements for cross-examination according to Indian law?"} |
|
] |
|
|
|
input_ids = tokenizer.apply_chat_template( |
|
conversation=messages, |
|
tokenize=True, |
|
add_generation_prompt=True, |
|
return_tensors='pt' |
|
) |
|
output_ids = model.generate(input_ids.to('cuda')) |
|
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True) |
|
print(response) |
|
``` |
|
|
|
## Citation |
|
If you use this model in your research, please cite: |
|
```bibtex |
|
@misc{legalpara-lm, |
|
title={InLawMate: A PEFT Model for Indian Legal Domain Understanding}, |
|
year={2024}, |
|
publisher={Aryaman}, |
|
note={Model trained on Indian legal documentation} |
|
} |
|
``` |
|
Our training data and procedure for synth data creation is outlined in https://github.com/DarryCrucian/law-llm |
