File size: 3,529 Bytes
b4b809f 719aab0 86da2a2 ece0b5f 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 719aab0 86da2a2 b4b809f 86da2a2 5a2e9fc |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 |
---
tags:
- autotrain
- text-generation-inference
- text-generation
- peft
library_name: transformers
base_model: allenai/Llama-3.1-Tulu-3-8B
widget:
- messages:
- role: user
content: What are the requirements for cross-examination according to Indian law?
license: other
---
# InLawMate-peft: Indian Legal Domain PEFT Model
## Model Description
InLawMate-peft is a Parameter-Efficient Fine-Tuned (PEFT) language model specifically optimized for understanding and reasoning about Indian legal documentation. The model was trained on a carefully curated dataset of nearly 7,000 question-answer pairs derived from Indian criminal law documentation, making it particularly adept at legal comprehension and explanation tasks.
## Training Data
The training data consists of nearly 7,000 high-quality legal Q&A pairs that were systematically generated using a sophisticated two-stage process:
1. **Question Generation**: Questions were extracted to cover key legal concepts, definitions, procedures, and roles, ensuring comprehensive coverage of:
- Legal terminology and definitions
- Procedural rules and steps
- Rights and penalties
- Jurisdictional aspects
- Roles of legal entities (judges, lawyers, law enforcement)
2. **Answer Generation**: Answers were crafted following a structured legal reasoning approach, ensuring:
- Legal precision and accuracy
- Comprehensive coverage of relevant points
- Clear explanation of legal concepts
- Professional legal discourse style
## Training Details
- **Base Model**: allenai/Llama-3.1-Tulu-3-8B
- **Architecture**: PEFT (Parameter-Efficient Fine-Tuning)
- **Training Epochs**: 3
- **Batch Size**: 2 (with gradient accumulation steps of 4)
- **Learning Rate**: 3e-05 with cosine scheduler
- **Sequence Length**: 1024 tokens
- **Mixed Precision**: BF16
- **Optimization**: AdamW with β1=0.9, β2=0.999
## Use Cases
This model is particularly suited for:
- Legal document analysis and comprehension
- Answering questions about Indian criminal law
- Understanding legal procedures and requirements
- Explaining legal concepts and terminology
- Assisting in legal research and education
## Limitations
- The model is specifically trained on Indian legal documentation
- Responses should be verified by legal professionals for critical applications
- The model should not be used as a substitute for professional legal advice
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"aryaman/legalpara-lm",
device_map="auto",
torch_dtype='auto'
).eval()
tokenizer = AutoTokenizer.from_pretrained("Aryaman02/InLawMate-peft")
# Example legal query
messages = [
{"role": "user", "content": "What are the requirements for cross-examination according to Indian law?"}
]
input_ids = tokenizer.apply_chat_template(
conversation=messages,
tokenize=True,
add_generation_prompt=True,
return_tensors='pt'
)
output_ids = model.generate(input_ids.to('cuda'))
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)
```
## Citation
If you use this model in your research, please cite:
```bibtex
@misc{legalpara-lm,
title={InLawMate: A PEFT Model for Indian Legal Domain Understanding},
year={2024},
publisher={Aryaman},
note={Model trained on Indian legal documentation}
}
```
Our training data and procedure for synth data creation is outlined in https://github.com/DarryCrucian/law-llm |