
## aurora-m-v0.1-biden-harris-redteamed
### The First Open Source Biden-Harris Executive Order Red-teamed Multilingual Model

Jan 24, 2024

### Model Description
This is version 0.1 of Aurora-m, a Starcoderplus Based 16B model that was continued pretrained on an additional approximatley 435B tokens. This version is an **experimental research version** that is meant to be used for multidomain and multilingual red-teaming research. This project was created as part of the MDEL efforts.

### Current languages: 
English, Japanese, Finnish, Vietnamese, Hindi, plus programming code

### Domain: 
chemical SMILEs formula, financial data, legal contracts, political debates, climate change data, ABC music notations, coding, math and many other domains.

### Dataset: 
- About 1.5TB of text data from the Stack, Refined Web, Red Pajama 1, and Pile dataset along with specific datasets created as part of the MDEL efforts.
- Several thousand red-teamed, and human reviewed and edited instructions to address generally safety concerns, and more specifically the concerns in the **Biden-Harris Executive Order on AI**.
- Public instruction datasets were also mixed in during continued pre-training.

### Red-team coverage:

Below is our reading of red-teaming requirements of the [Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. (2023, October 30). The White House.](https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/)

We focus specifically on this Section 3(d):
```
     (d)  The term “AI red-teaming” means a structured testing effort to find flaws and vulnerabilities in an AI system, often in a controlled environment and in collaboration with developers of AI.  Artificial Intelligence red-teaming is most often performed by dedicated “red teams” that adopt adversarial methods to identify flaws and vulnerabilities, such as harmful or discriminatory outputs from an AI system, unforeseen or undesirable system behaviors, limitations, or potential risks associated with the misuse of the system.
```

And this Section 3(k):
```
(k)  The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, such as by:

          (i)    substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;

          (ii)   enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyber attacks; or

          (iii)  permitting the evasion of human control or oversight through means of deception or obfuscation.

Models meet this definition even if they are provided to end users with technical safeguards that attempt to prevent users from taking advantage of the relevant unsafe capabilities.
```

#### So broadly, the Executive Order defines AI red-teaming as testing for flaws and vulnerabilities, including:

* Harmful or discriminatory outputs 
* Unforeseen or undesirable system behaviors. This connects to broader safety concerns outlined in the order.
* Limitations of the models itself. The aim is to assess the system's robustness and ability to fulfill its designed purpose.
* Potential risks associated with misuse of the system. This encompasses a wide range of concerns, including cybersecurity threats (as emphasized throughout the Order) and the potential for illegal or harmful acts. ("serious risk to security, national economic security, national public health or safety").

#### Specific Red Teaming Considerations

* Focus on Harm: Red teaming should evaluate scenarios where the AI model might facilitate harm to users or others, engage in illegal activities (including potential privacy/rights violations), or contribute to cybersecurity risks. These areas highlight specific examples of what might be considered 'harmful outputs' within the meaning of the Order.
* Cybersecurity: Given the Executive Order's significant emphasis on cybersecurity risks, red teaming must assess whether the AI model could facilitate cyberattacks or the discovery of software vulnerabilities ("At a minimum, the Secretary shall develop tools to evaluate AI capabilities to generate outputs that may represent nuclear, nonproliferation, biological, chemical, critical infrastructure, and energy-security threats or hazards.")
* CBRN Risks: Red teaming should explicitly examine whether the AI model can assist in the design, synthesis, acquisition, or use of chemical, biological, radiological, or nuclear (CBRN) weapons, a critical concern underscored by the Order.
* Evasion of Controls: Testing should include attempts to break alignment procedures or obfuscate harmful behavior with the goal of circumventing red team controls. This aligns with concerns about AI evading human control or oversight 
* Disinformation: The Order highlights the need for AI systems to be "robust and designed to resist efforts to mislead and manipulate by using deceptive or illegitimate tactics."  Also see reference to "misleading, inflammatory...or deceptive outputs,". 

We understand that specific red-teaming guidelines have not been set forth by NIST or the various agencies, so we focus on the broad concerns of the Act. We also believe that red-teaming reporting is only required for foundational dual-use models, and it is unclear if Aurora-m itself satisfies the definition. However, as an exercise we performed red-teaming with the Biden-Harris concerns in mind.
### Acknowledgement:
Training was conducted on the LUMI supercomputer, using compute resources generously provided by CSC - IT Center for Science, Finland. We thank them and all the participants of the MDEL efforts, which we will list in an updated version of this model card and our corresponding data card.

### Versions:
- This is version 0.1. 
- We will update this model page with more information and a link to our technical report.
- We will also create a dataset page for the training dataset which will include the names of the authors who contributed to the dataset.

### Disclaimer:
Nothing in this model card should be considered legal advice. Despite extensive red teaming and safety alignment efforts, our model may still pose potential risks, including but not limited to hallucination, toxic content, and bias issues. We strongly encourage researchers and practitioners to fully acknowledge these potential risks and meticulously assess and secure the model before incorporating it into their work. Users are responsible for adhering to and complying with their governance and regulations. The authors disclaim any accountability for potential damages or liability resulting from the use of the model.

**This is the first open source model that we know of that has been red-teamed based on the concerns of the Biden-Harris AI Executive Order, so there may be issues with our coverage of the Executive Action**

### Further Reading:


* https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/
* https://cset.georgetown.edu/publication/comment-on-nist-rfi-related-to-the-executive-order-concerning-artificial-intelligence-88-fr-88368/
* https://cset.georgetown.edu/article/what-does-ai-red-teaming-actually-mean/
* https://www.dwt.com/-/media/files/2023/12/20231208_item2_draft.pdf?rev=7601290707ab4feea140f9b7398d7844&hash=B6F06809847A5AE245E4EBA4F9C26083
  
### Citation
If you find our project useful, we hope you would kindly star our repo and cite our work as follows: 
```
@article{taishi2024aurorav01,
  author = {Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Dung Nguyen, Nam Pham, Tanmay Laud, Chien Vu, Terry Yue Zhuo, Ziyang Luo, Ben Bogin, Sonny Vu, Victor May, Huu Nguyen, Sampo Pyysalo},
  title = {aurora-m-v0.1-biden-harris-redteamed - The First Open Source Biden-Harris Executive Order Red-teamed Multilingual Model},
  year = 2024,
}
```
NOTE: We will update the above citation to include all authors as the model progresses.
{"license": "bigcode-openrail-m"}
transformers safetensors gpt_bigcode text-generation license:bigcode-openrail-m autotrain_compatible endpoints_compatible text-generation-inference region:us