The original code is available at the  llm-foundry repository.
Read more about it in the release blogpost
Usage tips

Learn more about some techniques behind training of the model in this section of llm-foundry repository
If you want to use the advanced version of the model (triton kernels, direct flash attention integration), you can still use the original model implementation by adding trust_remote_code=True when calling from_pretrained.

Resources