 Hey, so I'm back to talk through kind of some high-level concepts about this feature that we've referenced called automations. And as Hamel did a great job of describing, it's really this glue that lets you connect, you know, different actions that you're performing in weights and biases and hook them up with downstream events that you want to trigger automatically. And that's what this whole topic of automations is really about. That is about setting up these event-action pairs. So to clarify, an automation is just an event-action pair, where an event is something that takes place, like a specific change that takes place in the model registry, like a new model version is added, and that is hooked up to an action. So this is a downstream action that is triggered in response to the event happening. So it's nothing fancy. Again, it's this event action pair. And you can see here in the diagram that we have two types of events that you can set up. So either, you know, you want to add a new version. So the automation will listen to any changes in, you know, your registered model that we looked at earlier. And you can set one up that says, hey, I want you to kick off a model evaluation pipeline when any new model is added. So the event there is linking a new version to the registry. The second one we see here is adding a new alias. And what that means is you're adding a model version with a specific alias. So this might be something that you set up, for example, to kick off an action that should only happen when a model is in a specific stage of the model lifecycle. So quantized, adding the alias quantized might be a great event to use to trigger a downstream process of model quantization. Another one might be prod. So if you're adding the alias prod, that might kick off a downstream action that deploys the model to production. And on the right hand side, we see the action. So again, this is the downstream action that's going to be triggered in a response. The automation is going to listen for the event to happen, and then this downstream action will happen. In weights and biases, we currently have two options for these actions. The first one is a webhook. This might be a term that you guys are familiar with from software, and Hamill is going to spend a bunch of time kind of walking through that and some really good examples of how you set a web hook up locally, and then moving it on cloud, and kind of really to replicate what do we mean when weights and biases is going to kick off a web hook. But it allows you to kick off a downstream process in an external application. So maybe kicking off a model evaluation pipeline in your workflow orchestrator. It's essentially just a post request that allows you to hit an external endpoint in another application or server and lets you connect events to weights and biases to, you know, any other tooling you have. For example, tooling that might be managing your, you know, model CICD processes. The second type of action we see here is a launch job. So a launch job is a containerized and reusable and also configurable job that can be executed in various different compute environments. And we typically encourage this, you know, when you're working with any actions that, for example, like evaluation or model retraining, anything with heavy compute requirements. First the webhook is typically more helpful for handing off models to external systems like CI CD tools, like GitHub actions or any of the workflow orchestrators you might be using. And we'll spend some more time talking about webhooks. That will be kind of the main focus of some of the demos that Hema will be giving. But Launch is also, it's another product inside kind of the weights and biases family. Cool. So I put this diagram together. You know, I talked about kind of the anatomy of an automation, that it's this event-action pair. And you might be trying to understand, like, OK, like, why is this relevant to my CICD workflow? And really, like, how are automations useful to connect weights and biases to your CICD workflow? So I put down a few examples here. And you might, you know, on the left-hand side, the farthest column on the left might, you know, recognize a bunch of workflows that you're trying to automate. And, you know, the key thing to point out here is that you can use an automation to listen to changes in the model registry or to your artifacts to automate these processes. to your artifacts to automate these processes. So I think a great one that I have down here is model evaluation. So you're trying to make sure that any time a new model is a candidate for staging and it is linked to the model registry, it has to go through a system of evaluation tests. So the automation will listen to changes and say, oh, a new model was added. I'm going to go ahead and kick off this model evaluation process, potentially create an automated report, which pulls in all of the evaluation results so that they're ready to be kind of analyzed by the team. You can also apply automations and we'll spend less time on this, but for dataset artifacts, this is another great one where you set up an automation to listen to any new dataset versions that are added. And, you know, whenever a new version is added, you kick off this model retraining pipeline. This is a use case we see a lot with customers who are getting in a bunch of new data. So periodically, once a month, their data is ingested into Weights and Biases, and they want this automation to listen for a new data set so that the retraining process can be done automatically. Rather than someone saying, hey, we have a new batch of annotated data. Can you go in and retrain the model? And the last thing I'll note here, specifically for webhooks, that again, we'll spend some more time and demo this live, but you might be seeing, hey, talking to external systems, what about any authentication or secrets or access tokens that I might need? And is it safe to store those in weights and biases? So the answer is yes. You know, we have a secret store that safely allows you to store any of these secrets or access tokens that you're going to be required or you're the receiving server is going to be required to authenticate a webhook from weights and biases. And beyond kind of, you know of our cloud deployments for anyone and any customers that are on on-prem deployments, we offer integrations with secret managers for all three major cloud providers.
Introduction to Automations
Starting with Automations, we need to understand what an automation means in this context. In simple terms automation means setting up an event-action pair. An event is a specific change that takes place in the model registry (for example a new model is added to the model registry) and this triggers a response action. Weights & Biases supports 2 types of actions: launch jobs and webhooks. In the next lessons we will go deeper into each type.

You can also connect your automations to your existing CICD pipelines.  

Further reading:
Automations for Model Registry docs: https://docs.wandb.ai/guides/model_registry/automation?utm_source=courses&utm_medium=courses&utm_campaign=emm#create-a-webhook-automation

If you are interested in learning more about CICD check out Hamel's previous CICD for ML (GitOps) course: https://www.wandb.courses/courses/ci-cd-for-machine-learning