The inference stage metrics are not displayed in SageMaker studio
I have stored the value of the inference stage of the pipeline in json. Can you help me to display it in the metrics of the trials and experiments? The metrics for the training is visible though.  I have also tried with property_files was no success.Inference3584×994 326 KB
2022-06-14T07:33:28Z
Hello@Sanjita,could you please elaborate more on what you want to do and are currently doing?To track metrics with sagemaker-experiments you have to definemetric_definitionsin theHuggingFaceestimator, see example:notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub
2022-06-14T07:52:26Z
Hello@philschmid,I am running the Sagemaker pipeline for 4 steps [step_preprocess, step_train,step_inference,step_metrics] and I want to compare the evaluation results among different trials. I am able to see themetric_definitionsadded for the training stage but I am not able to add the metrics in inference stage for the step_inference,step_metrics. I would like to see the metrics for the test data in the last 2 steps.Training output3522×1220 484 KB
2022-06-14T08:09:28Z
I am not sure if this is possible through the UI already. But you can use the SDK as shown in theexampleto a get a pandas data frame per experiment and then compare and visualizefrom sagemaker import TrainingJobAnalytics

# Captured metrics can be accessed as a Pandas dataframe
df = TrainingJobAnalytics(training_job_name=huggingface_estimator.latest_training_job.name).dataframe()
df.head(10)