Monitoring Metric "Transform Fn"
Hi- It’s me again.I want to get a better sense of the system latency for the speech model I have deployed.AWS providesInvocation Endpoint MetricslikeModelLatencybut that is more end-to-end. I am particularly interested in how much time is spent with the forward pass. I think I have two options:1- HuggingFace model logspreprocess,predict, andpostprocesstimesherebut there is a bug as below where predict time is not captured correctly.Screen Shot 2021-08-19 at 5.23.12 PM2616×226 70.5 KB2-Here, the metricsTransform Fnis added to the context using the API describedhere.My question is  where does theTransform Fngo? I have hard time finding it on CloudWatch. I will also submit a PR to fix the predict time logging.Thanks so much!Deniz
2021-08-20T13:47:04Z
Hey@dzorlu,thank you for finding the bug, please let me know if you not manage to open PR, then I would take care of it.Maybe@dan21ccan tell more about thetransform_fnand where is stored?context.metrics.add_time("Transform Fn", round((predict_end - predict_start) * 1000, 2))
2021-08-20T20:02:47Z
Thanks@philschmid.Hereis the PR.