text
More open research updates 🧵
More open research updates 🧵
new_line
new_line
text
Performance estimation is currently the best way to quantify the impact of data drift on model performance. 💡
Performance estimation is currently the best way to quantify the impact of data drift on model performance. 💡
new_line
new_line
text
I've been benchmarking performance estimation methods (CBPE and M-CBPE) against data drift signals.
I've been benchmarking performance estimation methods (CBPE and M-CBPE) against data drift signals.
new_line
new_line
text
I'm using drift results as features for many regression algorithms, and then I'm taking those to estimate the model's performance. Finally, I'm measuring the Mean Absolute Error (MAE) between the regression models' predictions and actual performance.
I'm using drift results as features for many regression algorithms, and then I'm taking those to estimate the model's performance. Finally, I'm measuring the Mean Absolute Error (MAE) between the regression models' predictions and actual performance.
new_line
new_line
text
So far, for all my experiments, performance estimation methods do better than drift signals. 👨‍🔬
So far, for all my experiments, performance estimation methods do better than drift signals. 👨‍🔬
new_line
new_line
text
Bear in mind that these are some early results, I'm running the flow on more datasets as we speak.
Bear in mind that these are some early results, I'm running the flow on more datasets as we speak.
new_line
new_line
text
Hopefully, by next week, I will have more results to share 👀
Hopefully, by next week, I will have more results to share 👀
More open research updates 🧵

Performance estimation is currently the best way to quantify the impact of data drift on model performance. 💡

I've been benchmarking performance estimation methods (CBPE and M-CBPE) against data drift signals.

I'm using drift results as features for many regression algorithms, and then I'm taking those to estimate the model's performance. Finally, I'm measuring the Mean Absolute Error (MAE) between the regression models' predictions and actual performance.

So far, for all my experiments, performance estimation methods do better than drift signals. 👨‍🔬

Bear in mind that these are some early results, I'm running the flow on more datasets as we speak.

Hopefully, by next week, I will have more results to share 👀
❤️
afrideva