University of Pittsburgh

Detecting model performance deterioration using distribution divergence measures

Graduate Student
Date: 
Friday, December 4, 2020 - 1:00pm - 1:30pm

We study the correlation between the distribution divergence and model performance deterioration. The hypothesis is that as datasets deviate from the original distribution, the performance of the original model further deteriorates. We experiment with several divergence measures and multiple synthetic and real-world datasets to test this hypothesis. I will present the preliminary results of our study and discuss the future work including the development of a divergence criterion based on which to determine the significance of the divergence and to predict the model's degradation. Such criterion is of special importance in unsupervised problems where the test labels are unknown and the model's performance cannot be measured directly. Moreover, I will discuss a similar ongoing work in which we aim to detect performance deterioration in unsupervised domain adaptation methods. 

Copyright 2009–2020 | Send feedback about this site