Abstract
Influence functions offer a robust framework for assessing the impact of each
training data sample on model predictions, serving as a prominent tool in
data-centric learning. Despite their widespread use in various tasks, the
strong convexity assumption on the model and the computational cost associated
with calculating the inverse of the Hessian matrix pose constraints,
particularly when analyzing large deep models. This paper focuses on a
classical data-centric scenario--trimming detrimental samples--and addresses
both challenges within a unified framework. Specifically, we establish an
equivalence transformation between identifying detrimental training samples via
influence functions and outlier gradient detection. This transformation not
only presents a straightforward and Hessian-free formulation but also provides
profound insights into the role of the gradient in sample impact. Moreover, it
relaxes the convexity assumption of influence functions, extending their
applicability to non-convex deep models. Through systematic empirical
evaluations, we first validate the correctness of our proposed outlier gradient
analysis on synthetic datasets and then demonstrate its effectiveness in
detecting mislabeled samples in vision models, selecting data samples for
improving performance of transformer models for natural language processing,
and identifying influential samples for fine-tuned Large Language Models.