Ichiro Hashimoto

Ichiro Hashimoto

PhD student, Statistics

Publications

Overfitting of Spectral Gradient Descent: How Matrix Geometry shapes Generalization and Implicit Bias

arXiv, 2026

We analyze SpecGD in overparameterized matrix classification with corrupted labels under collapsed and dispersed shortcut geometries. In the dispersed model, vanishing shortcut correlations generate a statistically significant signal through a second-order effect, whereas exact orthogonality eliminates the signal entirely; in the collapsed model, the limiting geometry is qualitatively different and can favor SpecGD over GD. To identify these limiting directions beyond static spectral max-margin theory, we analyze the exponentiated-gradient dynamics of the dual weights and establish directional convergence and generalization guarantees.

Download Paper

Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks

Accepted to ICLR, 2026

In this paper, we prove directional convergence of network parameters of fixed width leaky ReLU two-layer neural networks optimized by gradient descent with exponential loss, which was previously only known for gradient flow. By a careful analysis of the convergent direction, we establish sufficient conditions of benign overfitting and discover a new phase transition in the test error bound. All of these results hold beyond the nearly orthogonal data setting which was studied in prior works. As an application, we demonstrate that benign overfitting occurs with high probability in sub-Gaussian mixture models.

Download Paper

Universality of Benign Overfitting in Binary Linear Classification

To appear in Bernoulli, 2025

This work provides a comprehensive study of benign overfitting for linear maximum margin classifiers, discovers a phase transition for the noisy model which was previously unknown and provides some geometric intuition behind it. We further considerably relax the required covariate assumptions in both, the noisy and noiseless case. Our results demonstrate that benign overfitting of maximum margin classifiers holds in a much wider range of scenarios than was previously known.

Download Paper