Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems
We study Incremental Gradient Descent in the small epoch regime and show that it exhibits severe slowdown especially in the presence of nonconvex components.
Jun 4, 2025
