SLowcalSGD for non-convex optimization

In this project We evaluate the SLowcalSGD algorithm on the non-convex MNIST classification task using a simple CNN, comparing it against LocalSGD and MinibatchSGD. We further extend SLowcalSGD by adding a STORM-style double momentum mechanism (μ²) and test it on both convex and non-convex problems. Our results show that SLowcalSGD consistently outperforms MinibatchSGD and LocalSGD. Unfortunately, FL-μ² didn’t show any benefit on SLowcalSGD.