Decentralized Learning via Random Walks on Graphs

The talk will be on Zoom: link here.

Abstract: We study the decentralized optimization problem via random walk for the empirical risk minimization problem. More specifically, we assume that the data are distributed over a network, and a random walk carries the global model, travels over the network, and trains the global model using the local data stored at local nodes. We focus on speeding up the training via the design of the transition probability of the random walk. We implement the importance sampling idea in centralized optimization, identify the entrapment phenomenon that slows down training convergence under specific configurations, and propose a novel algorithm, random walk with random jumps, to overcome the entrapment problem.

Short Bio: Zonghong Liu is a fourth-year Ph.D. candidate advised by Prof. Salim El Rouayheb in the Rutgers ECE Department. His research aims at implementing probability tools to design distributed optimization algorithms. His research interests also include information theory and statistics. He got his Master’s Degree in Statistics at Rutgers University in Oct 2019 and a B.S. in Chemical Physics from the University of Science and Technology of China in June 2016.