Learn to Communicate - Communicate to Learn
Thursday, Oct. 18, at 14:30 in room DEI/G 318
Machine learning and communications are intrinsically connected. The fundamental problem of communications, as stated by Shannon, “is that of reproducing at one point either exactly or approximately a message selected at another point,” can be considered as a classification problem. With this connection in mind, I will show that we can tackle the fundamental joint source-channel coding problem using modern machine learning techniques. We will first introduce uncoded "analog” schemes for wireless image transmission, and show their surprising performance both through simulations and practical implementation. This result will then motivate leveraging unsupervised learning techniques for wireless image transmission. Surprisingly, the "deep joint source-channel encoder" we have designed behaves similarly to analog transmission, and not only improves upon state-of-the-art digital transmission techniques, but also achieves graceful degradation with channel signal-to-noise ratio, and performs exceptionally well over fading channels even without channel state information or pilot signals.
In the second part of the talk, we will study the impact of communication on the performance of distributed machine learning algorithms in the presence of straggling servers. We will introduce both coded and uncoded distributed stochastic gradient descent algorithms, and study the trade-off between their average computation time and the communication load, defined as the number of computations transmitted from the computing servers, assuming stochastic processing speeds.
Deniz Gunduz received his M.S. and Ph.D. degrees in electrical engineering from NYU Polytechnic School of Engineering (formerly Polytechnic University) in 2004 and 2007, respectively. After his PhD, he served as a postdoctoral research associate at Princeton University, and as a consulting assistant professor at Stanford University. He was a research associate at CTTC in Barcelona, Spain until September 2012, when he joined the Electrical and Electronic Engineering Department of Imperial College London, UK, where he is currently a Reader (Associate Professor) in information theory and communications, and leads the Information Processing and Communications Lab.
His research interests lie in the areas of information theory, machine learning, communications and privacy. Dr. Gunduz is an Editor of the IEEE Transactions on Green Communications and Networking, and served as an Editor of the IEEE Transactions on Communications (2013-2018). He is the recipient of the IEEE Communications Society - Communication Theory Technical Committee (CTTC) Early Achievement Award in 2017, a Starting Grant of the European Research Council (ERC) in 2016, IEEE Communications Society Best Young Researcher Award for the Europe, Middle East, and Africa Region in 2014, Best Paper Award at the 2016 IEEE Wireless Communications and Networking Conference (WCNC), and the Best Student Paper Awards at the 2018 IEEE Wireless Communications and Networking Conference (WCNC) and the 2007 IEEE International Symposium on Information Theory (ISIT). He served as the General Co-chair of the 2018 International ITG Workshop on Smart Antennas, 2016 IEEE Information Theory Workshop, and the 2012 IEEE European School of Information Theory.