Machine Learning Distributed: Ring-Reduce vs. All-Reduce

My full article in medium.com: https://roman-kazinnik.medium.com/machine-learning-distributed-ring-reduce-vs-all-reduce-cb8e97ade42e