Distributed Deep Learning with BigDL and PySpark

What I chose among BigDL, Horovod and others for Deep Learning on Spark.

As I said, this post was only written in order to help anybody get the hands-on this interesting package. Of course, this is not an exhaustive search. For example, the following tests may be performed:

• Run this example as a spark-submit instead of doing it interactively from the PySpark Shell;
• Use more complex networks, such as convnets on the same MNIST or try importing pre-trained BERT models and fine-tune them using BigDL (I don’t even know if there’s the possibility)

When I have some time, I will definitely explore the two bullets above.
Bye and thanks for reading!