Skip to content

How to distribute joblib over multiple nodes using ray backend

joblib provides a simple helper class to write parallel for loops using multiprocessing (More examples of what it can do check here).

joblib deoes not natively distributed work across multi-nodes , hence we give a guide how to set up joblib with ray backend with the idea of distributing over multiple nodes on JUWELS or any kind of clusters.

You can find all codes for this recipe on the gitlab.