Hi,

I am just curious. In your document, you specify that:

• PS: do gradient reduction
• Worker: apply gradient update

So does it mean that each worker should hold all the parameters ?