Workers compute gradients on data and send updates to servers, then fetch newer parameters. Synchronous or asynchronous coordination changes consistency, fault tolerance, and communication behavior.
A parameter server is a distributed architecture in which dedicated processes store and update shared model parameters.
Workers compute gradients on data and send updates to servers, then fetch newer parameters. Synchronous or asynchronous coordination changes consistency, fault tolerance, and communication behavior.