cfdfedbd2e
* #496 Add pipeline module to parent pom ✨ * #496: Add main application class and test for pipeline * #496: Checkstyle format and add log messages on pipeline stages 🎨 * #496: Fill readme sections of pipeline ✨ * #496: Javadocs and checkstyle formatting 🎨 * #496: Follow PMD checks and add more explanation as block comment on App.java * #496: Apply requested PR changes by iluwatar 🎨 * #970: Replace log4j usage on commander pattern to Slf4j API 🎨 * #970: Replace log4j usage on dao pattern to Slf4j API 🎨 * #970: Replace log4j usage on data mapper pattern to Slf4j API 🎨 * #970: Remove log4j dependency on data transfer object pom 🔥 * #970: Replace log4j usage on module pattern to Slf4j API 🎨 * #970: Replace log4j usage on serverless pattern to Slf4j API 🎨 This also removes the aws log4j dependency * #970: Remove unnecessary gitignore line for log4j.xml 🔥 * #970: Remove remaining remnants of log4j 🔥 * #970: Replace System.out logging with appropriate logging methods 🎨 * #970: Replace System.out method references to Logger::info 🎨
layout, title, folder, permalink, categories, tags
| layout | title | folder | permalink | categories | tags | ||
|---|---|---|---|---|---|---|---|
| pattern | Master-Worker | master-worker-pattern | /patterns/master-worker-pattern/ | Centralised Parallel Processing |
|
Also known as
Master-slave or Map-reduce
Intent
Used for centralised parallel processing.
Applicability
This pattern can be used when data can be divided into multiple parts, all of which need to go through the same computation to give a result, which need to be aggregated to get the final result.
Explanation
In this pattern, parallel processing is performed using a system consisting of a master and some number of workers, where a master divides the work among the workers, gets the result back from them and assimilates all the results to give final result. The only communication is between the master and the worker - none of the workers communicate among one another and the user only communicates with the master to get the required job done. The master has to maintain a record of how the divided data has been distributed, how many workers have finished their work and returned a result, and the results themselves to be able to aggregate the data correctly.