The dinosaurs in the tar pit, or why the software is so hard to produce
Provides a summary of Fred Brooks' foundational book on software engineering, to pinpoint what are the methodological difficulties that the software systems encountered in the past.
Overall book summary
Claim: Preservation of the conceptual integrity of the product is critical. Large programming projects suffer management problems different in kind from small ones, due to division of labor (viii). The book works towards the way how conceptual unity can be achieved.
Supports/opposes who: the author is in conversation with a variety of his contemporaries, software engineering managers (IBM, MIT, Bell Labs, Computational Laboratory of Siberian Division, USSR).
Method: To answer Tom Watson's question why programming is hard to manage, compares the different management experiences in hardware and software development.
Why important: To managers, programmers and managers of programmers, helps them achieve higher productivity when producing software.
Relevance to my research: explores the methodological difficulties in software engineering.
Paradoxically, the book presents a negation to applying methods of industrial production management to software practice, but does not offer the new solution (which would be scrum, that one emerged a decade later). In other words, the critique of industrial methods is developed, while still viewing the software as a branch of this sphere, which has its own specificity, but the requirement for agile or other way of incremental delivery had still not been recognized. As it says in Wikipedia: "During the 1990s, a number of lightweight software development methods evolved in reaction to the prevailing heavyweight methods that critics described as overly regulated, planned, and micro-managed. These included: rapid application development (RAD), from 1991;[14][15] the unified process (UP) and dynamic systems development method (DSDM), both from 1994; Scrum, from 1995; Crystal Clear and extreme programming (XP), both from 1996; and feature-driven development, from 1997. Although these all originated before the publication of the Agile Manifesto, they are now collectively referred to as agile software development methods"
Summary of essays 1-3
1. The Tar Pit
Opposes: general public's belief that enthusiasts building software in a garage can surpass the efforts of large teams.
Claim: The difficulty of managing software production is...
Method: Narrative. Looks at key definitions: program, programming product, programming system, programming systems product. Compares joys (making useful things, learning) of programming to its woes (maldesign, poor implementation, incomplete delivery, bugs, product rapid obsolescence).
Notes
“the program construct .. is real in the sense that it moves and works, producing visible outputs separate from the construct itself. It prints results, draws pictures, produces sounds, moves arms” (7)
2. The Mythical Man-Month
Opposes: Optimism about tractability of software production process.
Claim: Lack of calendar time for delivery of software projects can be tackled via improvement of managerial techniques (estimation)
Method: A matrix of five reasons of delivery failures, then explanation for each in more detail.
Notes
Reasons for late deliveries in softEn:
Techniques of estimating are based on the assumption that everything will go well.
They confuse effort with progress (man-month as a dangerous myth, because only works for perfectly partitionable tasks, such as cotton-picking or reaping wheat (16))
Software managers are uncertain of those estimates.
Schedule progress is poorly monitored.
Adding manpower does not help to solve late deliveries - but effective management would (Brook's Law: "Adding manpower to a late software project makes it later" (25))
3. The Surgical Team
Opposes: A view that small sharp team is more effective than a large team of medium-skilled engineers. Also opposes the "conventional team"
Supports: Harlan Mills and centralized workflow organisation.
Claim: the problem with the small team concept is that it's too slow for really big systems (31).
Method: Comparative analysis of the two models, the "conventional one", where several features of the product are developed simultaneousy and the centralized one supported by Brooks, where "one does the cutting and the others give him every support that will enhance his effectiveness and productivity". While acknowledging that the latter approach supported by Brooks was a step ahead of the other model, both of them lacked the flexibility required for building a more effective workflow.
Notes
It is easy to notice that the model described here has an abundance of roles that have nothing to do with neither production nor decisions, which could be explained by the absence of powerful enough documentation, issue tracking or project management tools. For example, the "program clerk" is now replaced by Jira and Confluence, the "toolsmith" is outsourced to IT department, and the "language lawyer" is now embedded into back-end developer role. The lack of computational resources also means that front-end development is completely absent, since the visual side of software and other functionality it covers today was not there yet.
Setting aside the tendency of using "he" when detailing the roles in the model, it presents interest as a showcase of which methodological approaches to software engineering existed prior to agile. I would argue that scrum is improving this model at least in one major way - the whole of decision making is not any longer concentrated in one role. In Brook's model, "the surgeon", has the final say on all things software product. This role is quite mixed and complex and includes responsibilities for not only for designing the product, but also for coding, testing and documentation. The "copilot" is a product owner limited to only making suggestions and in any other way being secondary to the "surgeon", with the rest of the team hardly having any weight in decision making at all. Additionaly, the "administrator" role is overloaded with elements of different other roles, handling both communication between the team leader and the team together with office logistics. "The system, - Brooks says - is the product of one mind", in other words, it doesn't account - or accounts only externally - for the client and the user. This is crucial for understanding in which ways the view on software production has changed just a decade later.
My problem with this system is twofold. Firstly, the client is not represented in any way at all - perhaps because at the time of writing the concept of external agency was not fully formed so the company's product was fully ingrained into the work that the team does. In other words, everyone on the team was so familiar with the product that the product owner is not required. I would still suggest such close familiarity is not achievable, since it is clear that the product is quite complex. Secondly, as mentioned previously, the team is not involved in decision making on the practical level. This impedes the team's performance because it's harder for them to feed back on solutions which are seen as something external - and also leaves those people who are carrying out the tasks out of having their say in them, even though they are more qualified to make decisions.
This model leads Brooks to make a further claim, an importance of the divide between architecture and implementation, which to him means making decisions and creating the product. Would that suggest creation of a new working class though? The following essays promise to give further insights.
References
Brooks, Frederick P. The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley Publishing Company, 1975.