Managing The Design Factory Outline

A very readable text-file version of this outline

Title: Managing the Design Factory: A Product Developer’s Toolkit Author: Don Reinertsen Length: 288 pages Published: 1997 ISBN-10: 0684839911 ISBN-13: 978-0684839912

Outline by Anthony Panozzo ( copyright 2009

Introduction JIT acknowledged as most significant change since assembly line Only possible by questioning assumptions and reinventing manufacturing from scratch While product development is unpredictable, there are principles we can follow Danger of “best practices”: “the practice can get disconnected from its intent and its context and may acquire a ritual significance that is unrelated to its original purpose.” “The most effective way to deal with this problem is to try to retain a degree of thoughtfulness.” Use things in this book as tools, not as rules or rituals There are thinking tools and action tools, both important to product developers Reinertsen says the thinking tools are more important – why, not how


1 - Into the Design Factory Analogizes product development to a bank account that pays no interest and loses money We should try to view design as manufacturing because: - we have thought about manufacturing a lot - manufacturing works with mostly known quantities, repeatable - we have gotten useful tools that we can apply to design Must begin with asking: “What is the objective of the Design Factory?” Objective: to make a profit (see The Goal) Manufacturing : cooking food :: design : creating recipes “Our designs are not important because they are new, or because they are beautiful, or because they are innovative; they are only important if they make money. Our designs will only make money if we create recipes to turn material, labor, and overhead into valuable functionality better than our competitors do. The goodness of our recipe is important, but it is perishable.” “Thus, we need good recipes, and we need them before our competition gets them.” Discusses design-in-process inventory (“During the time that the recipe is incomplete we are holding an investment that is earning no money.”) DIP is worse that WIP because WIP typically has faster cycle time and the holding costs of design are higher due to perishability DIP typically not managed because of GAAP bias Design has an exponentially expensive cost of change, whereas manufacturing has linear CoC Design also has information coming in throughout process, especially late in process, when it happens to be most expensive to change Manufacturing is a repetitive, ideally low-variance process. Design is a unique, probably variant process. “…while we make no money by reinventing the wheel, we can make money by manufacturing the same wheel many times in a row.” “…risk, and the variability associated with it, are inherent and desirable characteristics of design processes.” Using manufacturing tools to reduce variability will probably backfire in design world. Some variance is needed. Final difference is expanding work – Herbert Simon explains this in The Sciences of the Artificial Engineering work expands to fill the time allotted to it, not because engineers are bad, but because engineers stop when they run out of time

Summary - similarities and differences between manufacturing and design factory Similarities: goal is to make money, use input resources to produce output, have inventories Differences: DF: CoC increases dramatically over time, have less information at beginning of process, variability higher and important, expandable tasks (“rubber finish line”)


2 - Making Profits Not Products Many companies try to change their development process, but cannot explain why they want to change it Imitation and obedience might work sometimes, but are not the most effective strategy “The great risk in behaving this way is that you are constantly being blown hither and yon by each shift in the managerial winds.” Economic model can be viewed as black box with inputs that we can tweak to try to make money Can systematically change inputs and observe output changes to improve our profitability Models give us ways of thinking about process, useful for teams and project managers to get more concrete context and make better decisions

Project models Done from the perspective of the development team Quantify what overruns of expense, cost, and performance, and for a one-month change in schedule will do to profitability Better to have simple model that everyone understands and agrees with than to have opaque model – inputs are everything Predict what a one-month change of schedule by analyzing market and potential competition to project delay scenarios Benefits of using project model: make better decisions using data, faster decision making, higher buy-in on team decisions

Application models Done from the perspective of the customer “When we design an application model we are trying to understand how specific product attributes affect the customer economics.” Hence, the application model allows development team to focus on efforts that are important to the customer Process: identify application economic drivers (AEDs), convert to dollars, model the overall economics, and translate to business rules Initial identification usually done internally by talking to sales, marketing, application engineering, and field service From this, we can make an initial attempt to quantify the dollar impact each of the AEDs have Then, we talk to the customer directly, changing our dollar estimates and even adding AEDs based on the feedback we get Different customers weight different factors differently, so stratify cost estimates Look at one-year, five-year, and lifetime costs Then convert this into something the team can use, similar to how project models show costs of percentage change Two caveats: not all customers of a product will have the same economic models, and customers do not necessarily understand their economics

Models of Process Economics Done from the perspective of the development company Similar to project models, but applied to enterprise level If you have a large number of products, break them into groups by cost of delay and revenue Basically showing process effectiveness and showing where the biggest gains can be realized

Practical tips for models Use cross-functional team to generate models for buy-in Financial people also important to include Avoid unnecessary complexity Input data is the key error component, and sales data is the most likely to be off Consider what the best format for using the model will be Prefer using an imperfect model and iterating rather than trying to come up with a perfect model

3 - Entering the Land of Queues Discussing queueing theory terms and concepts “In the stochastic world of queueing theory, we will experience delays even when we have excess capacity in the system.” While manufacturing factories wouldn’t dream of loading past 85 to 90%, but development is often past 100% “We will always need excess capacity to optimize the economics of a development process.” “Queues are not inherently bad; they are good or bad depending on their relationship to the critical path of a program.” Discusses cumulative flow diagrams and how they show inventory in queues Queueing theory proves most instinctive project management wrong

Levers to improve queues - increase capacity - more people (full-time, overtime, part-time, contractors) - higher productivity per person - tools - increase support - reduce non-value add activities - training - cross-train workers - recommends making people more general, since specialists cause queues - interesting implications for respect and compensation - manage demand - control the number of products in the system - control feature content per project - create a relief value to relieve demand (drop requirements, etc.) - reuse design content from existing projects - reduce variability - measure variation - reuse design solutions - process standardization - smaller batch sizes - using control systems - remove queue from critical path - reservation systems - capacity planning to prevent “predictable queues” - managing batch size - queue monitoring

Should not intermingle large batch transfers with small ones Little’s Law - average number of units in process is equal to the arrival rate times the average time a customer spends in the store “Process queues give us instant information about the health of our process.

4 - It’s All About Information “The optimum design process must generate valuable information cost-effectively.” –> 1) efficient 2) high-value Information theory Economic benefit is essentially ROI – value / cost Efficiency is cost per unit of information Value is the monetary value per unit of information Information is inversely proportionate to the probability of an event occurring - interesting, this goes along with my thoughts on model generation and how disproving your model gives you information The information content of a test is maximum when the probability of failure is fifty percent AJP: this could apply to TDD, other testing, and architectural tests (“does this feature work in our architecture?”) Information is better earlier – use small batch sizes to avoid delays in information Model for this is economic order quantity problem Small batch sizes control the size of the total error population of the product, enabling better completion date prediction, easier to fix defects since they were added recently Iterations also generate information early Instead of trying to do things perfectly the first time, put something out there and improve on it Discussing decision trees and how information fits into this (reminds me of real options) Should seek to generate information about possible failures by focusing on high-risk areas and where it has the highest economic significance Design failures only useful when they generate new information Must communicate failures to get the full value from them Expert knowledge comes from failures, because they are lower probability and contain more information Create checklists to prevent failures that will not add information because they are preventable Information theory balances top-down and bottom-up by focusing on where the presumed risks are

5 - Just Add Feedback Cannot predict the behavior of a system simply by understanding the behavior of its components This “implies that we will only be able to verify system-level behavior by testing at the system level, not…individual components.” Discusses how poor role definitions lead to too many meetings because no one knows who is authorized to make a decision - AJP: hmm, where have I seen this before… “…there is no communication tool that is as powerful as correct partitioning of the system.” “We discover that the optimum final structure for the system may not be the optimum structure for building a system.” Closed-loop solutions “can tolerate changes in the external environment well and they continue to perform well even when some of their elements are not performing well.” However, closed-loop solutions have more difficulty in troubleshooting, stability (chaos), accuracy Open-loop solutions focus on making process accurate as possible, closed-loop solutions focus on the feedback loop Discusses vector arithmetic and how it leads to law of large numbers with averaging The variability of a system will approach the variability of the most variable component


6 - Choose the Right Organization Organizations are systems, and as such some of the properties of systems apply to them Should focus on the interfaces rather than the functional blocks (relationships over people) Coupling, or level of feedback, will have a strong impact on the organization “It is only when external and internal coupling are properly balanced that we get an organization that responds well to changes in its environment.” Discusses common organization structures functional (vertical): + efficiency through economies of scale, - slow, poor communication internally and externally autonomous (horizontal): requires versatile resources, + speed, - teams disconnected from organization, inefficiency, lack of expertise hybrid: + performance, low cost. Divide responsibilities in detail, which is key for making decisions quickly First, reduce need to communicate by using well-partitioned architecture, well-defined responsibilities on the team, and dedicated team members Next, focus on improving communication Use colocation to improve communication, and realize that it doesn’t need to be all or nothing Discussion on communication methods is a bit outdated

7 - Design the Design Process Most companies create their design process through evolution, but this is too slow The design process outlines the method of using budget, manpower, and technology to create recipes The design process defines which activities should take place, the sequence, and who should do them Again, design is a one-time, unique process Analogizes design process to English language Standardization of low-level elements enables flexibility for high-level elements Cannot have flexibility at every level–this would result in chaos Modular processes use set inputs and desired outputs, and give choices for projects to make on the implementation of the processes Similar to design principles of keeping your interfaces set so that you can change underlying implementation Pattern languages focus on principles that should be present in good design processes Work arrival subprocesses - large batch of work causes problems in estimation, motivation, accuracy of requirements These times can be improved internally by changing the way that we work Must make trade-off in engineering between fixed cost of introducing product and holding cost of design-in-process inventory Uses game theory to show that mixing project sizes is the safest bet Coupling technology and product development has risks, but can result in efficiency as opposed to speed Control queues by having small batch sizes Need to optimize subprocesses by economics

Key design principles: degree of overlap, managing information profiles, decentralizing control and feedback, and location of batch queues Discusses how focusing on different optimizations leads to different processes Reiterates that large batch sizes should not precede small batch sizes

Discusses evolving the process through “lessons learned” Should be done as soon after the lessons are learned They should also be balanced and try to include independent people Make these sessions solution-oriented Note observations, and pick a small subset for subsequent action Review your review sessions to continually improve this process

8 - Product Architecture: The Invisible Design Key architectural principles - modularity, segregating variability, and interface management

Modularity: good for development expense, cycle time bad for gross margin (higher unit costs or performance weaknesses) These benefits are only realized if the system is partitioned correctly Module interfaces can be a source of poor performance (look at operating systems) There is a strong interaction between organization, architecture, and development process

Consider partitioning the system to concentrate key economic risks

While components get a lot of attention, interfaces determine performance, riskiness, and ultimately, value of a system Managing interfaces: - Be sure all interfaces are defined. Ask: “what is the worst thing subsystem A could do to subsystem B?” - Ensure that each interface has a single point of control - Have a mechanism for freezing or stabilizing the interface - locking too early locks in system-level trade-offs - establish adequate design margins in the interfaces

Discusses how many organization-wide improvements should be done with team whose sole purpose is the improvement Changing architecture to pursue specific economic objectives: development expense, unit cost, performance, and schedule To reduce development expense: - maximize reuse of existing designs (components generally must be designed with this in mind) - displace customization outside the system boundary (consider macro languages, plugins) - obtain nonstrategic subsystems from vendors To reduce unit costs: - tightly couple subsystems within the design - specify subsystems to minimize manufacturing costs - seek scale economies for key components To improve performance: - build the design around the key performance-limiting element - focus on system-level optimization - tightly couple systems within the design To improve speed of development: - fundamental shift: use a concurrent approach - reuse subsystems - stabilize architectural interfaces early - place design tasks with high schedule variability off the critical path

People in all departments should make decisions regarding architecture, they see different constraints

9 - Get the Product Specification Right Starts with strategy - how are you using your strengths to provide value? Select the right customer Market segmentation means not serving some customers so you serve the most profitable ones Both engineering and marketing must be involved to correctly assess cost and profitability Result of market segmentation should be a definition of target customer and a rough definition of customer needs that will be satisfied Understand the customer. Ask: What do customers want? Why do they want it? How do customers make their decisions? High ROI customer understanding tools: customer interviews, observation of customers, and focus groups Read the book for the pros and cons and tips for using these, I mostly skimmed it

A specification is a critical input for the design process A specification is a control device for the project Specifications should be brief – control the important things, not everything Controlling everything dilutes the product by missing important things and taking up design time by making things more complex Consider a catalog page for high-value requirements Can use qualitative or quantitative means of determining what is important to customers

Have a product mission: Why should a customer buy this product instead of the competitor’s? Need to have a good process to create a good specification Should reduce delays in program and have development team involvement Recommends fluid specifications that eventually are frozen for development purposes On the other hand, realize that specifications should be in motion if the target market is in motion Most companies treat specifications as open system – but this makes is prone to error Seeing the specification as flawed to begin with is more constructive This will cause us to work continuously through the product development lifecycle Use customer-driven process instead of specification-driven process “Its advantage is that customer-driven processes will almost never generate a product that nobody will buy.” In order to do this, we need specific mechanisms to get quick, accurate feedback Marketing people are very busy in a customer-driven process - AJP: in a lean startup, everyone is very busy in this process Methods include: using a customer partner, customer advisory boards or user groups, or proxy customers on the design team “The specification should never be the only tool telling the design team what is important to the customer.

Again, discusses how to change the specification process to pursue specific economic objectives: To reduce development expense: - thorough requirements that include downstream groups, like manufacturing - specification change process - will be strict about increasing work, liberal about reducing work To reduce unit costs: - eliminate all unnecessary elegance from the design - the cost goal should be broken down between subsystems and owned by the entire design team - also demands careful market segmentation - specification often subjugated to cost realizations by team For high performance: - intense contact with specific customers - models of application economics and performance benchmarks used for making tough trade-offs For development speed: - start the design before we have a complete specification - this is especially useful when requirements are changing rapidly anyway

“Too often companies do a poor job at these front-end activities in the development process. They move rapidly to create a list of features without spending much time selecting and understanding their customer. Such actions only create the illusion of rapid progress; companies that shortcut these steps pay the price later.”

10 - Use the Right Tools “Technology impacts development processes in three ways: it accelerates information flows, it improves productivity, and it reduces delays.” Information flow examples - pharmaceutical companies using databases to search for trends while trials going instead of at the end, using build servers Productivity comes from automating easily automatable tasks that are boring to humans Could see no increase or even drop in productivity due to steep learning curve or trying to do more than we would have manually Reducing delays comes from reducing process queues (see chapter 3)

Implementation Principles Technology changes process - if you are not prepared for your process to change, you are not getting the most out of your technology change Pay attention to economics - never use something for the sake of using something new Of course, this depends on having a business-level cost of delay approved by the finance department…

Technologies: automating design computations and decisions, prototyping and testing, improving communication, simplifying information storage and retrieval Reinertsen notes that one company improved technology for bidding and significantly reduced this cost and improved speed Dates himself, “It is likely that E-mail will become as ubiquitous as the telephone for most development teams.” Design reuse can be improved with better technology

11 - Measure the Right Things “…the purpose of controlling a process must be to influence economic outcomes.” Can only control what most leverages our economic outcomes “This simple concept is perhaps the most frequently violated principle in control system design for product development.” Discusses project control triangle: scope of work, resources, schedule Since product development has inherent variability, we must consciously choose which side of the triangle will bear the project’s variability Most companies unconsciously choose to fix scope and resources, at the expense of making schedule variable Discusses how setting other things as variable might be better for software, medical devices Understand how to decentralize control correctly by stratifying events by frequency and where in the organization decisions will be made - basically, empower people by letting them make decisions at the right levels; provide decision rules, rather than performance standards Choose metrics that are well-suited to control objectives: simple, relevant, leading indicators Leading indicators can be seen at the process queues Other leading indicators include turnover rates, manning level, specification freeze percentage

Project-level controls are similar to previous sections, focused on specific economic objectives: To reduce development expense: limit delegation, use expense budgets, and do piecewise funding of projects One key of budgets is to see things in aggregate so we don’t waste time when buckets overrun To reduce unit costs: limited delegation of authority (not very effective), using cost budgets, and design reviews (do them early) For high performance: - limit authority - performance budgets - design reviews - decision rules - design defect tracking systems - customer-driven process For development speed: - control schedule changes - time budgets (suggests internal and external to prevent Parkinson’s law) - schedule review meetings - use priority systems - monitor queues

Business-level controls measure the functioning of the overall development process Expense-focused controls measure the relative efficiency of our development process (ROI) Cost-focused controls can be internal (cost-performance ratio of the product) or externals (equivalent competitors) Performance can be measured in terms of market share gains, sales growth, or price premiums in the marketplace - using technical measures can be misleading – might be missing entire markets with wrong focus Speed can be measured internally by the fuzzy front-end, the development process, and the volume ramp of the product Cycle time can also be measured externally by comparing to competitors “It does not matter if we run faster than the competition as long as we win every race. We can do this by starting running before they do.”

“The key message of this chapter is that allocation of scarce time and attention to project control must be done with a keen understanding of the underlying economic leverage points.” “Don’t spend resources controlling things that don’t influence the overall economics of the business.”

12 - Manage Uncertainty and Risk Discusses making decision trees to weigh risks Not trying to eliminate risk – that would be impractical given the nature of product development Factors we can influence include: decrease the magnitude of the downside, reduce the probability of the downside, increase the magnitude of the upside Market requirement <–> Product Specification <–> Product design [Market risk] [Technical risk] Most product failures are caused by market risk: differing needs, trouble articulating needs, needs may change with time Ironically, companies spend millions of dollars testing to technical specifications, and little to see if the specifications meet market requirements Risk management strategy: obtain information to place intelligent bets, determine if the bets have paid off, backup plan in case we have lost

Can manage market risk by: - using a substitute product - simulating the risky attribute - making the design flexible - moving fast

AJP - interesting that he says that market risk is higher than technical risk and is overlooked, but then devotes more space to technical risk

Manage technical risk Have a buffer between projects so key participants are engaged at the beginning AJP - how does this fight the S-curve often displayed on projects or prevent overloading of resources? Manpower always is a management issue – when a product is a priority, we will see few problems For most projects the failure modes relate to integration, which means they lie in the interfaces rather than in the modules Advocates a meeting similar to planning poker where team estimates risk in different modules Risks should only be taken if they are economically justifiable If taking a risk is economically justifiable, we should resolve the uncertainty and create a backup plan Can resolve uncertainty with a model or simulation, or with a prototype These are good because you see things as they will be, and they can have an energizing effect on the organization Reduce system integration risk by: increasing reuse of interfaces, provide extra design margin in interfaces, increase amount of subsystem testing, force system integration earlier in project

Backup plans protect against failure on a key economic parameter As such, we seek to offset damage in one area by compensating with one of the other three parameters

World-class testing - tests have a very high information content Many companies misunderstand testing because they fail to distinguish between design testing and manufacturing testing

Goes into the four economic things again Cheap testing - rarely the most important objective in design testing, but if it is: - eliminate duplicate testing - test at the most efficient subsystem level - automate many testing processes - avoid overtesting the product Low unit cost impact - again, rarely the most important objective in design testing, but if it is: - eliminate product features that have been added solely to make the product easier to test (works in theory, not in practice) - use testing as a tool to fine-tune product costs Maximizing performance - try to maximize test coverage (need a keen understanding of the actual application of the product) - enhance the validity of the tests (testing should generate the same type of failures that are showing up in the field) - when we find competitor products that fail our internal tests, it is likely that our tests are not valid Fast testing - increase parallelism - use reliability prediction - decrease testing batch sizes - monitor and manage queues in our test processes Continuously improve your testing process - set explicit objectives, people to accomplish them, a plan, resources, and a schedule

PART IV: NEXT STEPS 13 - Now What Do I Do? Directed mostly at senior management Do your math - understand the economics of your development projects and processes Use decision rules - Use hard facts to make decisions. Apply intuition to model inputs and sanity-checking outputs, not to extrapolating outputs. Pay attention to capacity utilization - grasp the economics of queues Pay attention to batch size - lower it Respect variability - uncertainty is what creates information, and information creates value of product development. Create processes that function in a variable world. Think clearly about risk - see it as a process of placing rational bets on the future. Avoid punishing failure, instead communicate failure Think systems - don’t optimize the parts, optimize the whole by setting the right context Respect the people - sometimes “problem is system, not people” is misleading. Instead, “success is equally dependent on creating a workable system and populating this system with excellent people.” - “If we truly believe that such an attitude toward people is correct and profitable, we will devote our scarcest resource, management time, to developing our people.” Design the process thoughtfully Pay attention to architecture - obtain cross-functional reviews before product architectures are adopted Deeply understand the customer - understand why the customer wants the product, not just what they want Eliminate useless controls Get to the front lines Avoid slogans

Outline by Anthony Panozzo ( copyright 2009