Metrics. It’s seems as if today’s business world is ruled by them. Procurement tracks spend under management and savings. Manufacturing tracks defective parts per million and return on assets. Sales tracks number of prospects converted to active campaigns and then to closed sales. Metrics are essentially pervasive in just about every functional department in a manufacturer. I believe it all started with the efforts around ISO certifications and the spread of lean principles years ago. But where it all started is a moot point.
So how do metrics apply to engineering? There’s no doubt that the primary measure used in engineering is progress against the development schedule. Time to market has become such a dominant business differentiators that it has top of mind of c-suite executives and engineering’s priorities are set accordingly. But what about design completeness or a product’s innovation? Those aren’t nearly so easy to measure. One could do competitive analysis and market research but it’s not as explicitly defined. And in comparison to schedule metrics, I don’t think there’s any question which ones wins out.
But I believe there’s another problematic area. The following is a general list of the major activities in the design phase of the development cycle and some commentary on what can and cannot be measured.
- Project Kickoff: This is basically a singular event when engineering resources are assigned to a project to complete on schedule.
- Requirements, Constraints and Targets: Whether this is formal or informal, all design and engineering efforts must start with some requirements, constraints and targets against which the product is developed. Lots of deliverable completion dates and milestones could be measured like initial requirement authored, requirements validation started, requirements finalized and so on.
- Designing the Product: This is where the heavy lifting of invention, problem solving and creativity takes place. Engineers explore various design iterations and calculate their performance and characteristics trying to make the best design decisions possible. This set of activities is far more about gaining knowledge than completing a deliverable.
- Documenting the Product: This is where the documentation required to manufacture the product is created. It includes things like production drawings, 3D models, specifications and other hard deliverables.
- Product Testing, Verification and Validation: In these activities, the product design is tested to ensure it meets the requirements, constraints and targets initially defined. Progress against these activities can be measured easily as test result in a pass or fail state.
- Design Release: Like project kickoff, this is a singular event that marks the handoff of engineering deliverables to manufacturing as well as procurement to start order parts and producing the product.
So what’s my concern? It’s in how organizations are trying to instrument the effort to design the product with metrics. Almost all of the other activities have hard milestones or explicit deliverables to be completed. However, the activities around designing the product is about gaining the right knowledge to make the best design decisions. Outside of the ideation phase (prior to this design phase) where the initial product concept is created, the activities around designing the product is where all the innovation, invention, creativity and problem solving occurs. Further complicating the matters is that because the activities around designing the product are so deliverable oriented, specifically with 3D models, executives place pressure on engineering to rush to those activities as fast as possible. Which makes the scenarios described in Leslie Gordon’s post titled Pretty models must be backed up by math over at Machine Design all the more likely. And that’s leads to products that are represented well in 3D but really haven’t been engineered.
In summary, executive’s priority on metrics as a means to measure operational effectiveness works for a wide variety of functional departments, but fails and can even create dangerous situations when applied to activities around designing the product.
So, it’s that time again. What do you think? Have you seen metrics applied in you organization or to engineering work? Are there plausible and effective ways to measure activities around designing the product? Sound off. I’d really like to hear from you on this one.
Take care. Talk soon. And thanks for reading.
