Making critical decisions is rarely easy for executives. Often they rely on gut feelings and hard experience. Today, however, the basis for decision-making is changing. Executives are increasingly reliant on metrics and business analytics to measure organizational performance and make decisions.

For many organizations, this trend has driven operations to align more closely with company goals. But how has it affected engineering? To answer that question, insight into traditional engineering operations is needed.

The Traditional Autonomy of Engineering’s Black Box Operations

Engineering has always been highly technical. It requires expertise in high-end mathematics and physics to predict product form, fit and function. Because product design is so technical, it has been difficult for executives and non-engineering stakeholders to gain insight into the status of engineering projects. As a result, few outside of engineering departments have understood what engineering was accomplishing. From an operational perspective, engineering has therefore been seen as a black box. Requirements go in. Something highly technical happens next. Deliverables for product designs come out later.

The Executive’s Initiative to Drive Operational Transparency

Operational metrics: it seems as if they rule today’s business world. Procurement tracks spend and savings. Manufacturing tracks defective parts per million and return on assets. Sales track the number of prospects converted to closed sales. Service organizations track the percentage of first-time issue resolution. Metrics are present in every non-engineering organization.

The trend started with lean initiatives and ISO certifications. But regardless of its genesis, the driving philosophy behind it is simple: what isn’t measured can’t be improved. Today’s executives rely on operational performance metrics and business analytics to make decisions. This trend isn’t optional. It’s a CEO mandate for visibility from the shop floor to the top floor.

Today’s Metrics for Engineering are Deliverable-Based

During design release, engineers hand off deliverables such as drawings, models and specifications to those that provide the hardware and software to make the product, namely procurement, their suppliers and internal manufacturing. Because conforming to the development schedule is a key component of launching or delivering products on time, the natural metric to measure is the status of these deliverables as they progress towards completion. But just because it is the natural metric doesn’t mean it is the right metric.

The Problem with Deliverable-Based Metrics for Engineering

The problem with deliverable-based metrics for engineering lies in an immutable fact about design. Engineers hand off deliverables at design release. It’s not how they design products.

The implications of those two sentences are extremely important. Fundamentally, engineering is about iterating, exploring and solving problems. It’s about varying a design trait and understanding the impact on product performance. It’s about fully exploring the design space. It’s about trying something, seeing it fail, and then trying something else that succeeds. In aggregate, those successes and failures lead to great design.

In contrast, creating deliverables adds the least value to the design of the product. It doesn’t help make sound decisions. It actually doesn’t even communicate design intent. It merely defines what should be procured or manufactured. Engineers design great products by making the right decisions, not by finishing the drawing on schedule.

Ultimately, today’s deliverable-based metrics for engineering lead to a serious issue. When progress on deliverables lags, executives reprioritize deliverable work over design work. This leads to products with drawings, models, and specifications that are complete but haven’t been well engineered. In summary, deliverable-based metrics for development projects are misleading and a risk to the company.

The Challenge of Engineering Operational Transparency

  • Executives want insight into the progress of development projects. The progress of many development projects is tightly wound up in technical details that those outside engineering do not understand. Engineering leaders must find a way to communicate that progress with some balance between not enough and too many technical details.
  • The use of deliverable-based metrics is misleading. Well-engineered projects may lag in the completion of deliverables, while poorly engineered projects may be ahead of schedule with their deliverables. Using such metrics in oversight processes can result in the wrong decisions. Engineering leaders must derive new design-based metrics that measure the progress of well-engineered products.