How do you know whether Product Engineering is working?

A team can ship frequently without improving its product. It can also produce a positive customer result by exhausting people or accumulating technical risk. Measuring only one side creates an incomplete and sometimes dangerous account of Product Engineering.

The right question is not whether a team follows the prescribed ceremonies. It is whether its way of working helps it make better decisions, create valuable outcomes and retain the technical and human capacity to continue.

Start with the outcome the product exists to change

Define success in terms of a user or customer behaviour, service quality or problem that matters, and connect it to a legitimate organisational outcome. For example, a team might aim to help more eligible customers complete an application correctly, reducing avoidable contact and processing effort without increasing exclusion.

Choose a small set of measures that can show that change, establish a baseline where possible and document important guardrails. Quantitative data may reveal what is happening, while research, support conversations and operational evidence help explain why. Segment results so an overall improvement does not conceal harm to a particular group.

Do this before delivery. Adding measures after launch encourages teams to select whatever makes the release look successful. Where attribution is uncertain, be explicit about the other factors that may have influenced the result.

Balance value with delivery and technical health

Product outcomes are necessary but can take time to emerge. Delivery measures provide useful information about the system's ability to turn a decision into a safe change. Deployment frequency, change lead time, change failure rate and recovery time are established examples, but their interpretation depends on the service and workflow.

Technical evidence might include reliability, security findings, accessibility, maintainability, observability, dependency health and the effort required to make a change. The purpose is not to maximise every number independently. Faster delivery achieved through more failures is not an improvement; perfect stability achieved by never changing is not a product strategy.

Examine trends and relationships. A falling lead time alongside stable reliability and a better customer outcome tells a stronger story than any isolated metric. Discuss material exceptions rather than forcing every team into the same threshold.

Look for faster learning and healthier decisions

Product Engineering should reduce the time between an important assumption and useful evidence. Teams can observe how early Engineering and other disciplines join shaping, how often alternatives are considered, whether success is defined before building, and how quickly post-release evidence changes a decision.

Qualitative review is valuable here. Sample recent Features and trace the decision: Was the problem supported by evidence? Were feasibility and viability tested before commitment? Did the team know what it would measure? Did the result lead to continuation, adaptation or stopping? These questions expose learning quality without inventing a false precision score.

Team conditions also matter. People need clarity, access to customers and systems, psychological safety to surface uncertainty, and enough control to act on evidence. Regular team reflection and targeted surveys can reveal friction, but should lead to visible action rather than become another reporting burden.

Turn measurement into an improvement conversation

Review measures as a connected narrative: intended outcome, delivered changes, observed effects, delivery and technical health, what was learned and what happens next. Combine regular team review with less frequent leadership review so decisions occur at the right level.

Avoid using contextual measures to rank teams. Different products have different risks, architectures and demand. Once a measure becomes an individual target, people may optimise the number rather than the system it was meant to illuminate. Velocity, story points, utilisation and raw Feature counts are especially weak proxies for value.

Change the measures as the product and uncertainty change, while retaining enough continuity to see trends. AI may help detect patterns or summarise evidence, but it can also create confident explanations from incomplete data. Accountable people must validate data quality, investigate causes and decide what the evidence warrants.

Example

A team improving appointment booking initially reports releases and completed backlog items. It replaces this with a view of successful bookings, avoidable calls, abandonment by journey stage, lead time, failed changes, service incidents and two recurring team-friction questions.

After a release, bookings rise but calls do not fall. Research shows confirmation wording still causes uncertainty. The team changes the content and sees calls reduce, while delivery and reliability remain stable. The evidence demonstrates Product Engineering working not because output increased, but because the team detected an incomplete outcome and adapted safely.

FAQs

• What is the single best Product Engineering metric?

  There is no single best metric. Start with the outcome for the product and balance it with delivery, technical health and learning evidence. A small connected set is more useful than a universal composite score.

• Can DORA metrics measure Product Engineering?

  They can illuminate software delivery performance, but they do not show whether customers received value or whether the right problem was addressed. Use them as one part of a broader product and operating-system view.

• Should Product Engineering teams measure velocity?

  A team may use velocity cautiously for its own short-term planning when the underlying method is stable. It should not be treated as customer value, compared across teams or used as a productivity target.