How should Product and Engineering define success for a Feature?

A Feature can be built to specification and still leave the original problem untouched. Teams need a way to tell whether the intended benefit appeared, whether another effect offset it and whether further investment is justified. That starts during shaping, while the Feature boundary and instrumentation can still change.

Describe the change the Feature should cause

A success statement names who should experience a change, what they should be able to do and why that matters. It is narrower than a broad product goal but wider than an acceptance criterion. For example, “customers can complete an address change without calling support” describes a result; “the form is released” describes an output. Product should identify the intended customer and business outcome. Engineering should test whether the proposed mechanism and evidence are credible. An initial hypothesis may be enough when demand is uncertain. Record the uncertainty instead of promising a precise benefit without a baseline.

Use existing evidence and measurement methods

Current practice may use service analytics, research, support data, operational records and acceptance tests. Each answers a different question. Acceptance tests show whether agreed behaviour works; interviews explain how customers experience it; service data can show patterns at scale. GOV.UK guidance recommends deciding what to measure early and using performance data to improve services. Choose an outcome signal, one or two leading signals and a few guardrails such as failure rate or support demand. Define the population, data source and observation period. If reliable baseline data are absent, say so and establish measurement before claiming improvement.

Turn the measures into a joint learning plan

During shaping, Product and Engineering should ask which decision the evidence will inform: expand, adapt, investigate or stop. Product remains accountable for value and priority. Engineering owns the technical means of recording behaviour and the integrity of the delivered system. Analysts, design, support and operations may help interpret the evidence. A smaller Feature may test the proposition sooner if its effect remains observable. AI can summarise feedback or help explore segments, but only when source data are accessible and checked; a generated summary is not itself evidence of a customer outcome.

Review the result without false precision

Low volume, seasonality, parallel changes and selection effects can make attribution difficult. Separate what was observed from the team’s interpretation and record alternative explanations. A rising completion rate might conceal more calls from customers who never reached the form. A faster journey could increase processing errors. Set a sensible review window and an earlier trigger for harm. If results are inconclusive, decide whether more evidence is worth the time and cost. Preserve the original question so success is not redefined after the fact.

Example

In a hypothetical utilities service, customers repeatedly call to correct account details. Product proposes self-service editing and defines success as fewer avoidable calls without more billing errors. Engineering checks which edits can be safely completed and instruments successful changes and exceptions. The team records its current call and error baselines, releases a narrow change and agrees to review after a billing cycle. If calls fall but errors rise, Product and Engineering investigate the trade-off before extending the Feature.

FAQs

• Is adoption the same as success?

  No. Adoption shows that people used a Feature. The intended task, customer benefit and operational effect still need checking.

• What if the Feature has no measurable business result?

  Use proportionate evidence, including qualitative research or risk reduction, and state what cannot be measured reliably.

• Who chooses the metrics?

  Product owns the outcome question. Engineering, analysts and relevant specialists help select trustworthy measures and make the system observable.