How often should AI capability be reassessed?

A professional may demonstrate sound AI capability in a realistic task today. That evidence does not remain equally relevant if their role changes, the organisation permits a new use or the tool begins working with different sources.

Repeating a full assessment every few months would create unnecessary effort. Waiting indefinitely risks leaving people with capability evidence that no longer matches their work.

A proportionate approach combines a baseline review cadence with triggers based on change, consequence and observed practice.

Capability evidence has a context and a shelf life

An assessment shows what someone could do under particular conditions. It reflects the task, tool, information, rules, expected review and professional responsibility at that time.

Some capabilities transfer across change. Framing a problem, checking evidence and recognising uncertainty remain useful even when an interface changes. Other decisions may change materially. A new retrieval feature can alter source traceability. A move from internal drafting to customer communication can increase consequence and require stronger oversight.

Capability does not expire on one predictable date. Its relevance declines when the current work no longer resembles the conditions under which evidence was gathered. Reassessment should focus on that gap.

Calendar reviews provide discipline but not relevance on their own

A scheduled annual or periodic review can prevent capability maintenance from disappearing among operational priorities. It gives programme owners a moment to check roles, learning evidence and emerging needs.

The date alone does not determine what someone needs. A professional whose task and controls remain stable may need only a brief check. Another person may require support immediately after a role change, incident or material policy update, even if the next calendar review is months away.

Fixed repetition can also become a completion exercise. People may retake familiar content without practising the changed decision. The organisation gains a renewed record but little evidence about current capability.

Use the calendar as a safety net, then define events that trigger earlier or more focused review.

Use triggers to choose the right reassessment

Useful triggers include:

• Role change: the person gains new decisions, data access or accountability.
• Task change: AI moves into a different workflow or higher-consequence use.
• Tool change: capabilities, sources, settings or limitations change materially.
• Policy change: permitted information, review or approval requirements are updated.
• Incident or near miss: evidence suggests misunderstanding, over-reliance or weak control.
• Performance pattern: reviewed work shows recurring omissions, poor checking or inappropriate task selection.
• Long gap in use: confidence or recall may no longer match the task.

The response does not have to be a full assessment. A policy change may need a short knowledge check. A new source-grounding feature may justify a focused scenario. A new role with different consequences may require broader role-based practice and assessment.

Distinguish changes in the system from gaps in the person. If an approved tool behaves unpredictably, retraining the user may not solve the underlying problem. The issue may require technical investigation, revised controls or withdrawal of the use.

Keep reassessment proportionate and developmental

Match depth to novelty, consequence and existing evidence. Reuse valid evidence where the behaviour and conditions have not changed. Concentrate assessment on new decisions and known gaps.

Explain the purpose and evidence to the learner. Capability review should guide development, not become a hidden performance label. Apply privacy, employment and fairness requirements to any workplace evidence. Offer reasonable support and accessible formats.

Results should lead somewhere. A small gap may need a worked example, peer discussion or focused practice. A significant gap in a high-consequence task may require supervised use, stronger review or a temporary pause until capability is demonstrated.

Keep a concise record of the trigger, changed capability, evidence and response. This helps programme owners see patterns and decide whether learning materials or organisational guidance also need updating.

Reassessment is part of continuous capability maintenance. The aim is to keep professional practice aligned with current work, not to make people repeatedly prove skills that remain well evidenced.

Example

A customer-operations team was assessed on summarising approved internal material. Six months later, the organisation introduces a tool with new retrieval features and permits a broader document set.

The learning owner maps the changed decisions. The original evidence still supports task framing and general output review, but not source traceability with the new feature. Learners complete a focused scenario containing conflicting documents and explain which source supports each claim.

Reassessment concentrates on the new capability and risk instead of repeating the full programme.

FAQs

• Should AI capability be reassessed every year?

  An annual review can provide a useful baseline where no more specific requirement applies, but it should not be treated as a universal rule. Role, task, tool, policy, consequence and performance changes determine whether review is needed sooner or in greater depth.

• Does every tool update require reassessment?

  No. A minor interface change may need simple communication. Reassessment is more relevant when an update changes available information, output behaviour, user decisions, limitations, controls or the consequences of use.

• What if performance evidence shows a capability gap?

  Diagnose the gap before assigning training. Provide targeted explanation, practice, coaching or supervised use, then use a proportionate re-check. If the problem comes from the tool or workflow, address that system issue as well.