Defining objectives¶
An engagement without clear objectives produces activity rather than learning. The team does things, findings are documented, a report is produced, and three months later nobody can say what changed. The objectives are what determine whether the engagement was worth running.
PSL’s three-domain framing is useful here. Most engagement objectives address only the rational layer: what will be tested, what will be measured, what constitutes a pass. Rational objectives are necessary but not sufficient. An engagement can find exactly what it set out to find, produce a technically accurate report, and still result in nothing changing, because the emotional layer was not addressed (nobody feels safe naming the real finding) or the political layer was not addressed (nobody has the authority to act on the systemic ones). Objectives that only address the rational layer will produce rational findings. Whether those findings translate into organisational change depends on conditions the objectives did not account for.
SEM adds a second dimension: the most durable objectives test model assumptions rather than validate procedures. A procedure-validation objective asks whether the control works as specified. A model-testing objective asks whether the organisation’s understanding of its own capability is accurate. These are different questions. Procedure validation produces a finding that can be closed when the procedure is fixed. Model testing produces a finding that updates the organisation’s picture of itself, which is harder to close and more consequential.
Framing useful objectives¶
Objectives are more useful when grounded in what the engagement is actually for, which varies. A purple team exercise tests whether defences detect what they are assumed to detect. A risk workshop tests whether the organisation’s model of its own risk landscape is accurate. An audit engagement tests whether stated controls match operational reality. A tabletop tests whether the team behaves as the plan assumes they will under realistic pressure. The objectives for each of these are different, and the success criteria follow from them.
Whatever the engagement type, objectives are more useful when they name the assumption being tested rather than just the activity being conducted. “Run a phishing simulation” is an activity. “Find out whether staff reporting rates reflect genuine awareness or learned performance for the simulation period” is an objective. The second version specifies what learning the engagement is supposed to produce, which makes it possible to evaluate whether it succeeded.
Success criteria¶
Defining what success looks like before the engagement begins, and including criteria at all three layers, gives the debrief something concrete to work with: what the engagement found (rational), whether findings were reported honestly (emotional), and whether the organisation acted on them (political).
The debrief is where the non-rational criteria become visible, but only if the objectives were designed to surface them.