Phishing simulation vs social engineering assessment

Phishing simulation vs social engineering assessment: which to buy first

A phishing simulation measures how staff behave over time: how many open or click a deceptive message and how many report it. A social engineering assessment is a scoped test in which testers use phishing, phone pretexts or physical access attempts to reach a defined objective, such as obtaining a privileged credential or entering a restricted area. Choose the first purchase by the question you need answered. If the board or regulator is asking whether staff recognize and report lures, and you have no baseline, start with a recurring simulation program. If you need to know whether an attacker could actually reach a crown-jewel system through people, start with a scoped assessment, because only that produces an objective-based result.

The two are not interchangeable. A simulation tells you about behavior across a population but rarely shows whether a single lure would have succeeded against your controls, such as mail filtering, MFA prompts, help desk verification or callback procedures. An assessment shows control failure paths but samples only a few people. NIST SP 800-50 Rev. 1 treats simulations as a learning tool, with results guiding training and staff told that exercises happen at random, not as a means of punishing individuals. Confirm with your legal and HR teams how that principle applies before you run either activity. Ask any provider to state in writing whether exercises will be tracked by named individual and how results will be reported.

Phishing simulation vs social engineering assessment side by side

This table compares a recurring phishing simulation with a scoped social engineering assessment across purpose, scope, metrics, authorization and output. Read it to decide whether you need a measurement of staff behavior over time, a test of whether an attacker can reach a specific objective, or both.

Dimension	Phishing simulation	Social engineering assessment
Primary purpose	Measure and build staff recognition and reporting of deceptive email across the organization	Test whether people and processes can be manipulated into granting an attacker access to a defined objective
Primary question answered	How often do staff click, and how often do they report, a deceptive message?	Can a tester reach the agreed objective through people, and which control or procedure failed?
Pretext range	Usually email only, varied in difficulty; NIST SP 800-50 Rev. 1 notes the NIST Phish Scale can rate difficulty	Phishing plus phone pretexts, physical access attempts or multi-channel approaches, as agreed in scope
Core metrics	Click-through and report rates, per NIST SP 800-50 Rev. 1; trends across campaigns	Objective reached or not, techniques that worked, data or access obtained, and the path taken
Who runs it	Often run by the security awareness or training team, with a platform or vendor; confirm internal ownership	Authorized testers under a written scope; PCI SSC treats social engineering as a component of penetration testing
Legal review and authorization	NIST SP 800-50 Rev. 1 says legal should review planned exercises; confirm HR and works council requirements	Written rules of engagement and authorization for named people and channels; confirm third-party hosting limits such as Microsoft and AWS notice terms
Duration and effort	Recurring, lightweight campaigns; confirm effort with your provider	Fixed-scope engagement with preparation and reporting; confirm duration in the written scope
Recurrence	Regular, at random times, so staff cannot anticipate them	Periodic, tied to a risk decision, a change in attack surface or a control validation cycle
Main output	Trend data for training and awareness, plus targeted coaching content	Findings with root causes, control fixes and process changes, plus evidence for assessors

What each approach finds and what it misses

A phishing simulation and a social engineering assessment answer different questions, so each leaves a different gap. The cards below show what each one surfaces, what it cannot show, and the condition under which each gap matters for your program.

Click and report rates by group

A recurring phishing simulation sends a deceptive email and counts who clicks and who reports it, which NIST SP 800-50 Rev. 1 describes as the core measure of organization-wide exercises. Break results down by department, role and location, not only by company average. The finding matters most when one group consistently clicks more, since that points to a workflow or lure type to target with training. Confirm whether your provider reports report rates alongside click rates, because a low click rate with a low report rate still leaves staff unable to flag real attacks.

Lure difficulty by NIST Phish Scale

The NIST Phish Scale, referenced in NIST SP 800-50 Rev. 1, helps rate how hard a lure is to spot. Without a difficulty rating, a 20 percent click rate on an easy lure and on a hard lure can look identical, which misleads the training plan. Ask the provider to tag each template with its difficulty and to show results by tier. This matters when you want to compare campaigns over time, because a program that moves from easy to hard lures may show worse numbers while actually improving.

Gaps beyond email, phone and physical pretexts

A phishing simulation tests one channel. It does not show whether a stolen session token, an unmonitored SaaS login or a misconfigured mailbox rule could give an attacker access after a click. Map what a successful click would expose, such as single sign-on, file shares and privileged accounts, before you read the results as a measure of overall exposure. This gap matters most in organizations where email is protected well but identity or collaboration tools are not, since the simulation will look healthy while the access path remains open.

Chained access the test never follows

A social engineering assessment is scoped to reach an objective, so a tester may move from a help desk call to a password reset to a privileged account. A recurring phishing campaign stops at the click, so it cannot show whether that click leads anywhere. Ask for the attack path narrative in the report, including each step and the control that stopped or failed to stop it. This matters when your highest risk is privileged access reached through a sequence of small weaknesses rather than one bad email.

Help desk and verification weaknesses

A social engineering assessment often tests whether a help desk agent resets a password or changes an MFA device for a caller who cannot prove identity. NIST SP 800-115 lists posing as a help desk agent to request passwords among the validation techniques, which is why this scenario appears so often. Confirm in the scope which verification steps the testers may attempt and whether the help desk knows the exercise is running. This matters most where the help desk holds the ability to reset credentials for privileged users.

Physical entry and access path gaps

A social engineering assessment can include tailgating, badge cloning attempts or pretexts at reception, which a phishing simulation cannot reproduce. Results show whether staff challenge unfamiliar visitors and whether badge controls hold under pressure. Define in writing which physical locations are in scope and who holds the authorization letter that lets testers carry out entry attempts. This matters for facilities that host sensitive systems or regulated data, where a single unchallenged entry can defeat network controls.

Statistically meaningful trends across all staff

A social engineering assessment usually targets a sample of people selected for the objective, so it cannot tell you how the whole workforce would respond. Recurring phishing campaigns reach everyone over time and can show trends by group, which the assessment cannot. Use the assessment to find failure modes and the campaigns to measure prevalence. This matters when leadership asks how many staff would fall for an attack, since one assessment result is an anecdote and not a population rate.

Behavior change after the test ends

Neither approach proves that behavior persists once the exercise closes. Staff may remember a campaign and change how they act for a few weeks, then drift back. Plan a follow-up measurement with the same scenario type after training, and confirm how the provider tracks repeat clickers without singling them out, as NIST SP 800-50 Rev. 1 advises. This matters for programs that report to a board, because a result measured once says little about what staff do six months later.

How a social engineering assessment runs, step by step

The sequence turns a written scope into a tested set of human and process weaknesses, with each gate signed off by named people before the next one opens.

Document scope and rules of engagement
The client and the testing provider agree in writing on the objective, the in-scope channels (email, phone, in-person or a mix), the target groups, the systems and facilities in bounds, and the activities that are off limits. NIST SP 800-115 includes a rules of engagement template in Appendix B that teams use as the reference for documenting these terms. The document is done when the client's accountable executive and the provider's engagement lead have both signed it and each named target group is listed explicitly. Confirm with your legal team whether any channel requires a separate authorization.
Set the objective and the stop conditions
The team turns the business goal into a testable objective, such as obtaining a privileged credential, reaching a named system or gaining entry to a restricted area. The client supplies emergency contacts, the conditions that halt activity (for example, a real incident, a distressed employee or a safety concern), and the escalation path to reach those people at any hour. Sign-off comes from the executive sponsor and the security lead. The step is complete when the stop conditions are written down and every contact has confirmed they can be reached during the test window.
Gather intelligence within the agreed limits
Testers collect publicly available information about the organization, such as role titles, naming conventions, public contact routes and the tools staff use. They do not access non-public data unless the rules of engagement allow it. NIST SP 800-115 places this kind of review among its passive techniques, so the activity should pose minimal risk to systems. The output is a target profile that the client can review for sensitive content before any pretext is built.
Design pretexts and get them approved
Testers draft the scenarios they plan to use, such as a help desk agent asking for a password reset or a vendor requesting an invoice change. NIST lists posing as a help desk agent to request passwords among the target vulnerability validation techniques, which is why the pretext is the core artifact of this step. The client reviews each pretext for legal, HR and reputational risk, and the executive sponsor approves the list. The step is closed when every pretext has an approval record and none involves a prohibited action.
Execute across the agreed channels
Testers carry out the approved activities within the windows and channels defined in the scope. Each attempt is logged with time, channel, target, pretext used and response received, so that the team can reconstruct what happened. Where a stop condition is triggered, testers halt the affected activity and contact the escalation path. Provider and client both confirm the end of the execution window in writing, and the activity log is secured as sensitive evidence.
Confirm the objective was met or not
When the window closes, the provider states whether the objective was reached, partially reached or not reached, and records the path used. The client verifies the result independently where possible, for example by checking whether a credential was actually obtained or a door was actually opened. This verification stops overclaiming and gives the report a result that can be checked. Sign-off comes from the engagement lead and the client's security lead.
Debrief and report to the right audiences
The provider walks through the activity log with the client team and writes a report that separates the technical and process findings from the individual interactions. Results are shared in aggregate, and individuals are not named or singled out. Findings feed two places: training content for staff who responded, and control fixes such as callback verification, identity proofing at the help desk or badge procedures. The step is complete when each finding has an owner and the client has accepted the report.

Legal review, consent and third-party limits before any campaign

Before any phishing message is sent or any pretext call is placed, the legal team should review the plan. NIST SP 800-50 Rev. 1 says exactly this for organization-wide phishing exercises, and the reason is practical: a simulated lure that impersonates a bank, a payroll provider or an internal HR notice can create privacy, employment and contractual questions that no one in the security team can answer alone. Ask counsel to confirm the lure themes, the data captured (clicks, form submissions, credentials typed or not), how long that data is retained, and who inside the organization may see individual results. The same review should confirm that staff representatives or works councils are consulted where local employment law requires it.

Exercises should also be designed so they are not punitive and do not single out employees. NIST SP 800-50 Rev. 1 says staff should be told that exercises happen at random and that results guide learning. In practice, that means reporting by department or role rather than by name, agreed consequences that do not include disciplinary action for a click, and a clear route for staff to report a suspicious message they recognize as a test. A campaign that embarrasses people tends to hide the real signal, which is whether reporting works. Once the purpose is set, the awareness program can carry the follow-up, as covered in the sibling comparison with security awareness training.

Assessments that reach beyond email need more than a legal review. A social engineering assessment that uses phone pretexts or physical access attempts should run under a signed authorization letter that names the testing organization, the in-scope people and locations, the permitted techniques, the time window, the emergency contacts and what happens if a tester is challenged or detained. Written scope should say which outcomes count as success, what the testers may not do (for example, collecting personal data beyond what the objective needs) and how evidence is handled. Third parties add a separate constraint. Microsoft's cloud penetration testing rules of engagement prohibit phishing or social engineering of Microsoft employees, and AWS requires 2 weeks' notice for simulated phishing campaigns against AWS-hosted assets. These are examples of what you may hit, not a complete list. Confirm each provider's current terms, and any hosting or outsourcing contract that touches the targets, before planning.

How each approach feeds training or control fixes

A recurring phishing simulation produces population-level signals: which lures drew clicks, which teams reported suspicious messages, and how quickly reports reached the security queue. Awareness owners use these patterns to decide what to teach next. NIST SP 800-50 Rev. 1 frames these exercises as part of a learning program, so results should guide learning rather than serve as a scorecard for individuals. The exercises should be announced in policy as random, and the legal team should review each planned exercise before it runs.

Targeted refresher modules: Group results by lure theme, such as payroll, vendor invoices or password resets, and build short refreshers around the themes where staff struggled. Confirm the simulation platform can export results by lure and business unit.
Reporting-channel changes: If reports take a long path to the security team, the fix may be a clearer button, a shorter instructions page or a faster triage queue, not more training.
Non-punitive handling: Keep individual results out of performance reviews and limit access to named results. Document this rule in the program charter so HR and works councils can confirm it.
Difficulty tuning: Use the NIST Phish Scale to rate campaigns so that improvement is not confused with an easier lure set.
Feedback cadence: Set who receives monthly summaries and who owns the training backlog, usually the awareness lead with the security operations manager as reviewer.

Before relying on the output, confirm with your legal and HR teams how simulation data is stored, retained and shared.

What you receive from each approach

This table compares the records and reports each approach hands back, so you can judge whether a phishing simulation or a scoped social engineering assessment produces the evidence you need. Read each row as a question to ask the provider before signing, and request sample reports from both.

Output	Phishing simulation	Social engineering assessment
Planning record	Campaign design records covering templates, target groups, and timing	Rules of engagement, scope, objectives, and authorization letter naming permitted pretexts and channels
Core results	Per-group click and report results against the campaign design	Timeline of pretext attempts, with each channel tried and its outcome
Objective outcome	Not applicable, since the measure is user response rather than reaching a goal	Objectives reached or not reached, with the path taken to each
Difficulty rating	Difficulty ratings for each template, which NIST SP 800-50 Rev. 1 says can use the NIST Phish Scale	Narrative difficulty and defensive controls observed, rather than a standard scale
Evidence handling	Usually aggregate results with limited personal data; confirm retention and anonymization terms	Notes on how captured evidence, credentials, and personal data were stored, shared, and destroyed
Debrief and fixes	Learning follow-up plan for staff, plus recommendations for training and email controls	Debrief meeting and findings report with recommended fixes for process, technology, and training
Assurance status	Neither output is an audit report or certification; ask the provider to state this in writing	Neither output is an audit report or certification; ask the provider to state this in writing

What drives social engineering assessment cost and effort

Quotes for phishing simulations and social engineering assessments are built from a handful of scope decisions, not a standard package. The cards below separate the one-time setup work from the work that recurs, and list what to ask for before accepting a written scope.

Number of targets and channels

Cost follows the size of the population tested and the number of channels used, such as email, phone, SMS or in-person contact. A recurring awareness campaign to a fixed staff group is priced differently from an assessment that targets named roles, such as the help desk or finance approvers. Ask the provider to state the target count, the channels in scope and whether each channel is a separate workstream with its own reporting.

Phone and physical pretexts

Phone pretexts and on-site access attempts add the most effort, because they need tester time, scripted scenarios, safe-word and escalation rules, and a documented stop procedure. NIST SP 800-115 lists social engineering among target vulnerability validation techniques, so a phone or physical component is part of the same testing discipline, not a free add-on. Ask whether these channels are included, who approves each scenario, and how the tester proves an objective was reached without capturing unnecessary personal data.

Campaign frequency for simulations

A simulation program is a recurring service: each campaign needs a new lure, a send window, a mailbox setup and a results pull. Running four or more campaigns a year multiplies the design and reporting work, while a single scoped assessment is a one-time engagement. Ask for the campaign count, the cadence, and whether the quote covers template refreshes between runs or only the sends.

Legal review and authorization

Legal review is largely one-time work at the start, but it can recur when scope changes or when a provider outside your own organization is involved. NIST SP 800-50 Rev. 1 says the legal team should review planned phishing exercises before they run. Cloud providers add their own conditions: Microsoft's rules of engagement prohibit phishing or social engineering of its employees, and AWS asks for two weeks' notice before simulated phishing against AWS-hosted assets. Ask who drafts the authorization letter, who signs it and which third-party notices are required.

Lure design and custom content

Off-the-shelf lure templates cost less than lures written for your organization's vendors, internal processes and tone. Difficulty also matters: NIST SP 800-50 Rev. 1 notes the NIST Phish Scale can rate how hard a lure is to spot, and harder lures usually need more design and review time. Ask whether the quote includes custom lures, how many, who approves them, and whether the difficulty rating is recorded so results can be compared across campaigns.

Reporting depth and internal support

Detailed reporting, individual-level debrief sessions and metric trend reviews add analyst time, while aggregate click and report rates are cheaper to produce. Internal effort is often the largest hidden cost: a named sponsor, a contact in HR and legal, approvals for each send, and help desk coordination so that real incidents are not confused with the test. Ask what the provider delivers at each reporting tier, which metrics are excluded, and how many internal hours the scope assumes you will spend.

Questions buyers ask before hiring outside help

Can we run phishing simulations in-house instead of hiring outside help?

Yes, many teams run recurring awareness campaigns themselves, but the work is more than sending a test email. NIST SP 800-50 Rev. 1 describes organization-wide phishing exercises that measure click-through and report rates, and it expects a legal review before each exercise, staff told that exercises happen at random, and results used for learning rather than discipline. Someone must own template design, difficulty rating (the NIST Phish Scale can help), target lists, data handling, metrics and follow-up training. Ask whether your team has capacity for that cycle every month, and whether it can also run a scoped social engineering assessment, which needs a tester role, written authorization and reporting that goes beyond click counts.

What drives the cost of a social engineering assessment and how do we request a quote?

Cost follows the written scope, not a standard package. DeployOpen does not publish a price list, and most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. The main drivers are the number of channels in play (email, phone pretexts, physical access attempts), the number of people or sites in scope, the depth of intelligence gathering, the number of objectives, and how much reporting and debrief you require. Ask for a quote that lists each channel, the objective being tested, the target population, what is excluded, the legal and approval steps, the deliverables and the retest or follow-up options. Ongoing phishing campaigns and one-off assessments are priced differently, so request each one separately.

How long does a phishing campaign or social engineering assessment take to prepare and run?

Duration depends mostly on scoping, approvals and the channels involved, so there is no single timeline that fits every engagement. A recurring email campaign can be prepared and launched quickly once templates and approvals exist, while an assessment that includes phone pretexts or physical attempts needs more planning, legal review and coordination with the people who will be targeted. Lead times can also be set by third parties. Microsoft's cloud penetration testing rules of engagement prohibit phishing or social engineering of Microsoft employees, and AWS requires 2 weeks' notice for simulated phishing campaigns against AWS-hosted assets. Confirm any provider notice rules early, and ask DeployOpen to state the timeline in the written scope.

What does our team need to provide for a social engineering engagement?

The client provides approvals, named contacts, legal sign-off and clear stop conditions before any test begins. Typical items include a signed authorization letter that names the scope, the systems and people in scope, and the testing windows; an executive sponsor; a technical contact who can halt activity; a trusted contact for physical or phone tests who knows when staff may be tested; and an agreed process for handling captured credentials or personal data. Stop conditions should cover real incidents, complaints from staff, unexpected operational impact and any target that falls outside scope. Confirm which of these your HR, legal and works council processes require, since those reviews often set the critical path.

Can we run a social engineering assessment without a phishing simulation?

Yes. Social engineering is a separate test method, and a phishing simulation is only one channel within it. NIST SP 800-115 lists social engineering, such as posing as a help desk agent to request passwords, among target vulnerability validation techniques. PCI SSC's Penetration Testing Guidance defines social engineering as manipulating or deceiving people into divulging confidential or personal information and treats it as a component of penetration testing. An assessment built around a phone pretext or a physical access attempt may test whether staff verify identity before resetting access, which a click-rate campaign would not show. Choose the channel that matches the objective, and confirm with the provider how each channel is authorized and reported.

How do we avoid punitive outcomes from phishing tests and assessments?

Design the program so results guide learning and do not single out individuals. NIST SP 800-50 Rev. 1 says staff should be told that exercises happen at random, that results guide learning, and that exercises should not be punitive or single out employees. In practice, that means agreeing on reporting rules before launch, reporting results by group rather than by name, excluding individual click data from performance reviews, and pairing each campaign with short, immediate guidance for anyone who clicked. Ask the provider how it anonymizes results, how it handles captured data, and whether it will support a staff communication plan that your HR and works council teams approve.

Do phishing and social engineering results count as compliance evidence?

They can support a compliance file, but they do not replace an audit or certification on their own. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications, and independent auditors and certification bodies make those determinations. Metrics such as click and report rates show awareness trends. A scoped assessment shows whether a specific social engineering path reached an objective, and that output can inform control fixes such as call-back procedures or identity checks. Ask your auditor which control evidence it will accept, and confirm how the framework you follow treats awareness testing before you plan the reporting format.

How DeployOpen works on this

DeployOpen plans, implements and documents social engineering work inside the client's own infrastructure, and runs security testing as part of that scope. Phishing simulations and scoped social engineering assessments are agreed in writing before any message is sent or any call is made.

How we scope

The written scope names which pretexts, channels (email, phone, onsite) and target groups are in bounds.
Legal review of the planned exercise is part of scoping, with sign-off recorded before launch.
Engagements are fixed-scope, so deliverables and timelines are agreed in writing before work starts.
Scope also states what is excluded, such as named individuals, third-party assets and provider-hosted systems.

How we build

Campaign templates, pretexts and tracking are built and configured on the client's own platforms.
Metrics such as click and report rates are defined up front so results can be compared across runs.
Testers document each attempt, including the objective reached or blocked, for the findings record.
Pre-launch checks confirm that the testing team, mail infrastructure and escalation contacts are correct.

How we hand over

Deliverables include a report linking each result to a training action or a control fix.
Results are presented in a way that avoids singling out individual employees.
The client receives the templates, scope documents and metric definitions for reuse in future exercises.
Remediation items are assigned owners so that fixes can be verified in a later test cycle.

How we support

Ongoing support, upgrades and continuous testing are available as retainers.
A retainer can cover recurring awareness campaigns between formal social engineering assessments.
Support covers keeping the platform, templates and documentation current as the client's environment changes.
Clients should confirm retainer terms, response expectations and included activities in their written agreement.