Conditional Access: move from report-only to enforced without locking out users

Report-only is a test bench, not protection

A Conditional Access policy in report-only mode is evaluated during sign-in, but its grant and session controls are not enforced. It reveals which users, apps, devices and locations would be affected without blocking access yet. Its value therefore depends on disciplined analysis and a defined exit date.

Moving directly from observation to an All users, All resources scope can lock out administrators, break a legacy client or expose an unseen service dependency. A safe method combines emergency access, inventory, logs, What If, real tests and deployment rings.

• Microsoft — Analyze Conditional Access policy impact ↗
• Related guide — Entra emergency access accounts →

1. Establish prerequisites before changing policy state

An emergency exclusion must not become a daily bypass. These accounts should use phishing-resistant authentication, remain unused in normal operations and alert on every activity.

• Confirm licensing: Conditional Access requires Microsoft Entra ID P1; user-risk and sign-in-risk policies require P2.
• Maintain at least two cloud-only emergency accounts excluded from every Conditional Access policy, monitored and tested.
• Ensure pilot users registered the required methods and expected devices can become compliant.
• Keep an administrator able to reverse the change from a secure workstation without using a daily account.
• Define an owner, rationale, window, success criteria, stop condition and rollback procedure.

• Microsoft — Plan Conditional Access ↗
• Microsoft — Manage emergency access accounts ↗

2. Inventory and back up policies

Before the change, export every policy and record its groups, roles, resources, exclusions, conditions, controls and state. Look for overlap: several policies can apply to one sign-in and their requirements accumulate. An exclusion in one policy does not cancel another policy's requirement.

Use a convention such as CA01-AllApps-AllUsers-RequireMFA and keep an external register with the owner, change ticket and review date. The Conditional Access object has no governance owner field.

Connect-MgGraph -Scopes 'Policy.Read.All','Directory.Read.All'
$stamp = Get-Date -Format 'yyyyMMdd-HHmmss'
$folder = Join-Path $env:TEMP "CA-export-$stamp"
New-Item -ItemType Directory -Path $folder | Out-Null

$policies = Get-MgIdentityConditionalAccessPolicy -All
$policies | Select-Object Id,DisplayName,State,CreatedDateTime,ModifiedDateTime |
  Export-Csv (Join-Path $folder 'policies.csv') -NoTypeInformation -Encoding utf8
$policies | ConvertTo-Json -Depth 30 |
  Set-Content (Join-Path $folder 'policies.json') -Encoding utf8

$policies | Group-Object State | Select-Object Name,Count

• Microsoft Graph — conditionalAccessPolicy resource ↗
• Related guide — Microsoft Graph PowerShell →

3. Interpret report-only results correctly

Report-only does not trigger an MFA prompt or terms-of-use acceptance. User action required therefore means the user must act after enforcement, not that access will necessarily fail. Conversely, a compliant-device requirement can repeatedly prompt for a device certificate on macOS, iOS or Android even in report-only; Microsoft recommends excluding those platforms from that test if the prompt is disruptive.

In Entra, open Monitoring & health > Sign-in logs, select an event, then review Conditional Access and Report-only. Inspect the client app, actual resource, client type, device, location, authentication method and correlation ID.

• Microsoft — Troubleshoot Conditional Access sign-ins ↗

4. Measure trends, not a handful of events

The Conditional Access Insights and reporting workbook compares enabled and report-only policies across users, apps and time. It requires Entra ID P1, a Log Analytics workspace and sign-in log streaming. Use it to find a missed service or affected cohort, then inspect individual events for root cause.

The query below expands each policy result. Replace the name and analyze categories instead of applying a universal threshold. Data only exists after sign-in logs start flowing to the workspace.

let PolicyName = "CA01-AllApps-AllUsers-RequireMFA";
SigninLogs
| where TimeGenerated > ago(14d)
| mv-expand Policy = ConditionalAccessPolicies
| extend PolicyNameObserved = tostring(Policy.displayName),
         PolicyResult = tostring(Policy.result)
| where PolicyNameObserved == PolicyName
| summarize SignIns=count(), Users=dcount(UserPrincipalName)
    by PolicyResult, AppDisplayName, ClientAppUsed
| order by SignIns desc

• Microsoft — Insights and reporting workbook ↗

5. Add What If and real scenarios

What If simulates policies for an identity, resource, platform, IP address and other signals. Test at least an employee, administrator, guest, every platform, a remote site, an untrusted network and major applications. The tool clarifies scope; it does not replace an actual sign-in in a properly configured test environment.

Teams can request tokens for Exchange, SharePoint and other resources. A policy aimed at a dependency can break a feature even when the user thinks they are only using Teams. Audience reporting and end-to-end tests are essential.

• Microsoft — What If tool ↗

6. Resolve dependencies before enforcement

Workload identities use separate Conditional Access policies and licensing. Do not assume a user policy protects service principals. Inventory service accounts, connectors, scheduled jobs, shared devices and device-code flows separately.

• Microsoft — Conditional Access for workload identities ↗
• Related guide — Secure admin workstations →

7. Keep exclusions temporary and accountable

A broad exclusion quickly turns an effective policy into a theoretical control. Prefer a temporary static group per issue, with an owner, rationale, expiry and alerts on membership changes. Fix the cause, then remove the exception.

Named locations are a signal, not proof of identity. Excluding the entire office network from MFA lets a compromised device on that network inherit the same trust. Use such an exception only when the residual risk is accepted and documented.

8. Deploy in rings with exit criteria

Microsoft recommends at least one week in report-only for each policy, but the window must cover your actual cycle: leave, monthly jobs, travel and seldom-used apps. Change one policy or a small coherent set at a time so a failure remains diagnosable.

Conditional Access does not provide classic A/B testing between two enforced variants. Use ordered pilot groups and retain a report-only copy of a proposed change when comparing it with an existing enabled policy.

9. Controlled enforcement procedure

The Graph state for report-only is enabledForReportingButNotEnforced; enforcement is enabled. The example is intentionally read-only to prevent accidental activation. State changes should remain approved operations, ideally protected by Protected Actions and performed with Conditional Access Administrator rather than Global Administrator.

• Freeze the analyzed configuration and attach the JSON export to the change.
• Revalidate exclusion and sign-in for both emergency accounts.
• Confirm pilot exit criteria and support contacts.
• Move the target policy from Report-only to On in a monitored window.
• Immediately run positive and negative test-plan scenarios.
• Monitor failures, tickets, applications and affected cohorts during the wave.
• Expand only when evidence matches expectations.

$policyName = 'CA01-AllApps-AllUsers-RequireMFA'
$policy = Get-MgIdentityConditionalAccessPolicy -All |
  Where-Object DisplayName -eq $policyName

if (-not $policy) { throw "Policy not found: $policyName" }
if (@($policy).Count -ne 1) { throw "Policy name is not unique" }

$policy | Select-Object Id,DisplayName,State,CreatedDateTime,ModifiedDateTime
if ($policy.State -ne 'enabledForReportingButNotEnforced') {
  throw "Expected report-only state; no change should proceed"
}

10. Prepare a precise rollback

If a wave causes a critical lockout, use an emergency account from the designated workstation and return only the offending policy to Report-only or Disabled. A temporary exclusion for a trusted group can resolve a narrow case, but remove it quickly. Do not disable all Conditional Access to repair one application.

Capture time, correlation ID, user, application, resource and error code. AADSTS53000 through 53004 can point to compliance, join, approved app, CA blocking or risk. After stabilization, fix the dependency, rerun the pilot and document the decision.

• Microsoft — Roll back Conditional Access ↗

Decision checklist before enforcement

The goal is not zero report-only events. It is to understand every material category. A good policy is demanding for an attacker, predictable for a legitimate user and reversible for IT.

• Two working, excluded, monitored and tested emergency accounts.
• JSON export and policy-overlap inventory retained.
• Representative report-only period completed.
• Success, Failure, User action required and Not applied explained.
• What If and real tests completed for critical scenarios.
• Dependencies, guests, devices and automation validated.
• Minimal exclusions with owner and expiry.
• Pilot enforced without critical incident and support ready.
• Rollback tested with an accountable operator available.
• Post-enforcement monitoring and periodic review scheduled.

• Service — Microsoft 365 and Entra ID security assessment →
• Related guide — Microsoft 365 token theft and AiTM →