DORA testing and external exposure monitoring

When your IT lead brings you the report from a regular resilience test, it usually confirms one thing: the system withstood the planned loads and failure scenarios at a particular moment in time. Between two such tests, time passes. In that time a developer opens a new port for an integration, a cloud provider changes a default security setting, and a contractor connects their own service to your external API. If those changes stay invisible until the next scheduled test, the resilience that was signed off describes a configuration that no longer exists.

Regulation (EU) 2022/2554 (DORA) sets mandatory requirements for managing ICT risk. One of its key articles requires financial entities to test their digital operational resilience regularly. The regulation itself does not prescribe a single method or a single frequency for that testing — it sets the objective: to establish that the digital infrastructure can keep operating under significant stress and disruption. Which raises a question worth asking. Is a periodic snapshot of the system enough to meet the requirement, or are there blind spots that this approach cannot see by design?

What DORA testing checks

Article 26 of Regulation (EU) 2022/2554 requires financial entities to carry out regular testing of their digital operational resilience (DORA, Article 26). The purpose of that testing is to assess the ability of the organization to keep functioning under extreme but plausible threats. It is not a check that the servers are switched on. It is the modelling of scenarios such as large-scale cyberattacks, failures at critical suppliers, data loss, or the breakdown of whole infrastructure components.

It is worth understanding the nature of this control: it is a diagnostic procedure with a fixed horizon. You take the current configuration of your system, apply a set of stress factors to it, and analyse the outcome. That approach does one important job — it exposes weak points in the architecture and in the recovery processes. A test may show that when the primary communication channel fails, the backup channel comes up too slowly and breaches the agreed service levels (SLA). Or that backup copies cannot be restored within an acceptable time. Findings like these carry weight because they concern the fundamental resilience of the business, not only technical detail.

The approach has a hard structural limit, though: it works with what exists at the moment the test starts. The configuration of the system is treated as unchanged for the duration of the procedure. If the configuration changes between tests — and in any live IT environment it changes constantly — the results of the previous test become partly out of date.

Think of a medical check-up you passed a month ago. The results were good. If your habits have changed since then, your actual state of health already differs from what the record says. DORA testing is similar: it gives an accurate picture as of the date it was run, but it does not track what changes after that date.

For a financial organization the most sensitive area of risk often sits at the boundary with the outside world: public websites, API gateways, mail servers, access paths for external partners. This is where the attack surface forms — the set of all points through which an intruder may try to get into the system.

The difficulty is that the external perimeter is the most dynamic part of the infrastructure. New microservices are published to the internet daily. Old projects get forgotten and leave ports open. Cloud resources are created and destroyed automatically. All of these changes affect risk, and none of them are reflected in the results of the last scheduled DORA test.

If we rely on periodic testing alone, we are effectively deciding on the level of protection from data that has aged. Between tests, any of the following can happen:

Developers open a new API endpoint for an integration with a partner.
The cloud platform updates a network firewall configuration and accidentally opens access to an internal service.
A contractor creates a temporary environment that stays reachable from the internet after the work is finished.

None of these events will be discovered until the date of the next regular test comes round. In that interval the organization operates with a level of risk that is higher and undocumented. For the regulator that means the resilience being claimed is not backed by current data. For the business it means exposure that nobody has assessed.

It is worth being clear about one thing: this is not a flaw in the testing process. It follows from the nature of a discrete control mechanism. Periodic checks can never close the gap in time between the moments they are run. They give you a point, not a line.

Continuous observation: filling the gap in time

This is where a second type of control comes in: continuous observation of the external attack surface (External Attack Surface Management, EASM). Unlike DORA testing, which reacts to changes after the fact, EASM is proactive. It continuously scans and monitors the publicly reachable assets of the organization: domains, IP addresses, certificates, open ports, DNS records.

What does such observation contribute in the context of the DORA requirements?

It finds new public services as they appear

If a developer opens a new API, the monitoring system records the event almost immediately. That lets the security team add the new asset to the list of things being protected and — this part matters — to the plan for the next round of testing. The next DORA test then works with the current composition of the external surface.

It detects configuration drift

Drift is the situation where the actual state of a system differs from the reference state approved by security policy. An administrator may have opened a port temporarily for diagnostics and left it open. EASM systems compare the current state with a baseline and flag the deviations.

It supplies data for risk assessment between tests

If monitoring shows a sharp increase in the number of open ports, that is a signal for an unscheduled risk assessment. It does not replace a full DORA test, but it allows an informed decision: whether to break the planned schedule or wait for the next iteration.

Why they complement each other rather than replace each other

There is sometimes a temptation to pick one of the two approaches instead of the other: to assume that with continuous monitoring in place, deep testing is unnecessary — or the reverse, that regular penetration tests make monitoring redundant. Both views are mistaken, because the two instruments answer different questions.

Their characteristics compare as follows.

DORA testing

Question: can the system keep operating under extreme loads and disruptions?
Method: active scenario modelling, stress tests, chaos engineering.
Frequency: regular, according to internal policy.
Limit: works with a snapshot of the configuration at the time of the test. Does not see changes between tests.
Cost: high. It takes specialist resource, downtime or an isolated environment, and elaborate scenario preparation.

Continuous observation (EASM)

Question: which assets are reachable from the internet right now, and has their configuration changed?
Method: external scanning, metadata collection, change monitoring.
Frequency: continuous, 24/7.
Limit: does not assess internal resilience. Does not check logic flaws in code or business processes. It sees the surface only.
Cost: moderate. Automated execution, low demand on the company's own resources.

A simple conclusion follows from the comparison: neither method covers the whole range of risks that DORA addresses. Testing without monitoring is blind to changes at the perimeter. Monitoring without testing is blind to how deep the consequences of those changes go.

EASM may find that a new web server is reachable from the internet. That is an important fact. But the fact of reachability on its own says nothing about how critical that server is to operations, or how long it would take to restore it after a compromise. Only DORA testing gives that information, by modelling a compromise scenario for this specific asset and assessing the effect on business processes.

The reverse case: a DORA test may show that the backup system works correctly. If EASM then shows that the server holding the backups has unexpectedly become reachable from the external network, the risk context changes. The backup copy is no longer protected to the degree that was assumed at the time of the test.

Where the tooling does not help: an honest view

It would be wrong to claim that automated scanning and monitoring settle every DORA compliance question. Some areas depend entirely on human process and management decisions.

Interpreting the results

A tool can say: “port 8080 is open”. Deciding whether that is a policy violation or an accepted exception is a human judgement. Automation does not know the business context. It does not know that the port is needed for a particular integration with a partner bank, and that closing it would halt operations. Every EASM finding therefore has to pass through a risk assessment carried out by people.

Managing third parties

DORA gives considerable attention to the risks that arise from service providers (DORA, Article 30). Monitoring your own external surface does not give a full picture of your vendors. You can know which of your services are exposed, but not which vulnerabilities exist in the systems of your hosting provider or cloud supplier. That requires a separate process of gathering information from counterparties and assessing their security maturity. No external scanner substitutes for contractual obligations and audit requests to partners.

Organizational resilience

Part of the DORA requirements concerns human factors: staff training, incident response processes, communication with regulators. A technical tool will not teach employees to recognise phishing email, and it will not deliver a timely notification of a major incident to the ECB or a national regulator. Those are process tasks.

A compliance strategy therefore has to be hybrid. Technical means — testing plus monitoring — provide visibility and an evidence base. Process mechanisms — policies, training, engagement with suppliers — provide governance and accountability. One cannot stand in for the other.

Choosing the control: situation by situation

The summary below helps identify which type of control a given situation calls for. It does not replace detailed planning, but it gives a frame for the decision.

You need to demonstrate DORA compliance through regular testing

Recommended approach: periodic testing of digital operational resilience.
Why this one: a direct requirement of DORA Article 26. It produces evidence of resilience to stress factors.
Limits of the approach: expensive and slow; it produces a snapshot that ages quickly. It does not see changes between tests.

You need to know which services are reachable from the internet right now

Recommended approach: continuous observation (EASM).
Why this one: it gives a current map of the external surface in close to real time.
Limits of the approach: it does not assess the depth of vulnerabilities. It does not check application logic. It requires human interpretation.

A new public service has been found that has not been through testing

Recommended approach: an immediate risk assessment, plus adding the asset to EASM coverage.
Why this one: it makes the scale of the threat clear quickly and puts the asset under continuous monitoring.
Limits of the approach: it does not replace full testing. Without deep analysis the risk assessment can stay superficial.

Firewall configuration drift is suspected

Recommended approach: an unscheduled scan and a comparison with the reference state.
Why this one: it surfaces the discrepancy between the expected and the actual state quickly.
Limits of the approach: it can produce false positives. It does not explain the cause of the change — who made it and why.

Managing risks from third-party providers

Recommended approach: a process audit and document requests.
Why this one: DORA requires control over the supply chain. Technical scanners do not reach inside vendor systems.
Limits of the approach: it depends on the transparency and cooperation of counterparties. It is a slow process.

Preparing for an incident

Recommended approach: a combination — EASM data plus the scenarios from the most recent tests.
Why this one: EASM shows where a hit may land, testing shows how the system fails. Together they give a fuller picture of readiness.
Limits of the approach: it requires integrating data from different systems. Keeping both parts current takes effort.

In closing

DORA compliance is not a one-off exercise but a continuous risk management cycle. Periodic testing of digital operational resilience is a necessary element of that cycle, because it examines the fundamental strength of the system. It works in discrete time, though, and cannot see what changes in the interval between tests.

Continuous observation of the external attack surface covers that gap in time. It keeps a current map of exposure, so new risks surface earlier and testing plans can be adjusted. The two approaches are not competitors: together they form a closed feedback loop. Monitoring feeds testing with current configuration data, and test results sharpen the criteria for what monitoring should treat as important.

As a business leader you do not need to become an expert in scanner settings or chaos engineering methods. Your job is to confirm that both elements are present in the security strategy, and that there is a clear process for passing data between them. One question to ask your IT lead: “how does information about new external services reach our regular testing?” If the answer is not obvious, there is a gap in the DORA approach that is worth closing.

What continuous external exposure adds to DORA testing