Skip to main content

The question I would put to an executive team after the PHP2Shell emergency is straightforward: if we had to take a critical service offline this afternoon, how long before we could safely do business again? I would want an answer that includes the people making decisions, the suppliers we depend on and the work our customers still need us to do.

On 21 October, the PHP project released emergency fixes for two critical vulnerabilities capable of allowing remote code execution without authentication (PHP security advisory). Reuters subsequently reported millions of exploit attempts and precautionary service suspensions across companies and governments, including Orange, Japanese financial and transport organisations, and Finnish public services (Reuters).

We should be precise about what that tells us: exploit attempts and precautionary shutdowns are not evidence that every affected organisation was breached, and Unit 42 has retracted an earlier claim linking a government compromise to these particular vulnerabilities (Unit 42’s correction). My concern as a CIO is how we prepare for the scale of disruption such a situation can create, including when shutting a service down is the responsible decision.

Prepare for a problem bigger than your own company

I expect large, deliberately coordinated attacks to become a bigger problem in the years ahead. That is a planning assumption, not a claim that PHP2Shell was one centrally orchestrated campaign. I would rather build the capacity to handle that pressure than base our readiness on attacks remaining isolated and manageable.

The scenario I want us to consider in this post is uncomfortable. An organisation is responding to an incident while several of its suppliers are responding to theirs. The specialist planned to call is already supporting other clients. A platform expected to use for recovery is itself under review. Customers need answers before the technical picture is complete.

This changes the exercise. A recovery plan that works when everyone else is available may be inadequate when an entire ecosystem is under pressure. We should rehearse with some of those dependencies unavailable, and see which commitments we can still honour.

In my earlier article on assuming the breach, I argued for containment, tested recovery and clear ownership. I now want to put a clock against those commitments. How quickly can we establish what is exposed, make a defensible decision and restore something useful?

Agility has to survive the first difficult decision

It is easy to value agility when we are launching something new. I am more interested in whether we still practise it when someone has to suspend a revenue-generating service with incomplete information. Who can authorise that decision? What evidence do they need? Who can commit emergency spending, approve a temporary way of working or decide which customers receive limited service first? If all of those questions travel to the same executive, we have designed a queue at the moment we most need movement.

I would want authority agreed in advance, with clear limits and escalation points. The people closest to the problem should be able to take defined containment actions without waiting for a meeting. Decisions with wider consequences need a named accountable leader and an available deputy.

That still leaves room for judgement. Information will change, and a sensible decision at nine in the morning may need revisiting at eleven. Leadership has to make that revision possible without turning it into an admission of failure. Otherwise, people may defend yesterday’s decision when they should be responding to today’s evidence.

Rebuilding means deciding what we can trust

For me, “the systems are back” would be an incomplete recovery report. I want to know whether the business can rely on them, and what evidence supports that confidence. Where compromise is suspected or confirmed, I would expect the recovery plan to address clean environments, trusted identities, credentials, data integrity and the route by which the attacker might return. A patched application and a restored backup should not, on their own, be treated as permission to reconnect everything.

The business also needs a recovery order. Restoring a customer portal is of limited use if the teams behind it cannot validate requests or fulfil commitments. I would start with the smallest complete service we can deliver safely, then expand it as the supporting capabilities become trustworthy. That requires choices before a crisis. Which activities must continue? Which can run manually for a limited period? Which should stop because a degraded version would create unacceptable risk? Finance, operations, legal and customer-facing leaders need to help make those choices. IT cannot infer the company’s priorities from an application inventory.

Measure the ability to act

I would ask for exercises that produce elapsed times and uncomfortable findings. How long did it take to identify the affected services and their owners? How long before someone authorised containment? When could a business team complete a real transaction safely? The exercise should include a missing decision-maker, an unavailable supplier or an internal communication channel we cannot trust. Those are useful constraints because they force us to test the organisation around the technology. A successful technical restore should not conceal the fact that nobody could approve its return to service.

I would also measure how long we can sustain a reduced service. A workaround that depends on the same few people working through the night needs a time limit, relief and a plan for the backlog. Their effort deserves recognition; our readiness should not depend on their exhaustion. This is where resilience becomes a concrete investment discussion. We can compare the cost of shortening recovery with the business consequences of a longer interruption. We can decide where additional capacity, simpler dependencies or better-rehearsed decisions would make the greatest difference.

The work continues after service returns

My expectation is that the ability to react, rebuild and adapt will increasingly distinguish organisations that remain dependable under pressure. Prevention still deserves serious investment. Alongside it, I want evidence that we can operate through disruption without losing control of our decisions or making promises we cannot keep.

After an incident or exercise, I would keep the recovery work open long enough to change what slowed us down. An unclear responsibility needs an owner. A fragile dependency needs a decision. A temporary workaround needs either a deliberate place in the operating model or a retirement date.

The next major attack will arrive on its own timetable. Before it does, I want us to know how long we need to make a sound decision, restore a trusted service and get people working again. That is a much more useful answer than “we have a plan.”