Delta: CrowdStrike's offer to help in Falcon meltdown was too little, too late
Airline unimpressed with 'unhelpful and untimely' phone call from CEO, Falcon maker says claims untrue
Delta Air Lines has come out swinging at CrowdStrike in a letter accusing the security giant of trying to "shift the blame" for the IT meltdown caused by its software – and that CrowdStrike CEO George Kurtz's offer of support was too little, too late.
Last month, CrowdStrike pushed out a flawed update to its Falcon threat-detection system that crashed and disabled more than 8 million Microsoft Windows machines worldwide. That figure included more than 37,000 Delta computers, disrupting more than 1.3 million people's travel plans, according to a Thursday letter from Delta's attorney David Boies to CrowdStrike's lawyer Michael Carlinsky.
Soon after that breakdown, Delta threatened legal action against CrowdStrike and Microsoft, claiming the IT fiasco cost it more than $500 million. And indeed, a lawsuit is looking more likely by the minute as the airline ratchets-up its criticism of the security software developer. Today, Delta laid out its defense for how it handled itself in the wake of that disastrous Falcon update, which grounded planes and ruined millions of Delta customers' plans.
CrowdStrike tried to "blame the victim" in its sorry-not-sorry August 4 letter to the airline, according to Boies in his missive today, adding "there is no basis – none – to suggest that Delta was in any way responsible for the faulty software that crashed systems around the world." CrowdStrike had suggested Delta was responsible to some degree for the grief it had suffered in July.
Boies latest letter [PDF] cites the software developer's own preliminary post-incident review and root cause analysis, which Boies says proves that "CrowdStrike engaged in grossly negligent, indeed willful, misconduct with respect to the faulty update." This, in part, is due to the software company's admission that it didn't do a staged roll-out of its errant update.
CrowdStrike is facing a class-action lawsuit from investors for failing to do a staggered release of changes to Falcon, and in both of its postmortem examinations of the July 19 fiasco, the embattled security shop has pledged to improve its testing and do canary deployments of future updates.
But perhaps even worse: After borking Windows machines around the world, CrowdStrike didn't show a "sense of urgency or appreciation for the scale and scope of the damage" it was responsible for causing, the letter adds. Delta pushed back on CrowdStrike's claims of working "tirelessly" to help Delta restore its systems.
The only offer of help the airline got during the first 65 hours of the outage was the publicly available remediation website suggesting manual reboots of all affected computers, we're told. Plus, the automated fix from July 21 "introduced a second bug that prevented many machines from recovering without additional intervention," according to the letter.
- CrowdStrike hires outside security outfits to review troubled Falcon code
- CrowdStrike unhappy about Delta's 'litigation threat,' claims airline refused 'free on-site help'
- Microsoft punches back at Delta Air Lines and its legal threats
- Delta Air Lines dials up Microsoft's legal nemesis over CrowdStrike losses
By the time Kurtz called Delta CEO Ed Bastain — and this only happened one time, Boies asserts — on the night of July 22, it was "too late." The phone call was "unhelpful and untimely," arriving almost four days after the disaster, by which time "Delta had already restored its critical systems and most other machines," the letter claims.
CrowdStrike's earlier letter to Delta blamed the airline's "IT decisions" for the fallout, while a similar one sent from Microsoft's attorney essentially accuses Delta of using super-old and outdated gear.
And, it appears, Delta isn't going to let that slide, either. The airline talked up the "billions of dollars" it has invested in its IT, and added: "Reliance on CrowdStrike and Microsoft was the reason Delta took longer to fully recover" compared to its industry peers.
Here's what the letter says to this allegation:
Approximately 60 percent of Delta's mission-critical applications and their associated data — including Delta's redundant backup systems — depend on the Microsoft Windows operating system and CrowdStrike. Delta has long regarded CrowdStrike and Microsoft as reliable technology providers. Delta's reliance on CrowdStrike and Microsoft actually exacerbated its experience in the CrowdStrike-caused disaster.
The letter urges CrowdStrike to stop trying to "evade responsibility," and tell customers everything it knows about how and why the disaster occurred. "It will all come out in litigation anyway."
When asked about this August 8 letter from Delta, a CrowdStrike spokesperson told The Register:
Delta continues to push a misleading narrative. CrowdStrike CEO George Kurtz called Delta board member David DeWalt within four hours of the incident on July 19th. CrowdStrike's Chief Security Officer was in direct contact with Delta's CISO within hours of the incident, providing information and offering support.
CrowdStrike's and Delta's teams worked closely together within hours of the incident, with CrowdStrike providing technical support beyond what was available on the website.
This level of customer support led Delta board member David DeWalt to publicly state on LinkedIn: "George and his team have done an incredible job, working through the night in difficult circumstances to deliver a fix. It is a huge credit to the Crowdstrike team and their leadership that many woke up to a fix already available."
A Delta spokesperson said the airline "will decline to comment further." ®