The most expensive mistakes in tech history, and the lesson each one teaches

What happened: NASA's first Venus probe veered off course minutes after launch, and the range safety officer destroyed it 4 minutes 50 seconds into the flight. The cause was a missing overbar on an "R" in a handwritten guidance equation. Without it, the ground computer misread the rocket's movement and sent "corrections" it didn't need. Arthur C. Clarke later called it "the most expensive hyphen in history".

The lesson: the step where something written by a human becomes something run by a machine is where errors hide. Treat that hand-off, whether it's a spec becoming code or a change request becoming a script, as something to review, not just transcribe.

What happened: on its first flight, Europe's new rocket broke up less than a minute after launch. Its guidance software tried to fit a 64-bit number into a 16-bit integer. The value was too big, the conversion failed, and the guidance system shut down. The rocket and its payload, valued at around $500 million, were lost.

The lesson: code or configuration that worked somewhere else carries that system's assumptions with it. When you reuse a script, a template or a policy, check what it assumed, not just whether it ran last time.

What happened: NASA's investigation found the root cause was "the failed translation of English units into metric units" in ground software. One team's output was in pound-force seconds, and the navigation team expected newton-seconds. The spacecraft approached Mars far too low and was lost.

The lesson: every interface between two teams or two systems is a contract. Units, time zones, date formats and IDs need writing down, because each side will otherwise assume its own.

What happened: a trader meant to sell one share in J-Com at ¥610,000, and instead offered 610,000 shares at ¥1 each. A problem with the Tokyo Stock Exchange's own system stopped him cancelling the order. Mizuho booked a ¥40.7 billion charge, about $340 million at the time, buying the shares back.

The lesson: systems should question inputs that make no sense. An order to sell 610,000 shares for ¥1 each, or a bulk change to every user in a tenant, deserves a "are you sure?" that a person has to answer. And the undo button has to actually work.

What happened: according to the US SEC, Knight deployed new trading code alongside a function left over from 2005 that was never meant to be used again. When markets opened, the old function woke up. In 45 minutes, Knight's router sent more than 4 million orders and traded more than 397 million shares, for a loss of more than $460 million.

The lesson: dead code and stale configuration aren't harmless. Remove what you don't use, and make sure a deployment reaches every server, not just most of them.

What happened: an engineer running an established command to remove a small number of servers from Amazon S3 in Northern Virginia mistyped one input, and a much larger set of servers was removed. S3 in that region was down for around four hours, and with it, large parts of the internet. Analytics firm Cyence estimated the cost to S&P 500 companies alone at at least $150 million.

The lesson: AWS's fix wasn't "type more carefully". It changed the tool so it removes capacity more slowly and refuses to go below a safe minimum. Build guardrails into scripts and tools, so a single typo can't do unlimited damage.

What happened: over a bank holiday weekend, a power failure at a data centre took down British Airways' systems, and the backup systems didn't cope. 672 flights were cancelled and thousands of passengers were stranded. The cost was estimated at £58 million.

The lesson: a disaster recovery plan you haven't tested is a hope, not a plan. Fail over for real, on a schedule, and find out what breaks while it's quiet.

What happened: TSB moved its 5.2 million customers to a new banking platform in one go. The data moved, but the platform failed straight away, and branch, phone, online and mobile banking were all disrupted, for some customers until December. TSB put post-migration costs at £330.2 million, and regulators later fined it £48.65 million for failing to manage the migration and its outsourcing risks.

The lesson: big-bang changes put everything at risk at once. Move in waves, prove each one, and make sure you can stop and roll back before the next.

What happened: while making a routine manual adjustment to a loan, a Citi employee selected the wrong options in the bank's software, and Citi paid off a Revlon loan of around $900 million years early. The colleagues meant to check it didn't catch the mistake. Some lenders returned the money, but others kept about $504 million until an appeals court ruled in Citi's favour in 2022.

The lesson: a second pair of eyes only works if the screen makes the consequence obvious. If approving a change means reading cryptic options, the four-eyes check is theatre. Show people what will actually happen before they confirm.

What happened: a faulty update to CrowdStrike's Falcon security software sent Windows PCs into a crash loop. Microsoft estimated that 8.5 million Windows devices were affected, less than 1% of all Windows machines, but they included airlines, hospitals, banks and broadcasters. Insurer Parametrix estimated direct losses for US Fortune 500 companies alone at $5.4 billion.

The lesson: anything that updates everything at once can break everything at once. Roll changes out in rings, starting small, and make sure you can pause the rollout before it reaches everyone.