Developing a technology with safety in mind

Lessons from the Wright Brothers

Jun 16, 2023

In this update:

Full text below, or click any link above to jump to an individual post on the blog. If this digest gets cut off in your email, click the headline above to read it on the web.

Should I turn on Substack Chat for this newsletter?

Developing a technology with safety in mind: Lessons from the Wright Brothers

If a technology may introduce catastrophic risks, how do you develop it?

It occurred to me that the Wright Brothers’ approach to inventing the airplane might make a good case study.

The catastrophic risk for them, of course, was dying in a crash. This is exactly what happened to one of the Wrights’ predecessors, Otto Lilienthal, who attempted to fly using a kind of glider. He had many successful experiments, but one day he lost control, fell, and broke his neck.

*Otto Lilienthal gliding experiment. Wikimedia / Library of Congress*

Believe it or not, the news of Lilienthal’s death motivated the Wrights to take up the challenge of flying. Someone had to carry on the work! But they weren’t reckless. They wanted to avoid Lilienthal’s fate. So what was their approach?

First, they decided that the key problem to be solved was one of control. Before they even put a motor in a flying machine, they experimented for years with gliders, trying to solve the control problem. As Wilbur Wright wrote in a letter:

When once a machine is under proper control under all conditions, the motor problem will be quickly solved. A failure of a motor will then mean simply a slow descent and safe landing instead of a disastrous fall.

When actually experimenting with the machine, the Wrights would sometimes stand on the ground and fly the glider like a kite, which minimized the damage any crash could do:

*The Wrights flying their glider from the ground. Wikimedia / Library of Congress*

When they did go up in the machine themselves, they flew relatively low. And they did all their experimentation on the beach at Kitty Hawk, so they had soft sand to land on.

All of this was a deliberate, conscious strategy. Here is how David McCullough describes it in his biography of the Wrights:

Well aware of how his father worried about his safety, Wilbur stressed that he did not intend to rise many feet from the ground, and on the chance that he were “upset,” there was nothing but soft sand on which to land. He was there to learn, not to take chances for thrills. “The man who wishes to keep at the problem long enough to really learn anything positively must not take dangerous risks. Carelessness and overconfidence are usually more dangerous than deliberately accepted risks.”
As time would show, caution and close attention to all advance preparations were to be the rule for the brothers. They would take risks when necessary, but they were no daredevils out to perform stunts and they never would be.

Solving the control problem required new inventions, including “wing warping” (later replaced by ailerons) and a tail designed for stability. They had to discover and learn to avoid pitfalls such as the tail spin. Once they had solved this, they added a motor and took flight.

Inventors who put power ahead of control failed. They launched planes hoping they could be steered once in the air. Most well-known is Samuel Langley, who had a head start on the Wrights and more funding. His final experiment crashed into the lake. (At least they were cautious enough to fly it over water rather than land.)

*The wreckage of Langley's plane in the Potomac River.*

The Wrights invented the airplane using an empirical, trial-and-error approach. They had to learn from experience. They couldn’t have solved the control problem without actually building and testing a plane. There was no theory sufficient to guide them, and what theory did exist was often wrong. (In fact, the Wrights had to throw out the published tables of aerodynamic data, and make their own measurements, for which they designed and built their own wind tunnel.)

Nor could they create perfect safety. Orville Wright crashed a plane in one of their early demonstrations, severely injuring himself and killing the passenger, Army Lt. Thomas Selfridge. The excellent safety record of commercial aviation was only achieved incrementally, iteratively, over decades.

*The wreck of the crash that killed Lt. Selfridge. Picryl*

And of course the Wrights were lucky in one sense: the dangers of flight were obvious. Early X-ray technicians, in contrast, had no idea that they were dealing with a health hazard. They used bare hands to calibrate the machine, and many of them eventually had to have their hands amputated.

*An X-ray experiment, late 1800s. Wikimedia*

But even after the dangers of radiation were well known, not everyone was careful. Louis Slotin, physicist at Los Alamos, killed himself and sent others to the hospital in a reckless demonstration in which a screwdriver held in the hand was the only thing stopping a uranium core from going critical.

*Recreation of the Slotin “demon core” incident. Wikimedia / Los Alamos National Lab*

Exactly how careful to be—and what that means in practice—is a domain-specific judgment call that must be made by experts in the field, the technologists on the frontier of progress. Safety always has to be traded off against speed and cost. So I wouldn’t claim that this exact pattern can be directly transferred to any other field—such as AI.

But the Wrights can serve as one role model for how to integrate risk management into a development program. Be like them (and not like Slotin).

Original post: https://rootsofprogress.org/wright-brothers-and-safe-technology-development

Links and tweets: The illusion of moral decline, turnspit dogs, and more

Announcements & opportunities

News

Ted Kaczynski, the “Unabomber,” has died in prison at 81. See Kevin Kelly’s summary of and rebuttal to his philosophy

Requests for books/sources on…

Other queries

Quotes

Tweets & threads

Infrastructure that looks like sci-fi

Maps & charts

Holding period of stocks over time

Revenue evolution for various industries

When Singapore needs more land, it builds more

It used to be hazardous to breathe outdoor air for much of the year

Original post: https://rootsofprogress.org/links-and-tweets-2023-06-14

J.K. Lund

Jun 17, 2023

Today, risk aversion appears to prevail over everything that we do. But without risk, there can be no progress. We must strike a balance between obsessive risk mitigation and the need for continued progress.

One outlier today is SpaceX. They are willing to employ an iterative approach, much like the Wright Brothers, building, testing, and failing to figure out where the flaws reside. SpaceX is comfortable with rockets blowing up, because the data these explosions provide ultimately will lead to safer designs.

Expand full comment

1 reply by Jason Crawford

Elle Griffin

Such a good piece!!!!! And so relevant today.

1 more comment...

The Roots of Progress

Discussion about this post