In January 1937, the USSR commissioned a census. It was the first census conducted since 1926. The census was not intended for public dissemination. Instead, it was developed purely for leadership decision-making purposes. When the data was collected and reported, it showed a massive level of mortality due to the famines of 1932-3. The data, while accurate, was unacceptable. As a result, all of the statisticians involved with the project were arrested and executed. Again, the data was never intended for public consumption. The leadership themselves could not allow themselves to be exposed to data that contradicted what they wanted to hear.
The 1937 census was a powerful illustration of the most corrosive effects of autocratic management: the loss of real-world feedback in the service of effectively closed-loop management ambition.
There are many other examples of this effect. In periods leading up to government personnel decisions in Chinese provinces, GDP figures are routinely inflated – when measured against harder to fudge information such as electricity and rail car usage. In fact, when comparing official GDP vs. nighttime electric light visibility, researchers have discovered a clear pattern: the more autocratic the government, the higher the deviation between GDP growth and the observable economic signal of electric light usage. In fact, authoritarian regimes inflate their GDP growth by a factor of 15 to 30%. Normally, this inflation is seen as a side effect of regimes wanting to maintain their own legitimacy. Sort of like Bernie Madoff reporting consistent earnings growth to keep clients.
But there is another possible reality, hinted at again by the 1937 census. There is the possibility that bad data is produced because the leadership themselves want to hear it. As a leader, largely blind to what is happening far below you, you want to reward the successful. Top-line data becomes a means of doing exactly this. Data is fudged in the service of ambition. In the absence of other feedback mechanisms (whether through a truly free press, elections or market acceptance) that data metastasizes into a cancer that can threaten an entire state apparatus.
I believe we are seeing a side effect of this in the Ukraine war. Russian equipment is proving to be far less effective than almost everybody believed possible prior to the conflict. Russia had spent heavily on modernization and was considered to have the second-most powerful conventional military in the world – ahead of China. And yet they are facing tremendous problems in Ukraine. Compare their loss of thousands of soldiers and the verified loss of hundreds of tanks and even dozens of aircraft with the U.S. invasion of Iraq. During the conventional part of those conflicts, the U.S. lost very few soldiers and almost no equipment.
So, what has gone wrong for Russia?
There are many answers to this question, from weather to logistics. As an aerospace quality manager, I want to focus on one particular answer: autocratic quality management.
Setting aside any justifications for this conflict, Vladimir Putin had to weigh the possible outcomes of the decision to go to war. I imagine there were three fundamental beliefs that drove his decision making:
1) He had far superior military technology
2) He had a truly professional army
3) He had broad support from Russian speakers in Ukraine – and perhaps even from Ukrainians fed up with their own incompetent government (though that apparent incompetence might actually have reflected a far more functional system built on the exposure of faults – like a somewhat free press).
All three statements could have been mirrored by US planners prior to the second Iraq War. Except that in the U.S. case, the first two would have been true. In Russia’s case none have been conclusively borne out.
So, what went wrong? One possible answer is that the Russian leadership was effectively selfdeceived. They succeeded at deceiving everybody else, too. Everybody knew the Russian military was superior in every way. This sort of self-deception has been endemic in Russian history. Certainly, we can look at Stalin. But Czar Nicholas II believed he could crush the Japanese army and navy. He believed he was destined, arguably as G-d’s chosen bulwark against the Yellow Peril, to do so. Bad data (based on a show army that marched impressively with shiny weapons) probably played a role. However, we don’t need to go all the way back to Stalin. Two of the most frightening Soviet fighter jet programs were similarly cursed.
The MIG-25 so frightened the US that they forced a complete re-engineering of the F-15. The MiG had been observed flying over Israel during the 1973 war at 63,000 feet and up to Mach 3.2. The U.S. spent $1.1 billion (real money in those days) to counteract the dreaded Foxbat. In fact, the aircraft’s performance was almost laughable. Effective combat range was 300 kilometers. And those very scary Israeli runs? They actually destroyed the engines, which had to be swapped out afterwards. The whole aircraft was window dressing. It is quite possible the senior leadership of the USSR believed some of what the West did.
In a way, the MiG-29 was even more revealing. It performed wonderfully in some ways, particularly mechanically. But the avionics were woeful. The Soviet war-fighting methodology was top-down and so MiG-29 pilots weren’t given the systems necessary to make decisions on their own. Targets had to be effectively called out for them by radar operators. The U.S. considering using its own stock of M-G-29s in Kosovo but decided against it because they were more trouble than they were worth.
Both aircraft looked almost impossibly good. Senior management could easily believe they were. On parade and demonstration, they were stars. Who would think they needed to dig deep down and actually witness the full-scope of performance? It can be hard to even understand where to start? Are you going to fly alongside to see range? How are you going to assess avionics? The leadership is blind to the reality due to an simple truth: the higher you climb, the less detail you know. If you are in a culture that encourages lying upwards, the national leadership can be presented with a completely inaccurate picture.
Yes, these are old aircraft. The MiG-25 was introduced in 1970 and the MiG-29 in 1982. Then again, the F-15 was introduced in 1972 and the F-16 in 1974. It is not hard to imagine that the management problems have persisted, just as some of the aircraft have.
Of course, design is only one stage in which quality can have an impact. The same sorts of problems can persist through manufacturing, maintenance and usage (e.g. actual fighting). American gun enthusiasts have long praised the survivability of the AK-47 and its elegant simplicity. The reality is that the gun had to be elegant and simple because manufacturers and soldiers were simply not dependable!
So, how do Western aerospace and military equipment avoid these problems? Why have Javelin missiles worked? Why did the U.S. military so completely overwhelm Saddam’s tanks and artillery? How can Israel routinely intercept Iranian drones while Iran and Russia seem incapable of dealing with Israeli – and even Turkish – drones? Just yesterday, several hundred Iranian drones were apparently destroyed by three Israeli drones, while last week two Iranian drones were downed on their way to Gaza.
The answers, again, have to do with quality and culture. Western military contractors, for all that their cost structures are routinely mocked, approach quality in a fundamentally different way than the Russians do. Israelis have yet another path, which is also effective.
The modern American quality process has its genesis in World War II. During the early 1940s, Bell Labs came up with a “systems engineering” practice in order to connect military planning with research and development decisions. RAND (a contraction of “Research ANd Development”) was founded as a result. It promulgated systems engineering processes. A key part of it is often represented via the graphical “V-model”. You start with a rigorous project definition, including extensive validation of requirements. Sometimes the top-level requirement process is ridiculous (see The Pentagon Wars (https://www.youtube.com/watch?v=ir0FAa8P2MU)). But design is only a small part of the actual process. Validation of requirements, and then their verification (“are the requirements met?”), dominate the process. You combine this with audit of the process itself as well as continual risk assessments and change controls and you can produce highly complex quality products. Sure, you’ll often overshoot your budget in surprising ways – but you will produce a quality product. Unless the management process is undermined by senior leadership (e.g. 737MAX), it will produce quality.
The Israeli (and SpaceX) process is a little different. Instead of vast development projects, their method is more agile-like. Small iterative steps are taken with quality verified at each stage. This process allows for failures; but keeps them within scope. SpaceX blows up lots of rockets, but rarely with actual payloads. There have been two or three payload failures since 2010 (one possible failure is classified). Only 1.4% of their actual payloads have suffered partial or complete failures. The agile-like learning process has kept the failure impacts largely within the testing environment. The agile-learning process again emphasizes learning and improvement.
In both of these approaches, especially within the aerospace industry itself, there is a further incentive to quality. If there are failures (for example, errors made by a pilot), data is collected, root causes are analyzed and changes in process are rolled out. What is not carried out is punishment. Unless there was actual malfeasance nobody is punished. This encourages the sharing of data and enables a culture of continuous improvement. There is no 1937.
Contrast that with the Russian space program: In 2016, the Planetary Society reported that Russia had 15 launch failures from 2011 to 2016 ranging from explosions of satellite-ferrying rockets to placing satellites in the wrong orbits. There were 207 launches for a failure rate of 7.2% – 5 times as high as SpaceX. As a result of one of these failures, the Russian government put three managers on trial due to mishandling a change request. Numerous others were named and shamed. This approach fails to identify process issues while encouraging the falsification of data through an organization. “Cover Your A–” becomes far more important than trying to figure out what went wrong, and how to do it better.
For its part, Israel has rolled out missile defense not in one great leap forward but through a process of constant upgrades. Getting something in the field fast was worth absorbing the resulting imperfections (the greatest of which has been a high rate of cancer among operators). The general principle has been: the systems being built are better than anything else out there and they will meet more and more requirements soon. This process also yields quality. Not through an overwhelming V-model, but through iterative and continual test and improvement. The tolerance for error inherent in this approach can be very effective for design. However, it is not great for manufacturing (see Tesla’s somewhat-infamous build-quality). Israel manufactures actual Iron Dome interceptors in the U.S. for a reason.
The Russians have a method of minimizing their own quality issues. Go to the Boeing Museum and you’ll see a space shuttle next to another launch vehicle, the Soyuz. The Soyuz, was extremely simple and got the job done. The other was far more complex (disastrously so). The Americans developed the wrong requirements, while the Russians kept their requirements tightly defined and thus had a human launch system that far outlasted the space shuttle. (Almost poetically, regarding quality problems, the only Russian space shuttle was destroyed in a snowstorm in 2002 when its hanger collapsed due to poor maintenance. Sadly, a number of staff were also killed.)
The problem is that there are limits to simplicity when dealing with highly complex avionics, active protection-systems for tanks, anti-aircraft interceptor systems and the like. In these cases, there is no simple solution and so elegant engineering and tight requirements aren’t enough for an effective piece of hardware. This is especially true when headline performance (speed, rate of fire, radar range) is treated as more important than meeting real-world requirements. As with the fall of the USSR, a grand facade was exposed. $40,000 Javelin missiles are routinely taking out $2 million tanks while $1 million Turkish drones can destroy a $50 million anti-aircraft system – and then repeat the performance again and again. Yes, the Russian method is faster and cheaper. But if the equipment doesn’t end up working – especially when going up against better designed and manufactured systems – then it is entirely useless.
Looking at the conflict through a systems quality lens, it is entirely possible that Russian failures may well be centered on problems of management and culture. Only good news flows uphill and for all the famous Russian bureaucracy, the essence of systems engineering had not been internalized into the design, manufacturing, training, maintenance or deployment of military forces. The result is a “paperwork army”. A paperwork army Putin himself could have thought was far more capable than it actually was. Culture begats process which begats culture. But if only good news is rewarded, the very existence of a positive cycle is short-circuited.
We may be witnessing the result of exactly that in Ukraine.
In the longer-term, the impact of autocracy on quality – particularly of complex systems – may well end up undermining other autocratic geopolitical players.
One can, at least, hope this will be the case.
—
Joseph Cox lives in Modiin, Israel and has written 11 books. The latest is A Multi Colored Coat. It is funnier that this was 🙂
