‘Physics must be rebuilt’
 

Quantum theory, with its paradoxes and uncertainties, its mysteries and challenges to intuition, is something of a refuge for scoundrels and charlatans, as well as a fount of more serious but nonetheless fantastic speculation. Could it explain Consciousness? Does it undermine causality? Everything from homeopathy to mind control and manifestations of the paranormal has been laid at its seemingly tolerant door.

Mostly that represents a blend of wishful thinking, misconception and pseudoscience. Because quantum theory defies common sense and ‘rational’ expectation, it can easily be hijacked to justify almost any wild idea. The extracurricular uses to which quantum theory has been put tend inevitably to reflect the preoccupations of the times: in the 1970s parallels were drawn with Zen Buddhism, today alternative medicine and theories of mind are in vogue.

Nevertheless, the fact remains that fundamental aspects of quantum physics are still not fully understood, and it has genuinely profound philosophical implications. Many of these aspects were evident to the early pioneers of the field – indeed, in the transformation of scientific thought that quantum theory compelled, they were impossible to ignore. Yet while several of the theory’s persistent conundrums were identified in its early days, one can’t say that the physicists greatly distinguished themselves in their response. This is hardly sur prising: neither scientists nor philosophers in the early twentieth century had any preparation for think ing in the way quantum physics demands, and if the physicists tended to retreat into vagueness, near-tautology and mysticism, the philosophers and other intellectuals often just misunderstood the science.

This penchant for pondering the deeper meanings of quantum theory was particularily evident in Germany, proud of its long tradition of philosophical enquiry into nature and reality. The British, American and Italian physicists, in contrast, tended to conform to their stereotypical national pragmatism in dealing with quantum matters. But even if they were rather more content to apply the mathematics and not wonder too hard about the ontology, these other scientists relied strongly on the Germanic nations for those theoretical formulations in the first place. Germany, more than any other country, showed how to turn the microscopic fragmentation of nature into a useful, predictive, quantitative and explanatory science. If you were a theoretical physicist in Germany, it was hard to resist the gravitational pull of quantum theory: where Planck and Einstein led, Arnold Sommerfeld, Peter Debye, Werner Heisenberg, Max Born, Erwin Schrödinger, Wolfgang Pauli and others followed.

This being so, it was inevitable that the philosophical aspects of quantum physics should have been coloured by the political and social preoccupations of Germany. As we shall see, it was not the only part of physics to become politicized. These tendencies rocked the ivory tower: the kind of science you pursued became a statement about the sort of person you were, and the sympathies you harboured.

Unpeeling the atom

The realization that light and energy were granular had profound implications for the emerging understanding of how atoms are constituted. In 1907 New Zealander Ernest Rutherford, work ing at Manchester University in England, found that most of the mass of an atom is concentrated in a small, dense nucleus with a positive electrical charge. He concluded that this kernel was surrounded by a cloud of electrons, the particles found in 1897 to be the constituents of cathode rays by J. J. Thomson at Cambridge. Electrons possess a negative electrical charge that collectively balances the positive charge of the nucleus. In 1911 Rutherford proposed that the atom is like a solar system in miniature, a nuclear sun orbited by planetary electrons, held there not by gravity but by electrical attraction.

But there was a problem with that picture. According to classical physics, the orbiting electrons should radiate energy as electromagnetic rays, and so would gradually relinquish their orbits and spiral into the nucleus: the atom should rapidly decay. In 1913 the 28-year-old Danish physicist Niels Bohr showed that the notion of quantization – discreteness of energy – could solve this problem of atomic stability, and at the same time account for the way atoms absorb and emit radiation. The quantum hypothesis gave Bohr permission to prohibit instability by fiat: if the electron energ ies can only take discrete, quantized values, he said, then this gradual leakage of energ y is prevented: the particles remain orbiting indefinitely. Electrons can lose energy, but only by making a hop (‘quantum jump’) to an orbit of lower energy, shedding the difference in the form of a photon of a specific wavelength. By the same token, an electron can gain energy and jump to a higher orbit by absorbing a photon of the right wavelength. Bohr went on to postulate that each orbit can accommodate only a fixed number of electrons, so that downward jumps are impossible unless a vacancy arises.

It was well established experimentally that atoms do absorb and emit radiation at particular, well-defined wavelengths. Light passing through a gas has ‘missing wavelengths’ – a series of dark, narrow bands in the spectrum. The emission spectrum of the same vapour is made up of corresponding bright bands, accounting for example for the characteristic red glow of neon and the yellow glare of sodium vapour when they are stimulated by an electrical discharge. These photons absorbed or emitted, said Bohr, have energies precisely equal to the energy difference between two electron orbits.

By assuming that the orbits are each characterized by an integer ‘quantum number’ related to their energy, Bohr could rationalize the wavelengths of the emission lines of hydrogen. This idea was developed by Arnold Sommerfeld, professor of theoretical physics at the University of Munich. He and his student Peter Debye worked out why the spectral emission lines are split by a magnetic field – an effect discovered by the Dutch physicist Pieter Zeeman in work that won him the 1902 Nobel Prize. (This Zeeman effect is the magnetic equivalent of the line-splitting by an electric field discovered by the German physicist Johannes Stark – see page 88.)

But this was still a rather ad hoc picture, justified only because it seemed to work. What are the rules that govern the energy levels of electrons in atoms, and the jumps between them? In the early 1920s Max Born at the University of Göttingen set out to address those questions, assisted by his brilliant students Wolfgang Pauli, Pascual Jordan and Werner Heisenberg.

Heisenberg, another of Sommerfeld’s protégés, arrived from Munich in October 1922 to become Born’s private assistant, looking as Born put it ‘like a simple farm boy, with short fair hair, clear bright eyes, and a charming expression’. He and Born sought to apply Bohr’s empirical description of atoms in terms of quantum numbers to the case of helium, the second element in the periodic table after hydrogen. Given Bohr’s prescription for how quantum numbers dictate electron energies, one could in principle work out what the energies of the various electron orbits are, assuming that the electrons are held in place by their electrostatic attraction to the nucleus. But that works only for hydrogen, which has a single electron. With more than one electron in the frame, the mathematical elegance is destroyed by the repulsive electrostatic influence that electrons exert on each other. This is not a minor correction: the force between electrons is about as strong as that between electron and nucleus. So for any element aside from hydrogen, Bohr's appealing modelbecomes too complicated to work out exactly.

In trying to go beyond these limitations, however, Born was not content to fit experimental observations to improvised quantum hypotheses as Bohr had done. Rather, he wanted to calculate the disposition of the electrons using principles akin to those that Isaac Newton used to explain the gravitationally bound solar system. In other words, he sought the rules that governed the quantum states that Bohr had adduced.

It became clear to Born that what he began to call a ‘quantum mechanics’ could not be constructed by a minor amendment of classical, Newtonian mechanics. ‘One must probably introduce entirely new hypotheses’, Heisenberg wrote to Pauli – another former pupil of Sommerfeld in Munich, where the two had become friends – in early 1923. Born agreed, writing that summer that ‘not only new assumptions in the usual sense of physical hypotheses will be necessary, but the entire system of concepts of physics must be rebuilt from the ground up’.

That was a call for revolution, and the ‘new concepts’ that emerged over the next four years amounted to nothing less. Heisenberg began formulating quantum mechanics by writing the energ ies of the quantum states of an atom as a matrix, a kind of mathematical grid. One could specify, for example, a matrix for the positions of the electrons, and another for their momenta (mass times velocity). Heisenberg’s version of quantum theory, devised with Born and Jordan in 1925, became known as matrix mechanics.

It wasn’t the only way to set out the problem. From early 1926 the Austrian physicist Erwin Schrödinger, working at the University of Zurich, began to explicate a different form of quantum mechanics based not on matrices but on waves. Schrödinger postulated that all the fundamental properties of a quantum particle such as an electron, or a collection of such particles, can be expressed as an equation describing a wave, called a wavefunction. The obvious question was: a wave of what? The wave itself is a purely mathematical entity, incorporating ‘imaginary numbers’ derived from the square root of -1 (denoted i), which, as the name implies, cannot correspond to any observable quantity. But if one calculates the square of a wavefunction – that is, if one multiplies this mathematical entity by itself – (More strictly, one calculates the so-called complex conjugate, the product of two wave functions identical except that the imaginary parts have opposite signs: +i and -i.) – then the imaginary numbers go away and only real ones remain, which means that the result may correspond to something concrete that can be measured in the real world. At first Schrödinger thought that the square of the wavefunction produces a mathematical expression describing how the density of the corresponding particle varies from one place to another, rather as the density of air varies through space in a sound wave. That was already weird enough: it meant that quantum particles could be regarded as smeared-out waves, filling space like a gas. But Born – who, to Heisenberg’s dismay, was enthusiastic about Schrödinger’s rival ‘wave mechanics’ – argued that the squared wavefunction denoted something even odder: the probability of finding the particle at each location in space.

Think about that for a moment. Schrödinger was asserting that the wavefunction says all that can be said about a quantum system. And apparently, all that can be said is not where the particle is, but what the chance is of finding it here or there. This is not a question of incomplete knowledge – of knowing that a friend might be at the cinema or the restaurant, but not knowing which. In that case she is one place or another, and you are forced to talk of probabilities just because you lack sufficient information. Schrödinger’s wave-based quantum mechanics is different: it insists that there is no answer to the question beyond the probabilities. To ask where the particle really is has no physical meaning. At least, it doesn’t until you look – but that act of looking doesn’t then disclose what was previously hidden, it determines what was previously undecided.

Whereas Heisenberg’s matrix mechanics was a way of formalizing the quantum jumps that Bohr had introduced, Schrödinger’s wave mechanics seemed to do away with them entirely. The wavefunction made everything smooth and continuous again. At least, it seemed to. But wasn’t that just a piece of legerdemain? When an electron jumps from one atomic orbit to another, the initial and the final states are both described by wavefunctions. But how did one wavefunction change into the other? The theory didn’t specify that – you had to put it in by hand. And you still do: there remains no consensus about how to build quantum jumps into quantum theory. All the same, Schrödinger’s description has prevailed over Heisenberg’s – not because it is more correct, but because it is more convenient and useful. What’s more, Heisenberg’s quantum matrices were abstract, giving scant purchase to an intuitive understanding, while Schrödinger’s wave mechanics offered more sustenance to the imagination.

The probabilistic view of quantum mechanics is famously what disconcerted Einstein. His scepticism eventually isolated him from the evolution of quantum theory and left him unable to contribute further to it. He remained convinced that there was some deeper reality below the probabilities that would rescue the precise certainties of classical physics, restoring a time and a place for everything. This is how it has always been for quantum theory: those who make great, audacious advances prove unable to reconcile them to the still more audacious notions of the next generation. It seems that one’s ability to ‘suppose’ – ‘understanding’ quantum theory is largely a matter of reconciling ourselves to its counter-intuitive claims – is all too easily exhausted by the demands that the theory makes.

Schrödinger wasn’t alone in accepting and even advocating indeterminacy in the quantum realm. Heisenberg’s matrix mechanics seemed to insist on a very strange thing. If you multiply together the matrices describing the position and the momentum of a particle, you get a different result depending on which matrix you put first in the arithmetic. In the classical world the order of multiplication of two quantities is irrelevant: two times three is the same as three times two, and an object’s momentum is the same expressed as mass times velocity or velocity times mass. For some pairs of quantum properties, such as position and momentum, that was evidently no longer the case.

This might seem an inconsequential quirk. But Heisenberg discovered that it had the most bizarre corollary, as foreshadowed in the portentous title of the paper he published in March 1927: ‘On the perceptual content of quantum-theoretical kinematics and mechanics’. Here he showed that the theory insisted on the impossibility of knowing at any instant the precise position and momentum of a quantum particle. As he put it, ‘The more precisely we determine the position, the more imprecise is the determination of momentum in this instant, and vice versa.’

This is Heisenberg’s uncertainty principle. He sought to offer an intuitive rationalization of it, explaining that one cannot make a measurement on a tiny particle such as an electron without disturbing it in some way. If it were possible to see the particle in a microscope (in fact it is far too small), that would involve bouncing light off it. The more accurately you wish to locate its position, the shorter the wavelength of light you need (crudely speak ing, the finer the divisions of the ‘ruler’ need to be). But as the wavelength of photons gets shorter, their energy increases – that’s what Planck had said. And as the energy goes up, the more the particle recoils from the impact of a photon, and so the more you disturb its momentum.

This thought experiment is of some value for grasping the spirit of the uncertainty principle. But it has fostered the misconception that the uncertainty is a result of the limitations of experimentation: you can’t look without disturbing. The uncertainty is, however, more fundamental than that: again, it’s not that we can’t get at the information, but that this information does not exist. Heisenberg’s uncertainty principle has also become popularly interpreted as imputing fuzziness and imprecision to quantum mechanics. But that’s not quite right either. Rather, it places very precise limits on what we can know. Those limits, it transpires, are determined by Planck’s constant, which is so small that the uncertainty becomes significant only at the tiny scale of subatomic particles.

Political science

Both Schrödinger’s wavefunction and Heisenberg’s uncertainty principle seemed to be insisting on aspects of quantum theory that verged on the metaphysical. For one thing, they placed bounds on what is knowable. This appeared to throw causality itself – the bedrock of science – into question. Within the blurred margins of quantum phenomena, how can we know what is cause and what is effect? An electron could turn up here, or it could instead be there, with no apparent causal principle motivating those alternatives.

Moreover, the observer now intrudes ineluctably into the previously objective, mechanistic realm of physics. Science purports to pronounce on how the world works. But if the very act of observing it changes the outcome – for example, because it transforms the wavefunction from a probability distribution of situations into one particular situation, commonly called ‘collapsing’ the wavefunction – then how can one claim to speak about an objective world that exists before we look?

Today it is generally thought that quantum theory offers no obvious reason to doubt causality, at least at the level at which we can study the world, although the precise role of the observer is still being debated. But for the pioneers of quantum theory these questions were profoundly disturbing. Quantum theory worked as a mathematical description, but without any consensus about its interpretation, which seemed to be merely a matter of taste. Many physicists were content with the prescription devised between 1925 and 1927 by Bohr and Heisenberg, who visited the Dane in Copenhagen. Known now as the Copenhagen interpretation, this view of quantum physics demanded that centuries of classical preconceptions be abandoned in favour of a capitulation to the maths. At its most fundamental level, the physical world was unknowable and in some sense indeterminate. The only reality worthy of the description is what we can access experimentally – and that is all that quantum theory prescribes. To look for any deeper description of the world is meaningless. To Einstein and some others, this seemed to be surrendering to ignorance. Beneath the formal and united appearance of the Solvay group in 1927 lies a morass of contradictory and seemingly irreconcilable views.

These debates were not limited to the physicists. If even they did not fully understand quantum theory, how much scope there was then for confusion, distortion and misappropriation as they disseminated these ideas to the wider world. Much of the blame for this must be laid at the door of the scientists themselves, including Bohr and Heisenberg, who threw caution to the wind when generalizing the narrow meaning of the Copenhagen interpretation in their public pronouncements. For Bohr, a crucial part of this picture was the notion of complementarity, which holds that two apparently contradictory descriptions of a quantum system can both be valid under different observational circumstances. Thus a quantum entity, be it an insubstantial photon or an electron graced with mass, can behave at one time as a particle, at another as a wave. Bohr’s notion of complementarity is scarcely a scientific theory at all, but rather, another characteristic expression of the Copenhagen affirmation that ‘this is just how things are’: it is not that there is some deeper behaviour that sometimes looks ‘wave-like’ and sometimes ‘particle-like’, but rather, this duality is an intrinsic aspect of nature. However one feels about Bohr’s postulate, there was little justification for his enthusiastic extension of the complementarity principle to biology, law, ethics and religion. Such claims made quantum physics a political matter.

The same is true for Heisenberg’s insistence that, via the uncertainty principle, ‘the meaninglessness of the causal law is definitely proved’. He tried to persuade philosophers to come to terms with this abolition of determinism and causality, as though this had moreover been established not as an (apparent) corollary of quantum theory but as a general law of nature.

This quasi-mystical perspective on quantum theory that the physicists appeared to encourage was attuned to a growing rejection, during the Weimar era, of what were viewed as the maladies of materialism: commercialism, avarice and the encroachment of technology. Science in general, and physics in particular, were apt to suffer from association with these supposedly degenerate values, making it inferior in the eyes of many intellectuals to the noble aspirations of art and ‘higher culture’. While it would be too much to say that an emphasis on the metaphysical aspects of quantum mechanics was cultivated in order to rescue physics from such accusations, that desideratum was not overlooked. Historian Paul Forman has argued that the quantum physicists explicitly accommodated their inter pretations to the prevailing social ethos of the age, in which ‘the concept – or the mere word – “causality” symbolized all that was odious in the scientific enterprise’. In his 1918 book Der Untergang des Abendlandes (The Decline of the West), the German philosopher and historian Oswald Spengler more or less equated causality with physics, while making it a concept deserving of scorn and standing in opposition to life itself. Spengler saw in modern physicists’ doubts about causality a symptom of what he regarded as the moribund nature of science itself. Here he was thinking not of quantum theory, which was barely beginning to reach the public consciousness at the end of the First World War, but of the probabilistic microscopic theory of matter developed by the Scottish physicist James Clerk Maxwell and the Austrian Ludwig Boltzmann, which had already renounced claims to a precise, deterministic picture of atomic motions.

Spengler’s book was read and discussed throughout the German academic world. Einstein and Born knew it, as did many other of the leading physicists, and Forman believes that it fed the impulse to realign modern physics with the spirit of the age, leading theoretical physicists and applied mathematicians to ‘denigrat[e] the capacity of their discipline to attain true, or even valuable, knowledge’. They began to speak of science as an essentially spiritual enterprise, unconnected to the demands and depradations of technology but, as Wilhelm Wien put it, arising ‘solely from an inner need of the human spirit’. Even Einstein, who deplored the rejection of causality that he saw in many of his colleagues, emphasized the roles of feeling and intuition in science.

In this way the physicists were attempting to reclaim some of the prestige that science had lost to the neo-Romantic spirit of the times. Causality was a casualty. Only once we have ‘liberation from the rooted prejudice of absolute causality’, said Schrödinger in 1922, would the puzzles of atomic physics be conquered. Bohr even spoke of quantum theory having an ‘inherent irrationality’. And as Forman points out, many physicists seemed to accept these notions not with reluctance or pain but with relief and with the expectation that they would be welcomed by the public. He does not see in all this simply an attempt to ingratiate physics to a potentially hostile audience, but rather, an unconscious adaptation to the prevailing culture, made in good faith. When Einstein expressed his reservations about the trend in a 1932 interview with the Irish writer James Gardner Murphy, Murphy responded that even scientists surely ‘cannot escape the influence of the milieu in which they live’. And that milieu was anti-causal.

Equally, the fact that both quantum theory and relativity were seen to be provoking crises in physics was consistent with the widespread sense that crises pervaded Weimar culture – economically, politically, intellectually and spiritually. ‘The idea of such a crisis of culture’, said the French politician Pierre Viénot in 1931, ‘belongs today to the solid stock of the common habit of thought in Germany. It is a part of the German mentality.’ The applied mathematician Richard von Mises spoke of ‘the present crisis in mechanics’ in 1921; another mathematician, Hermann Weyl (one of the first scientists openly to question causality) claimed there was a ‘crisis in the foundations of mathematics’, and even Einstein wrote for a popular audience on ‘the present crisis in theoretical physics’ in 1922. (Experimental physicist Johannes Stark’s 1921 book The Present Crisis in German Physics used the same trope but spoke to a very different perception: that his kind of physics was being eclipsed by an abstract, degenerate form of theoretical physics – see page 91.) One has the impression that these crises were not causing much dismay, but rather, reassured physicists that they were in the same tumultuous flow as the rest of society.

This was, however, a dangerous game. Some outsiders drew the conclusion that quantum mechanics pronounced on free will, and it was only a matter of time before the new physics was being enlisted for political ends. Some even managed to claim that it vindicated the policies of the National Socialists.

Moreover, if physics was being in some sense shaped to propitiate Spenglerism, it risked seeming to endorse also Spengler’s central thesis of relativism: that not only art and literature but also science and mathematics are shaped by the culture in which they arise and are invalid and indeed all but incomprehensible outside that culture. It is tempting to find here a presentiment of the ‘Aryan physics’ propagated by Nazi sympathizers in the 1930s (see Chapter 6), which contrasted healthy Germanic science with decadent, self-serving Jewish science. And given Spengler’s nationalism, rejection of Weimar liberalism, support for authoritarianism and belief in historical destiny, it is no surprise that he was initially lauded by the Nazis, especially Joseph Goebbels, nor that he voted for Hitler in 1932. (Spengler was too much of an intellectual for his advocacy to survive close contact. After meeting Hitler in 1933, he distanced himself from the Nazis’ vulgar posturing and anti-Semitism, and was no favourite of the Reich by the time he died in Munich in 1936.)

One way or another, then, by the 1920s physics was becoming freighted with political implications. Without intending it, the physicists themselves had encouraged this. But they hadn’t grasped – were perhaps unable to grasp – what it would soon imply.