In 1998 the universe was found to be falling apart faster

On a crisp autumn night in late 1997, at the Cerro Tololo Inter-American Observatory nestled in the Chilean Andes, two rival teams of astronomers trained their sights on the vast expanse of the universe. The four-metre Blanco telescope served as their tool of choice, capturing images of distant galaxies in a quest to discover and analyse type Ia supernovae. These supernovae, the violent deaths of white dwarfs, are considered 'standard candles'. By knowing their intrinsic brightness and comparing it with their apparent brightness as seen from Earth, astronomers can calculate their distance. This was the task of the Supernova Cosmology Project led by Saul Perlmutter at Berkeley, and the High-Z Supernova Search Team, helmed by Brian Schmidt at Mount Stromlo in Australia and Adam Riess at Berkeley. Both teams were in a race against time and each other to measure the deceleration of cosmic expansion. After all, the universe had been expanding ever since the Big Bang, and it was widely assumed that the gravitational pull of its matter was slowing this expansion. By mid-1998, however, the answer they found was startling: the universe's expansion was not decelerating—it was accelerating.

Why anyone expected deceleration

To understand why the discovery of the universe's acceleration was so shocking, we must travel back to the early 20th century, specifically to the 1920s when Edwin Hubble observed that distant galaxies were moving away from us, heralding the discovery of cosmic expansion. This observation was made possible by Henrietta Leavitt's period-luminosity relation, which had allowed astronomers to measure distances to far-off celestial objects with unprecedented accuracy. By the 1980s, Hubble's discovery had cemented itself as a cornerstone of cosmology, leading to the conclusion that the universe had originated from a hot, dense state. This was further corroborated by the discovery of the cosmic microwave background radiation in 1965, providing a snapshot of that primordial heat.

As the 20th century progressed, cosmologists were deeply engrossed in the question of the universe's future. Gravity, the fundamental force binding matter, was expected to eventually slow down the cosmic expansion. The debate revolved around whether the universe contained enough matter to reverse the expansion, leading to a 'closed' universe scenario, or if it would expand indefinitely in an 'open' universe model. Both scenarios presupposed a deceleration of expansion. The teams observing supernovae in the late 1990s aimed to resolve this debate by measuring just how much the expansion was slowing. Yet, instead of providing answers, their data opened a new set of questions.

What the supernovae showed

When the results came in, both teams, despite their independent methodologies and data sets, reached the same startling conclusion. The distant type Ia supernovae, which were at redshifts between 0.5 and 0.8, appeared dimmer than would be expected in a universe slowing its expansion. They were consistent with a model in which the universe's expansion had been slower in the past and had since accelerated. This meant that the cosmic expansion was not just continuing but was increasing in pace. Presentations at scientific meetings in 1997 and 1998, followed by the publication of two seminal papers—Riess et al. in the *Astronomical Journal* in September 1998 and Perlmutter et al. in the *Astrophysical Journal* in June 1999—confirmed this finding beyond doubt.

The implications were profound. The combined data from both teams could not be explained by any cosmological model that included only matter and radiation. Something else, an unknown force, was driving the expansion. This mysterious component was christened 'dark energy', and its influence was calculated to be roughly twice that of all known matter in the universe. Yet, this discovery left the scientific community with more questions than answers. What exactly was dark energy? As of 2026, it remains one of the most significant enigmas in modern physics.

How the cosmological constant came back

The concept of dark energy revived an almost forgotten element from the early days of theoretical physics: Einstein's cosmological constant, denoted by the Greek letter Λ. In the 1910s, while developing his theory of general relativity, Einstein introduced this constant to allow for a static universe, which was the prevailing belief at the time. However, once Hubble's observations confirmed an expanding universe in 1929, Einstein deemed the constant unnecessary and allegedly referred to its introduction as his 'greatest blunder'.

The findings from the supernova studies in the late 1990s breathed new life into the concept. A cosmological constant effectively describes a constant energy density inherent to the vacuum of space itself, causing space to expand at an accelerating rate. This notion of a 'vacuum energy' fits the supernova data closely, aligning with the observations to within the current measurement precision. Thus, the cosmological constant, once considered obsolete, re-emerged with a new physical interpretation, suggesting that Einstein's discarded term might indeed hold the key to understanding the accelerating universe.

Why this is a problem

The cosmological constant provides a mathematical explanation for the accelerating universe, but it simultaneously presents a profound theoretical challenge. In quantum field theory, the vacuum is not empty but is filled with zero-point fluctuations of all the quantum fields. Calculating the energy density of these fluctuations yields a value that is approximately 10^120 times greater than what is observed. This discrepancy is frequently cited as the 'worst prediction in physics', illustrating the vast gap between theoretical calculations and observational reality.

Furthermore, the smallness of the cosmological constant's observed value is critical for the universe's current state. It is just right to permit the formation of cosmic structures like galaxies, stars, and planets. If the value were larger, it could have prevented galaxies from forming, while a much greater value would have hindered any structure formation at all. This raises the question of why the constant is so finely tuned to allow life as we know it to exist. Several hypotheses have been proposed to address this conundrum, including quintessence, which suggests a slowly-changing scalar field, and the multiverse theory, which posits that the constant varies across different universe regions, with our universe having a value suitable for life. Yet, these theories lack definitive, testable predictions, leaving us with an empirical constant that defies theoretical understanding.

The 2011 Nobel and what has been done since

In recognition of their groundbreaking observations, Saul Perlmutter, Brian Schmidt, and Adam Riess were awarded the 2011 Nobel Prize in Physics for their discovery of the accelerating expansion of the universe through distant supernovae. This accolade celebrated the observational achievement, while the interpretation of dark energy remained a topic of intense research and debate.

Since the initial findings, efforts to understand dark energy have intensified. The Dark Energy Survey (DES), which operated from 2013 to 2019 using the same Blanco telescope at Cerro Tololo, has provided more precise characterisation of dark energy, confirming consistency with a cosmological constant to within a few percent. Other approaches, like baryon acoustic oscillation measurements from the Sloan Digital Sky Survey, have independently supported these conclusions. The Planck satellite has also contributed significantly by measuring the cosmic microwave background. When combined with other data, such as from supernovae and baryon acoustic oscillations, Planck's observations suggest that dark energy accounts for about 68 percent of the universe's total energy content. However, despite these advances, the interpretation of dark energy remains as elusive as ever.

What the next decade might do

Looking forward into the 2020s, several significant astronomical facilities are poised to enhance our understanding of dark energy. The Vera C. Rubin Observatory, operational from 2025, promises to revolutionise our view of the sky by producing a continuous survey, which will identify thousands of type Ia supernovae each year. These observations are expected to refine our measurements of the universe's expansion with unprecedented precision.

Meanwhile, the European Space Agency's Euclid space telescope, launched in 2023, is mapping the universe's geometry up to a redshift of 2 through techniques such as weak gravitational lensing and galaxy clustering. The forthcoming Nancy Grace Roman Space Telescope, set for launch by NASA in 2027, will extend supernova observations to higher redshifts, reducing systematic uncertainties. By the early 2030s, these combined efforts will allow us to determine dark energy's equation of state with a precision well below 1 percent. Should the data continue to align with a cosmological constant, the theoretical challenges deepen; however, even slight deviations might herald new physics. While the empirical landscape is set to improve markedly, the theoretical journey depends heavily on the unfolding data.

The discovery in 1998 that the universe's expansion was accelerating rather than slowing marked a pivotal moment in our understanding of the cosmos. Over two decades later, the cause of this acceleration—accounting for approximately 68 percent of the universe's total energy budget—remains the largest unexplained observation in physics. This mysterious force, dubbed dark energy, dictates not only the current state but also the ultimate fate of the universe. If its density remains constant, the cosmic expansion will perpetuate indefinitely, leading to a future where galaxies drift beyond the cosmic horizon, irretrievably lost to us. If the energy density increases, the expansion could tear apart galaxies, stars, and even atoms—a scenario known as the 'Big Rip'. Conversely, a decrease could one day result in a recollapse of the universe. The precise nature of dark energy continues to evade explanation, underscoring the profound mysteries yet to be unraveled in our quest to comprehend the cosmos.