“Your whole life as an astronomer, you learn that the universe is expanding, but it should be slowing down,” said Tod Lauer, an associate astronomer at the National Optical Astronomy Observatory who is investigating dark energy. “But we find out it’s speeding up. That’s the most incredible shock we’ve had in cosmology in the last 40 years.”

Gary Hinshaw, an astrophysicist at the NASA Goddard Space Flight Center, went even further. “It’s the biggest surprise in cosmology in 100 years,” he said.

Figuring out the nature of dark matter and dark energy is now the biggest question in the universe. NASA and the Department of Energy plan to spend billions of dollars to find the answers on Earth and in space. They’ll use giant particle colliders and new telescopes that will look at exploding stars, ancient sound waves and gravity-induced distortions of light.

Dark matter’s existence was predicted in the 1930s, when the astronomer Fritz Zwicky focused on a clump of galaxies called the Coma cluster. Based on the amount of light coming from the cluster, which is one way of estimating how much mass and gravitational force it had, one would have expected the galaxies on the edge of the cluster to move more slowly than those on the inside. But the galaxies on the edge were moving much faster.

Other astronomers later found the same phenomenon on a smaller scale. Stars on the edge of galaxies were orbiting much faster than would be expected based on the mass astronomers could see.

With dark matter’s existence now generally accepted, scientists have turned to the problem of finding out what it is. Black holes and stars and large planets too dim to be seen could account for some of it. But physicists say the answer probably lies with a subatomic particle or with particles we haven’t yet discovered.

“Dark matter is material that seems to interact by gravity but, as best as we can tell, does not emit any kind of light,” said Richard Mushotzky, a senior scientist at NASA Goddard.

The astronomer Edwin Hubble discovered decades ago that other galaxies seemed to be moving away from us in the Milky Way. The farther away they were, the faster they were going. But astrophysicists figured that gravity would cause the universe’s expansion to gradually slow down.

Then in 1998, two teams upended that theory by looking at Type Ia supernovae, rare stellar explosions that briefly release tremendous, consistent amounts of light. By carefully measuring how the light from these events shifted toward the red end of the visible light spectrum, they found that the expansion had accelerated in the last few billion years.

Albert Einstein hinted at the existence of a mysterious force like this when he proposed the concept of a cosmological constant. After Hubble made his observations, Einstein repudiated the constant as the greatest blunder of his career. Now it looks as if he spoke too soon.

But cosmologists admit they have no clue what dark energy really is. It could be Einstein’s constant, or a form of anti-gravity that changes over time, or evidence for string theory, the idea that the universe is composed of tiny strings and dimensions we can’t see.

Finding the answer has become one of NASA’s top priorities. The Joint Dark Energy Mission, an effort involving NASA and the Energy Department, seeks to send a new telescope into space tailored to find out more about dark energy. So far, three candidates have emerged: the Dark Energy Space Telescope, or Destiny; the Advanced Dark Energy Physics Telescope, known as ADEPT; and the SuperNova Acceleration Probe, or SNAP.

All would detect thousands of Type Ia supernovae. ADEPT would measure ancient sound waves that rumbled through the early universe, leaving tiny variations in how galaxies are distributed. SNAP and Destiny would use gravitational lensing to measure how the distribution of matter across the universe could be affected by dark energy.

Charles Bennett, a professor of physics and astronomy at Johns Hopkins University who is leading the ADEPT team, said the mission would probably be selected by the middle of next year, and the telescope would be in space a decade from now.

For Saul Perlmutter, a University of California at Berkeley cosmologist who led one of the teams that discovered dark energy and is now leading the SNAP effort, a final answer is tantalisingly near, but it was finding the question that was the real joy. “We’ve gotten amazingly close to what we feel is a full picture,” he said. Dark energy, he said, was “the little crack in the blind that gives us some insight into what we’ve never thought before”.