Instead, let me write a couple of words about the timescales that theoretical physicists need to resolve some of their greatest puzzles because his work is an interesting example how things can often work out.
Revolutions and normal science
Theoretical physicists have known for many centuries that conceptual breakthroughs are among the most valuable outcomes of their work and they have been trying to find out such breakthroughs, together with their colleagues crackpots (who are separated by a fuzzy wall from the physicists). There have been periods in which the progress in physics was extremely fast; there have been periods in which the progress was slower.
During Newton's life, a huge amount of old and new facts about mechanics was unified under a universal theoretical umbrella. Similar revolutions occured in the second half of the 19th century - with electromagnetism - and in the late 1920s - with quantum mechanics. People were still learning the basic laws that are needed to understand ordinary phenomena we observe every day - and it more or less continued to be the case in the Standard Model revolution of the 1960s. This is why the progress in the four revolutions mentioned above - among others - depended on both theoretical as well as experimental results and their intense interactions.
Theoretical revolutions
There have also been other revolutions in which the progress was mostly theoretical because the research was focusing on profound aspects of our Universe that are, however, less important for everyday life. Einstein's discoveries of both special and general relativity and most of the results of string theory in the last 20-30 years are good examples.
Needless to say, physicists spend most of their lives in the state of confusion and physics as a field works on "normal science" throughout most of the human history, if you allow me to use Kuhn's terminology. Physicists and scientists in general often work on projects that turn out to be dead ends. Sometimes they work on theories that turn out to be wrong and all their perceived advantages evaporate once a better theory is found. Sometimes they try to find an exact solution of a problem that is eventually revealed to have no exact solution or at least no simple exact solution.
Re-adjusting hopes and values
As time goes by, physicists must make strategic decisions whether the research of a particular question or a set of ideas continues to be promising or meaningful. There is no rigorous algorithm how such decisions can be made. They must be able to decide whether the expected time and efforts needed to crack a certain question is a good investment given the likelihood that it will work; whether a given idea or a hypothetical result is valuable enough. It should be obvious that they are never quite sure whether it is the case. If you work on a certain theory and some details either don't work out right or you don't know how to make the next step, you're never sure whether this state of affairs has deep fundamental reasons or whether it can change by tomorrow. It seems clear that whatever our criteria are, the perceived confidence that we are on the right track with a particular idea should decrease with time in the case that the idea keeps on failing, but no one can calculate how quickly it should decrease.
String field theory - an example
Let me use string field theory as my main example. Physicists have known for two decades that the perturbative definition of string theory is not enough to understand various subtle issues that are important for the comparison with experiments - or for internal conceptual questions of the theory such as the vacuum selection problem. In gauge theories such as QCD, we may put the fundamental Lagrangian on a lattice and our computer may give us results whose range of validity exceeds the perturbative realm.
The most obvious counterpart of such an approach turned out to be string field theory. Indeed, it was shown that it could reproduce the perturbative scattering amplitudes. Moreover, its formulation looked like a generalization of the QCD Lagrangian, in a sense, and one could have imagined that string field theory could become a non-perturbative definition of string theory; an approach to string theory that would be taught as the most fundamental approach in 30 years.
These hopes turned out to be flawed in various ways: string field theory for closed strings (and superstrings) is much less well-defined and needs infinitely many terms in the Lagrangian. Strings only look like "elementary" objects at weak coupling because they're the lightest objects, but at stronger coupling, a new kind of democracy between the strings and other objects such as branes emerges. Any approach based on strings only contradicts this principle of democracy and is disfavored, especially for the questions about the strongly coupled physics for which string field theory was thought to be relevant or even useful.
String field theory vs. the duality revolution
In the mid 1990s, it was already widely appreciated that string field theory is probably going to be just a different language to understand perturbative physics of string theory - a language that has the advantage that it allows us to calculate off-shell physics, a counterpart of Green's functions instead of just the on-shell scattering amplitudes. Tachyon condensation - as included in Sen's description of D-brane annihilation in terms of a condensation of the tachyon field - became the primary arena on which string field theory can show its muscles.
If we focus on cubic string field theory, there have been many successful numerical calculations that showed that Sen's conjectures are valid with a very good accuracy. And there have been many attempts to prove the conjectures analytically by:
- finding the exact solution to the cubic string field theory that corresponds to a complete destruction of the D-brane on which the open strings can terminate;
- proving that there are more general "lump" solutions that represent lower-dimensional branes as left-overs of the annihilation process;
- proving that the energy density of the solution agrees with the tension of the D-brane that has disappeared or the other branes that survived;
- demonstrating that no physical linearized excitations around the vacuum solution exist.
Some of the attempts were very formal and could not survive the translation to the well-defined language of creation operators that represent excitations of the single-string ground state. Some of the attempts had some problems with singularities at the midpoint. Around 2000, a new wave of interest in string field theory started. Many people jumped on this field because of a combination of two reasons:
- the bandwagon effect - the desire to work on something that many other people care about;
- legitimate new techniques and results that have been revealed - wedge states, sliver states, and other things.
However, the tasks had not been solved in the 2000 wave although many relevant tools have been proposed in literature. String field theory again became a nearly-dead field around 2003 or so. Surprisingly, the exact solution that allows one to attack all these questions about tachyon condensation analytically has been found at the end of 2005 by Martin Schnabl.
Patience and evaluation of hopes
Although the exact solution (so far) does not really answer any of the "big open questions" we have about string theory, it is a very cute and robust mathematical result and an example that patience is sometimes needed to see the answers to the questions that many people ask.
His solution uses the tools that were thought to be promising - but it is using them more efficiently. On the other hand, it also resembles some approaches from the past that turned out to be unsuccessful - such as the approaches that assign a special role to the midpoint. How is it possible that these unsuccessful approaches were suddenly "improved" so that the big questions of this small subfield could have been nailed down completely?
Well, there has never been a theorem that "similar" methods can never work. One of the problems with the unsuccessful methods was that the papers that used tthese methods were too similar to each other. They were not making any real progress. Martin's main line of reasoning was not really an attempt to make these methods work. He was solving much more well-defined problems; it just happened that the final solution turned out to be "morally similar" to some of the methods that did not work in the past.
Such developments often occur in theoretical physics. Athough we may often be failing for many years to solve a certain question using certain methods, it is still completely possible that the question will be fully answered by very similar methods in which some subtle technical details are corrected.
On the other hand, there are also many situations in which such a hope is unjustifiable. Some people may be using methods that are so flawed or hopeless that a failure that continues for several years or decades is not a sign of bad luck but rather evidence of a very inherent problem with the whole line of reasoning.
SFT vs. LQG
Let's compare cubic string field theory and loop quantum gravity. One may like or dislike each of them because of many different reasons but when one looks at these theories technically, she sees that while loop quantum gravity cannot generate any mathematically nice results that would go beyond 20 pages on which the theory can be explained, despite decades of efforts, string field theory naturally leads to a rigorous incorporation of very many mathematically attractive structures - such as conformal field theory, conformal transformations, Bernoulli numbers, Euler-Ramanujan formula, noncommutative geometry, and many others. There are good reasons to think that if some details don't work out right in string field theory for several years, it may be just a temporary problem. This hope could not work in loop quantum gravity.
The main goals of string theory are of course much more ambitious than the exact solution for the tachyon condensation on a D25-brane. We are searching for the theory of everything. Many of us are still convinced that there are good reasons to believe that such a theory will be able to calculate all physically relevant things, including the couplings and particle masses.
50 years?
If it took 20 years from the birth of cubic string theory - and 5 years from the invention of the directly relevant tools - to find the exact solution for the closed string vacuum, how many years do we need to complete string theory to the extent that most reasonable people will have to agree that it is the established theory of everything?
We can never rule out the possibility that a subtle breakthrough will occur next week, and it will take several months for people to complete the details and compute the masses of all leptons, for example. Less optimistically, we may need 50 years or more and new hints from the experiments to make progress. No one knows and the precise figure will depend on many things, including good luck.
One thing is clear: we have many more indications that we are on the right track than what we had in the process of answering the purely mathematical problem of the solution to the cubic string field theory. As long as the quest for a deeper understanding of physical reality continues, we must try to fucus on the ideas that look most promising. In the broader scheme of things, the place to look at is undoubtedly string theory.
