There are two problems. One is "structural unemployment." Loosely, it means that the structure of the domestic economy does not fit with the economic realities, creating a large, permanent class of unemployed people unless something changes. In our case, we shifted from a manufacturing-based economy in the 1970s to a services based economy until about 2001, when we became a consumer based economy. That is to say, most of the jobs and economic growth in this country were sustained by consumers buying things. This does not mean we had eliminated all jobs in the service industries (by which I mean things like software support and develop or financial services, not just consumer services) or manufacturing, but the majority of actual economic activity was people buying things rather than, say, making things and exporting them to other countries to buy.
The problem is, (a) jobs in consumer sector do not, traditionally, pay as well as manufacturing jobs or skilled service jobs; (b) this trick only works as long as people can afford to buy things.
Meanwhile, we developed other problems. Our deteriorating infrastructure being a biggy. In theory, if we were willing to either tax ourselves we could have created more jobs in construction and maintenance or in public transportation. But we didn't. So deteriorating infrastructure is another cost on economic production as a whole.
The second problem is how do you get out of structural unemployment. There are, as far as I can tell, three ways:
1. Massive government investment designed to actually change the nature of the economy. This works on a theory that what we have is a mismatch between our current supply of workers and capacity and overall global demand. So you try to predict what is useful and where -- in a very general way -- you think the future goes. For example, if you think future economic activity will depend heavily on broadband infrastructure, you might want to invest in building out fiber networks, either directly by having the government build them or stimulate construction indirectly by providing subsidies/tax incentives (which are the same thing from an economic perspective).
The risk of this, of course, is that if you guessed wrong you will have spent a lot of money for no good reason. You also may spend the money inefficiently. But certain forms of spending can be reasonably predicted to yield economic returns for the economy as a whole. It is also important to distinguish between certain forms of protectionism, i.e., propping up inefficient structures and rewarding bad behavior, which do not do any significant benefit and may cause harm. Agricultural subsidies that encourage us to waste resources and grow food inefficiently are one example. There may be other reasons to do such things (e.g., for national security, or to preserve salaries so that consumers can buy goods and services), but they don't solve the structural problem.
2. Have the private sector invent a whole new sector of the economy. This works particularly well in combination with #1. For example, the U.S. had a general industrial policy in investing in communications research for military purposes (via DARPA) and for civilian purposes (NSF). This combination of modest investment (including construction of the first backbone, NSFNet) ultimately produced the commercial Internet, which generated several hundred billions of dollars in economic activity in the 1990s that no economist could have predicted.
As an aside, this is why we ought to favor pretty widespread government research for relatively modest amounts (the total federal investment in the development of the internet was fairly small overall, although I don't have the exact number). Think of it as having our own venture capital fund, but with the benefit accruing (at least in part) to the economy as a whole.
The risk here is that it is entirely unpredicatable. There are a number of things that could be the next "big thing" and create a new sector of the economy. But none of them have broken through.
3. Go through the traditional boom-bust cycle. The problem with that is it is extremely painful for those caught up in it. Argentina used to be the leading economy in South America until their economy collapsed. At its height, the official inflation rate stood at 50%. Their economic recovery today is largely due to the growth of Brazil.
I should note that the Argentines (and much of South America, as well as a number of African nations that had their own economic collapses as a consequence, like Nigeria) followed the prescription being urged by those favoring "austerity" and deficit reduction as the primary means of economic growth. What Krugman calls the "pain caucus" is, in fact, about causing short and medium term economic pain. Economist who justified this for the developing world in the 1990s urged that it was a necessary pre-requisite for countries to move away from their state-dominated economies to an ultimately more prosperous market economy. And even here, the incentive to participate was IMF loans for infrastructure builds (contingent on austerity measures and reducing international debt, which could then only be paid by "attracting foreign capital," which could only be attracted with a combination of the sale of government assets and legal reforms designed to attract foreign investment). Because you can't have economic activity without some form of infrastructure to distribute goods and services.
Anyway, to make a long story short, the Obama administration tried approaches #1 and #2 in a rather half-hearted way in 2009. The stimulus bill was in large part about trying to restructure the economy as well as about helping state governments pay workers, extend unemployment, fix existing infrastructure, and other Keynesian "pump-priming" measures. So was the GM bailout, which included a requirement for serious structural changes in the auto manufacturing sector. As we all know, these efforts -- modest to begin with in the overall scheme of things -- came under fierce attack as detrimental to the economy on the grounds that they (a) increased the deficit, and (b) the United States should not have an industrial policy, because it is bad when government "picks winners" in the market. As a consequence, these efforts have been curtailed and we have instead a focus on austerity (but with no raise in revenue).
Unsurprisingly, we return now to the interrupted bust part of the cycle. Perhaps, if free market economists are correct, it is doomed to be that way and we are better off getting it over with quickly so we can rebuild post collapse. But I don't think that's what people are expecting, or policians promising, when they go down the austerity road.