Edward: I think I DO GET IT!
The Obama Administration believes that we "simply must transition away from carbon based fuels" as you chartacterize it. The PROBLEM is the public does not and is not willing to bankrupt themselves trying to live into political correctness.
I'm not giving lip service to GHG emissions reductions. I am celebrating the 40% reduction we are getting as a result of the displacement of coal by natural gas. Low gas prices have proven 10 times more effective at achieving GHG emissions reduction than the onerous EPA regulations on coal-----let it happen! And if it does happen on the President's watch he can take credit for it.
But, as you suggest, gas prices will not always be low so enjoy this while it lasts. As E&P firms shift focus from gas to NGL and Oil gas oversupply will moderate to equilibrium and prices will naturally rise to market clearing levels. But gas prices can go up to $5-6 before coal will begin again be competitive and by then the market share of coal and gas will have shifted to less coal and more gas.
Fossil fuels are not going away anytime soon so a policy of making war on coal is pure politics and should be named as such.
You will never win the argument for wind and solar on the basis of exigent (externality) costs. There is a place for both wind and solar in our supply mix and we should encourage them on a least cost, best fit basis. BUT---they must compete based upon their LCOE for a place in that supply stack just like every other technology without stacking the deck in a zero sum game for some technologies and against others.
You forget that fossil fuels were the winners in the competition for incremental capacity additions because for decades each additional power plant added reduced the marginal cost of energy. It was only beginning in the 1970's with high inflation and out of control capital costs for the first generation of nuclear plants that the marginal cost of new plants were higher not lower than the average embedded cost. Since then we have used a "least cost, best fit" regulatory standard forcing supply side options to compete side by side with demand side options for a place in the competitive supply stack.
The advent of renewable portfolio standards and the tax subsidies for renewable energy hijacked that integrated resource planning standard that serves us well. By mandating the % of renewable energy regardless of cost, making renewable energy "must take" ---at least in California, and using tax subsidies to change their place in the LCOE price competition we have jury rigged the system to pick winners.