While true, I never worried about it much. I know that we have a strong tradition for the 6,000 year thing for the beginning of human history, with a tie in that the world ends after 6,000 years (although not only does this require a bunch of leaps of supposition, the same verse cited in Psalms for this makes equal sense for interpretations of 24,000 years or 48,000 years until the world expires). My brief response is: Torah is not a history book, or a science book. Nor is it a standard cosmology.
It's a cook book. :-)