The big questions are hard to answer. But Mary C. Daly, Bart Hobijn, and Joseph H. Pedtke set the stage for a more insightful discussion in their short essay, "Disappointing Facts about the Black-White Wage Gap," written as an "Economic Letter" for the Federal Reserve Bank of San Francisco (September 5, 2017, 2017-26). Here are a couple of figures showing the black-white wage gap, and then seeking to explain what share of that gap is associated with differences in state of residence, education, part-time work, industry/occupation, and age. The first figure shows the wage gap for black and white men; the second for black and white women.
Here are some thoughts on these patterns:
1) The black-white wage gap is considerably larger for men (about 25%) than for women (about 15%). Also, the wage gaps seem to have risen since the 1980s.
2) The three biggest factors associated with the wage gap seem to be education level, industry/occupation, and "unexplained."
3) The "unexplained" share is rising over time time. As the authors explain: "Perhaps more troubling is the fact that the growth in this unexplained portion accounts for almost all of the growth in the gaps over time. For example, in 1979 about 8 percentage points of the earnings gap for men was unexplained by readily measurable factors, accounting for over a third of the gap. By 2016, this portion had risen to almost 13 percentage points, just under half of the total earnings gap. A similar pattern holds for black women, who saw the gaps between their wages and those of their white counterparts more than triple over this time to 18 percentage points in 2016, largely due to factors outside of our model. This implies that factors that are harder to measure—such as discrimination, differences in school quality, or differences in career opportunities—are likely to be playing a role in the persistence and widening of these gaps over time." The authors also cite this more detailed research paper with similar findings.
4) In looking at the black-white wage gap for women, it's quite striking that this gap was relatively small back in the 1980s, at only about 5%, and that observable factors like education and industry/occupation explained more than 100% of the wage gap at the time. But as the black-white wage gap for women increased starting in the 1990s, an "unexplained" gap opens up.
5) It is tempting to treat the "unexplained" category as an imperfect but meaningful measure of racial discrimination, but it's wise to be quite cautious about such an interpretation. On one side, the "unexplained" category may overstate discrimination, because it doesn't include other possible variables that affect wages (for example, one could include previous years of lifetime work experience, or length of tenure at a current job, scores on standardized tests, or many other variables). In addition, the variables that are included like level of education are being measured in broad terms, and so it is possible that, say, a blacks and whites with a college education are not the same in their skills and background. On the other side, the "unexplained" category could easily understate the level of discrimination. After all, education levels and industry/occupation outcomes don't happen in a vacuum, but are a result of the income, education, and jobs of family members. For this reason, noting that a wage gap is associated with some different in education or industry/occupation may reflect aspects of social discrimination. The kinds of calculations presented here are useful, but they don't offer final answers.
In short, the black-white wage gap is rising, not falling. The wage gap is also less associated with basic measures like level of education or industry/occupation than it was before. I can hypothesize a number of explanations for this pattern, but none of my hypotheses are cheerful ones.