understanding the mind V: color vision

Why do we use words like white, black, red, yellow, etc. to describe human skin color when the related skin tones aren't "really" white, black, red, etc.?

There are four types of light sensitive cells in the retina: the rods and the S, M, and L cones (for, roughly, "short," "medium," and "long" wavelengths). These cells contain a molecule which changes shape when hit with a baseline level of photons, but each are sensitive to light waves of different wavelengths.
E. Bruce Goldstein, Sensation and Perception, 7th edition, 2006

The S, M, and L cones interact to produce color vision. We can test the degree of sensitivity to each type of cell at a variety of different wavelengths to determine it's sensitivity profile.

Brian A. Wandell, Foundations of Vision, 1985

These three types of color cell are "wired" into two opponent color circuits, the red-green and the blue-yellow circuits.

E. Bruce Goldstein, Sensation and Perception, 7th edition, 2006

This process separates the highly correlated M and L cone signals in order to provide a richer color space. A consequence of the wiring from three wavelength detectors to two opponent color circuits is a circular color space, familiar to many as the color wheel. When we graph this color circle against the dimension of brightness, we get a spindle shaped space corresponding to the subjective perception of color.

Paul M. Churchland, The Engine of Reason, The Seat of the Soul, 1996

Peter Gärdenfors has observed that if we consider the spindle shaped subspace of this color space which corresponds to possible human skin tones and attach our basic color words to the corresponding parts of this subspace, we can retrieve the use of these terms in describing human skin tone.

Peter Gärdenfors, Conceptual Spaces: The Geometry of Thought, 2004

Here we have an example of a linguistic structure, an analogy, which is suggested, perhaps even forced, by the physiological structure of human perception. . . . and how many more also are?