Ok, the color you use doesn't matter. The end.
What's that, you want more... ok, here we go...
As discussed previously, an object's reflective properties determine what wavelengths of light it reflects back, which also determines how much overall light is reflected. In layman's terms we can say simply that a red object reflects red light. As a side note, it's important to realize that just about every object in the real world reflects back a little bit of every color. That is to say, even the reddest red apple is not really all that red!
What color does a white object reflect then? All of them (same with black and gray, but we'll deal with them later). As such, white objects and surfaces tell us a lot about the lighting conditions in the given area of a painting (or in the real world, for that matter). So a white sheet in bright sunlight appears very nearly white; in shadow on a sunny day it may almost appear to match the blue sky (because only the ambient light of the sky reaches the shadow); if a red apple is sitting on the sheet then you'll see some reddish colors in the apple's shadow, and so on.
That's how it all really works, but in painting "reality" is only part of the story... Because value is soooo overwhelmingly important in visual perception, the fact is, the colors you choose to use in painting white don't matter all that much, in terms of potentially appearing to be "wrong." The question is, what do you want to do with color in your picture? The way white is handled in a picture tells what the artist is doing, because all other color factors have been filtered out, so to speak.
Sometimes there is a clear logic behind the handling of white, for example, here the warm light of the fire hits the white surfaces facing it, giving them a yellowish color, while those facing the other way (ostensibly lit by the purply sky) are cooler:
In this picture there is no such logic - the "shaded" areas of the white shirt are colored yellow simply so they harmonize with the strong yellow background. Many times it works great to simply paint your whites in colors pulled from the rest of the picture. Maybe that has the same sort of effect as "ambient light"...
In practice artists tend to move freely between logical color, abstract color, and color for design. The good news is, if you have your value organizations right, you have a tremendous amount of flexibility in how you handle color, without fear of the picture breaking apart. This is true of all colors, but as noted, observing how white is handled in a picture really exposes the underlying color approach. Check out all the "whites" in this picture:
Sure the warmer colors are allegedly coming from the lamplight, and the cooler colors from the sky, but as you can see I am able to move quite freely between blue, purple, red, pink, yellow, orange - whatever I want to make the kind of impact I want - without worrying at all about the picture's color becoming incohesive (spell checker tells me that's not a word, but I like it). Here's a selection of these "whites" from the snow scene, in isolation:
It can be really stunning to see that, basically any color can appear as white. Here are some notable examples of "whites" from a few of my pictures - a very wide range of hues, saturation and values:
And, for reference, the larger versions of the pictures these "whites" come from:
So if you're ever looking at some amazing painting of a white tablecloth, with all sorts of pinks and purples in the shadows, and wondering how on earth the artist could have mapped all those colors out, or observed them so carefully... remember that there is a lot more flexibility in how you handle the color of white than you may think.