Some artists rely heavily on visual reference, others work solely from imagination. There's nothing wrong with either approach - you can produce great art both ways. The important question is whether or not you are currently using reference in the way that's best for your art and your artistic development. Depending on where you are you may benefit from learning to use reference more, learning to use it less, or learning to use it differently.
If you want to expand your ref-less capabilities, here are two exercises to consider:
1) Force yourself to create an image solely from imagination. Do not give up when the going gets tough, but think, think, think about what you are trying to draw and paint, what its form is like, what that form would look like under the given conditions, etc. When you are away from the image, keep your eyes open for the answers to those nagging questions you couldn't get right - everything from color to anatomy to form and space. Working without reference makes the gaps in your abilities crystal clear, and may be disappointing, even embarrassing. THAT'S A GOOD THING. In fact, as a side note, anytime you look at your work and think "that looks horrible" count yourself lucky that you can see it. The alternative is much worse.
Anyway, back to working without ref - even though you may see some real weaknesses in your ability to depict certain things, at the same time you may find your work has a certain power and personality that was lacking before, and this only improves with practice. When you have to figure out visual things in your own head you understand them in a way that is much deeper, more profound, powerful and personal, and this is reflected in the artwork. Once you've given it your all, I mean really gone to the wall, then you can go obtain some reference material and use it to "fix" or simply check your work. You will likely never forget the answers to the particular questions you struggled with in that picture, whereas you'd used reference from the start it would have been in one ear (or eye) and out the other.
2) Create a painting from imagination, and obtain reference material by sketching only, whether from life or photos. It's better to draw from life because we tend to empathize and identify with our subject more fully and physically when it or he or she is a real thing. Art-making is not just in the eyes, it's in the body.
Then use the sketches as your reference for the painting. Don't look at the actual subject or photos while painting, just look at your sketches.
To those of you who say using ref doesn't work with your creative process...
I say, yeah, that's true, it doesn't fit well at all. So just like anything else you need to figure out how to deal with that. If we could just draw and paint completely from memory it'd be a lot easier to simply "create." For some genres and styles this is quite possible, but for others it isn't. In some cases it is possible, but not necessarily desirable (even though working without ref may feel good, that doesn't mean it's best for your art). Some artists say they're too lazy or, the most common assertion, "I usually don't have time to pull ref." Not having time is not a good excuse for anything to do with art and quality. If you're in the proud to be ref-less group, try this exercise:
1) Take some piece you've already finished, then go find or shoot some reference and study it alongside your work. Do you see anything there that might improve some aspects of your work without, of course, weakening the overall power and statement? I don't think any representational artist could honestly answer "no" to this question. You will see something of value. That means you'll either see mistakes in your work, or you'll simply see things in the refs that you like better than how you painted them initially. Maybe real hair doesn't blow in the wind exactly the way you imagined it, and/or you like this look more than how you rendered it from imagination. What's important is whether the work wants it, not whether the process feels good the first couple of times you try it. Lots of bad habits become perpetuated simply because a different way of working feels uncomfortable at first. In the beginning this will likely feel as forced and as debilitating as not using reference does to the artist who his chronically dependent on it.
My basic approach to using reference
The core practice alluded to in the above exercises is actually the same for both situations (whether you use ref too much or too little), and that is: work as much as you can without reference, then use it as needed, and always draw a lot in general.
I fell into this approach the same way I fall into all my artistic practices - rigorous, unrelenting and (I hope) brutally honest self-evaluation and evaluation of other artists' work.
I saw many artists' work really suffering from being too derived from photo ref, no matter how great the work was, and I saw this in some of my early work where I was too reliant on photo refs. So then I went a year without using any ref, and though my ability to draw and render from imagination went through the roof (relative to where I was), the reality is I (and I don't think any representational artist) can really close that final 5-10% gap that proper use of reference can inform.
So now I run the gamut in terms of my use of reference, even within a single image. Sometimes my use of reference is more akin to using a spell checker at the very end of the process. Frequently I look at a piece of reference and go, "oh, well, I got it right" or "I like the way I painted the hand from imagination better." Other times I shoot photo reference for specific parts of the piece. Sometimes I prefer to create specific reference from real life using pencil studies. I often create entire pieces without reference at all, because in some cases that's what works for the art. In the vast majority of cases I look at a lot of reference for inspiration and ideas, but then don't use it at all while actually painting.
The point is to be open, flexible, and intelligent about using reference, not rigid and dogmatic. LOOK AT THE WORK HONESTLY. Once you are able to embrace a wider spectrum of possibilities with respect to employing reference, you will be much more flexible and dynamic in how you approach your work, for example, using reference more for some parts of an image than others, finding all the answers you need in reference that is only a "loose fit", or giving your work some extra punch and polish by judicious applicatoin of good reference material relatively late in the process. In the end the trick is to be able to gauge the appropriate use of reference material in your work, and not be stuck in habits that prevent you from creating the best work you can.
You need to get to the point where a quick Google image search to see what wet hair tends to look like, or to fully and deeply understand the rigging of a tall ship (or whatever) does not cause a prohibitive hiccup in your process, but at the same time you need to be quite capable of doing any picture of anything whatsoever without any ref at all. If you can do those two things then you can do anything (artistically, that is).