That's what we refer to as throwing a grenade into the room, Marc. I think your premise is false, and I suspect that you're smiling when you say that. I concede that at the "connoisseur shows", like ours, many artists and patrons used to look down their noses and sniff at a fantasy or a sci-fi piece. But those days are long behind us, and both genres are well-represented in the broader hobby of painting miniatures.
If you first painted a fantasy figure in 1973, and you're asking now, 41 years later, if you should give up on them, I really do think you're just looking for a rise out of the rest of the Planeteers.
As the others have said, paint what you like, don't let anyone discourage you.
Prost!
Brad