I find it kind of amusing that if you are only interested in people at or above a certain level of physical attractiveness, it's considered bad, shallow, or exclusive, but if you are only interested in people at or above a certain level of intelligence, it isn't. Both are traits are inherent and predetermined, and you can't really, drastically change them. Yeah, you can do things to try to improve both you appearance (make-over, plastic surgery) and your intelligence level (reading, studying) but if you were born with below average intelligence, you're never going to be a genius, just like you're never going to look like an Adonis if you were born to look like a garden gnome.
I personally don't think that my preference for guys who are quick-witted is any less, or more, shallow than my preference for guys who are taller than I am.