One of the key behavioral effects of attractiveness is increased visual attention to attractive people. This effect is often explained in terms of evolutionary adaptations, such as attractiveness being an indicator of good health. Other factors could influence this effect. In the present study, we explored the modulating role of sexual orientation on the effects of attractiveness on exploratory visual behavior. Heterosexual and homosexual men and women viewed natural-looking scenes that depicted either two women or two men who varied systematically in levels of attractiveness (based on a pre-study). Participants’ eye movements and attractiveness ratings toward the faces of the depicted people were recorded. The results showed that although attractiveness had the largest influence on participants’ behaviors, participants’ sexual orientations strongly modulated the effects. With the exception of homosexual women, all participant groups looked longer and more often at attractive faces that corresponded with their sexual orientations. Interestingly, heterosexual and homosexual men and homosexual women looked longer and more often at the less attractive face of their non-preferred sex than the less attractive face of their preferred sex, evidence that less attractive faces of the preferred sex might have an aversive character. These findings provide evidence for the important role that sexual orientation plays in guiding visual exploratory behavior and evaluations of the attractiveness of others.