Social context modulates action kinematics. Less is known about whether social context also affects the use of task relevant visual information. We tested this hypothesis by examining whether the instruction to play table tennis competitively or cooperatively affected the kind of visual cues necessary for successful table tennis performance. In two experiments, participants played table tennis in a dark room with only the ball, net, and table visible. Visual information about both players' actions was manipulated by means of self-glowing markers. We recorded the number of successful passes for each player individually. The results showed that participants' performance increased when their own body was rendered visible in both the cooperative and the competitive condition. However, social context modulated the importance of different sources of visual information about the other player. In the cooperative condition, seeing the other player's racket had the largest effects on performance increase, whereas in the competitive condition, seeing the other player's body resulted in the largest performance increase. These results suggest that social context selectively modulates the use of visual information about others' actions in social interactions.