Task-irrelevant background sound can disrupt performance of visually based cognitive tasks. The cross-modal breakdown of attentional selectivity in the context of reading was addressed using analyses of eye movements. Moreover, the study addressed whether task-sensitivity to distraction via background speech on reading was modulated by the cognitive demands of the focal task. Two randomly-assigned groups of native Chinese participants read the same set of Chinese experimental sentences while being exposed to meaningful speech, meaningless (foreign) speech, or silence. For one group, participants were instructed to judge whether the sentences made sense (i.e., semantic acceptability task); for another, participants were instructed to detect whether the sentences contained a noncharacter (i.e., noncharacter detection task). Results showed no significant effect across sound conditions for the noncharacter detection task. For the semantic acceptability task, however, there was a substantial disruptive effect of the meaningfulness of the speech. Compared with reading with meaningless speech or reading in silence, the meaningful speech increased numbers of fixations, regressions, regression path, and total reading times. These results suggest that the disruption of reading by background speech is jointly dependent on the nature of the speech and the task process deployed, thereby favoring an Interference-by-Process account over Interference-by-Content and Attentional Diversion accounts of distraction to reading by background sound.