When solving mental arithmetic problems, one can easily be distracted by someone speaking in the background and this distraction is greater if the speech comprises numbers. We explored the basis of this disruption by asking participants to solve mental addition problems (e.g., "45 + 17 = ?") in three different conditions: background speech comprising numbers in ascending order (e.g., "61, 62, 63, 64, 65"), background speech comprising numbers in descending order (e.g., "65, 64, 63, 62, 61"), and quiet. Performance was best in quiet, worse in the descending numbers condition, and poorest in the ascending numbers condition. In view of these findings, we suggest that disruption arises as a by-product of preventing the primed, but inaccurate, candidate responses from assuming the control of action. Alternative explanations are also discussed.