This article describes two experiments investigating listeners’ accuracy in estimation of speaker age as well as the listeners’ confidence that their estimates were correct. In Experiment 1, listeners made age estimates based on spontaneous speech. In Experiment 2, the estimates were based on read speech. The purpose of the study was to explore differences in accuracy and confidence depending on speech material, speaker characteristics (gender and age) and listener gender. Another purpose was to examine the realism in the listeners’ confidence ratings in estimations of spontaneous versus read speech. No differences in accuracy or confidence were found due to speech material type. Although accuracy was higher in estimates of male speakers, confidence was higher in estimates of female speakers, effects that were also dependent on speaker age. Possible acoustic and linguistic explanations behind the age and gender effects are discussed. As the correlation between confidence and accuracy was weak, it was concluded that confidence should not be relied on as an indicator of accuracy in estimation of speaker age.