Dortmund, who missed a late penalty, did everything right in the first half but somehow failed to score, with Jude Bellingham also hitting the woodwork.
Mainz struck with their first chance after the break as Levin Oztunali picked up the ball in midfield, charged towards the Dortmund goal, beat one defender and then released an unstoppable left-footed missile from 25 metres out.
Mainz then hit the post a little later before Thomas Meunier levelled in the 73rd minute with his first goal for the club.
Marco Reus then failed to put Dortmund in the driving seat, firing his penalty wide a minute later before Mainz hit the post again in the 78th in an exciting finale.
Dortmund are fourth on 29 points, with Bayern, who face Freiburg on Sunday, in top spot on 33.