With the victory, the NBA-leading Bucks (37-6) extended their winning streak to five games.
Kemba Walker scored a game-high 40 points to lead the Celtics, who took the floor without Jaylen Brown. The 6-foot-6 forward suffered a sprained right thumb in a 13-point loss to Detroit on Wednesday.
"We saw a play late in the first half and we think that might have been when it happened," Boston coach Brad Stevens said. "Maybe it got worse as the game went on. He didn't say much about it at halftime. Certainly, after the game it was sore. Today, more sore."
Stevens said they will do more tests on Brown's thumb on Friday.
Marcus Smart, who started in place of Brown, had 24 points.
Antetokounmpo, who also had seven assists, had his 35th double-double of the season.
After trailing by as many as 27, the Celtics outscored Milwaukee 36-22 to pull within 127-123 with 38 seconds to play. But Smart missed a 3-pointer and Middleton knocked down one of two free throws to seal the victory.
Antetokounmpo scored the first basket of the night and the Bucks never trailed.
Milwaukee hit seven of their first 10 shots, including their first four attempts from 3, and led 36-20 after the first quarter. Brook Lopez had three of those 3-pointers and had 11 points in the opening quarter. Lopez finished with 16 points.
Milwaukee set a new season-high with 76 points in the first half. The Bucks had a 76-58 lead at the break.
Boston stormed out in the second half, cutting Milwaukee's lead to 87-81 with 5:16 left in the third. But the Bucks ended the quarter on a 19-6 run.
Gordon Hayward, who finished with seven points on 1-of-10 shooting for the Celtics, scored his only field goal on a 3 at the end of the third.
Donte DiVincenzo had a career-high 19 points off the bench for Milwaukee.
Celtics: Boston entered the night having won five straight when trailing by at least 15 points after the first quarter dating back to Nov. 10, 2017. That streak, which was the longest such streak in NBA history, ended.
Bucks: Milwaukee has a three-game winning streak against the Celtics for the first time since winning four straight over the 2012 and 2013 seasons.
Celtics: Host the Suns on Saturday.
Bucks: At the Nets on Saturday.
Follow Keith Jenkins on Twitter: https://twitter.com/MrKeithJenkins
More AP NBA: https://apnews.com/NBA and https://twitter.com/AP-Sports
Giannis shares how he's been keeping busy during NBA hiatus
Giannis Antetokounmpo acknowledges it's been tough staying in form but he has found ways to stay productive.
Next steps to get the Chicago Bulls back to contention
How far are the Chicago Bulls away from contending under a new lead basketball executive?
What if the NBA created a 4-point line?
Jackie MacMullan and Ramona Shelburne ponder the idea of a 4-point line in the NBA.