The win lifted Brendan Rogers' side to fourth on 12 points, while the Gunners stay in 10th place on nine after six games.
Arsenal were left ruing their failure to score in the first half hour when they pushed forward time and again, racking up 10 shots and six corners to Leicester's one and zero respectively.
They went even closer to opening the scoring a few minutes later when Alexandre Lacazette, standing in front of goal, somehow nodded another cross from Tierney wide of the far post.
Vardy, who came on with half an hour to play having missed Leicester's previous two games with a calf injury, broke the deadlock in the 80th minute, heading home Cengiz Under's cross.
The striker's 11th Premier League goal against Arsenal -- more than any other player apart from Wayne Rooney -- gave his side their first win away to Arsenal since 1973.
"We weathered the storm, we started getting into it, we played well and came away with a 1-0 win. Its big for us but its just the next game. We had a couple of bad results and just wanted to put it right," Vardy said afterward.
Rodgers congratulated his side for winning the match against a top side like Arsenal, adding that he believes the league race will be "wide open."
"It feels good, it's a long time is 47 years, we just wish supporters were here to enjoy it. A good team performance, we were defensively strong and we had a lot of quality," Rodgers said after the game. "We always felt we wanted to bring Jamie into the game at 60 minutes, to stretch the game behind."
Meanwhile, Arsenal manager Mikel Arteta was left to rue Lacazette's overruled goal.
"I'm really disappointed, we had it in our hands in the first half and I don't know how the hell the goal was disallowed," the Spaniard said. "They caught us with space at the back and in that moment we lost the game."