Everton took the lead after 22 minutes when former Arsenal forward Alex Iwobi crossed the ball into the box where it bounced off the head of Dominic Calvert-Lewin and then Holding before entering the net.
Arsenal levelled on 35 minutes when Tom Davies brought down Ainsley Maitland-Niles in the box and a penalty was awarded. Pepe sent Jordan Pickford the wrong way before placing the ball in the bottom left corner.
Arsenal improved in the second half but were unable to find a score to level the game.
The result puts Everton second in the table while Arsenal remain in 15th.
Arsenal's start is their worst since the 1974-75 season but Mikel Arteta, who has lost 13 and won 13 from his 34 games in charge, said his team were still up for the fight.
"The boys are absolutely still fighting. They're hurt right now," he said.
"We lost the game on two key moments. We dominated the game. We created enough chances at least to not lose," he added.
"We are in a big fight. We lost against Burnley without them having a shot on target, and now we lose because of a couple of chances. We need to generate more to win football matches. When it comes to small details, the other team wants it more."
The Merseysiders lost four times in October and November but are unbeaten in four games and back in the hunt at the top of the table.
"We did very well in the last few games. Brilliant performances and fantastic results. We have shown more consistency," said Everton manager Carlo Ancelotti.
"This season we started well, then we had a difficult moment, but right now I'm really pleased," said the Italian, who marked his first year in charge of the side.