Goalkeeper Badra Ali Sangare lost control of the ball and spilt it for Sierra Leone's Alhaji Kamara to snatch a 95th-minute goal as they grabbed their second point of the tournament after holding defending champions Algeria 0-0 last week.
Ivory Coast now have four points while Sierra Leone are on two with Algeria to take on Equatorial Guinea in Sunday's second Group E match at the Japoma Stadium.
They looked to be heading for a deserved win when Sangare somehow managed to lose control of the ball, allowing Steven Caulker to feed Alhaji Kamara.
Yet the substitute striker was stumbling as he tried to get to the pass before making contact and steering it home to set off frenzied celebrations.
It was a superb result for the Leone Stars, returning to the finals for the first time in 26 years.
"Best of all is the spirit we have shown. Just to be here is a win for us," said coach John Keister.
The Ivorians squandered an early penalty opportunity as Wilfried Zaha was brought down in the 12th minute by a clumsy challenge from Sierra Leone captain Umaru Bangura, offering Frank Kessie the opportunity to put them ahead from the spot.
Yet his kick was saved by Mohamed Kamara, who had a man of the match performance in Sierra Leone's previous game against Algeria.
Haller's opener arrived in the 25th minute and was set up by a defence-splitting pass from Zaha before being expertly converted, earning the Ivorians a halftime lead.
Sierra Leone made full use of rare second-half opportunity to snatch an equaliser with a powerful 55th minute shot from Musa Kamara.
Ivorian woes at the end were exacerbated by Eric Bailly going off with what appeared to be a cut to the head while Sangare was taken off on a stretcher after his howler, leaving the Ivorians with a possible goalkeeping crisis for their last group match on Thursday against Algeria.