Arsenal pulled level just before halftime through Danny Welbeck, who is, contrary to rumors, still alive.
Such an effort provoked scenes of sheer jubilation at the Emirates Stadium and came just two minutes after fellow substitute Aaron Ramsey looked to have rescued a point for the inconsistent Gunners after blasting in from close range.
Friday night, Arsenal versus Leicester kicked off the new English Premier League season with a dramatic 4-3 goal fest. It was one of the great days for me in recent weeks [when he said he would stay].
Arsenal are now unbeaten in 22 Premier League games against Leicester City - winning 15 and drawing seven - and they have won 11 successive Premier League home games against the Foxes. "Obviously we can build on it from there".
Marc Albrighton served in a ideal ball from the left, curling a cross for Vardy, who capped the sequence against the run of play with his first goal of the season.
"That's what we're aiming to do", he added.
IAN KINGTON GettyImages
Arsenal stepped things up, Schmeichel saving from Alex Oxlade-Chamberlain and home debutant Sead Kolasinac, but they were careless in possession and shortly before the half-hour, Leicester went in front for the first time.
"We're gonna win the league!" sang the Leicester fans and they nearly had a third goal to cheer when Okazaki's header from Christian Fuchs's cross bounced just wide of Cech's left-hand post.
One new recruit, defender Harry Maguire, felt Leicester gave the hosts a run for their money and - despite heading back to the east midlands with nothing to show for it - believes the display will hold them in good stead.
"The season started with a handsome game which met all expectations".
"I opened the door for him at some stage as I had many strikers and it's important for him to play but I don't want him to go", he said.
"But as much as I can I will try to help the team to reach our target and tonight we needed this win as everybody knows the importance of the first game for confidence and we keep going".