As the first Slovenian side to appear in a European quarter-final, there was a clear sense of occasion with the home crowd looking to inspire their team into a fast start.
They were almost rewarded when Logan Delaurier-Chaubet found space in the box, but he could only find the side-netting with his drilled effort.
The visitors dominated possession without really threatening, but they did find the crucial breakthrough just before the half-hour mark.
The captain, Luca Ranieri, eluded the defence as he drove deep into the box before firing the ball through Ricardo Silva from a tight angle.
Celje had a chance to level just before the half when the ball was poked through to Klemen Nemanic, but he lacked composure and sliced the ball well wide.
Despite a bright start after the break, the hosts fell further behind just after the hour mark when a VAR review ruled that Rolando Mandragora had been fouled in the box, and he dusted himself down to smash the resultant penalty into the back of the net.
The home side bounced back quickly with a penalty of their own just five minutes later.
Aljosa Matko was clumsily wiped out by Marin Pongracic, and Delaurier-Chaubet calmly sent David de Gea the wrong way to halve the arrears.
The visitors appeared to strike again with ten minutes to go, only for the offside flag to curtail Nicolo Zaniolo’s celebrations, before the hosts took control as they searched for an equaliser.
Ultimately, they couldn’t find the goal, leaving Fiorentina to take the 2-1 advantage into the second-leg next week, where they’ll be favourites to win their 12th consecutive two-legged tie in the Conference League.
Flashscore Man of the Match: Luca Ranieri (Fiorentina)