Steven Gerrard's Rangers finished with nine men in a 2-0 home loss to Slavia Prague, as the Czech side won 3-1 on aggregate.
Peter Olayinka gave Slavia an early lead at Ibrox and Rangers' hopes of a comeback suffered a double blow with Kemar Roofe and then Leon Balogun sent off for the Scottish champions in the second half. The former caught the goalkeeper in the face with his boot while Balogun was dismissed for a second yellow card.
Nicolae Stanciu's free-kick immediately after Balogun's dismissal ensured victory for Slavia.
Ajax made sure of their passage into the next round as a David Neres goal and Dusan Tadic penalty rounded off a 5-0 aggregate win over Young Boys.
Gerard Moreno's double in Spain gave Villarreal a second 2-0 victory over Dynamo Kiev in as many weeks.