Ulm's Steffen Kienle and Vitalij Lux were their side's heroes, as the side were able to hold on despite Goncalo Paciencia's final minute goal.
It is only the first time that the holders of the German Cup have lost in the first round since Kaiserlautern lost to Greuther Furth in 1996.
Bayern Munich went through, but needed a late winner in order to do so against fourth division Drochtersen/Assel.
Lewandowski was able to tap home the only goal in minute 81 in front of a sell-out crowd.
"It was a tough game, which can happen in the cup," Niko Kovac told reporters following the game.
"We didn't actually want that and knew what to expect. We acted too slowly, but the bottom line is that we are through."