It was the second major shock at this ATP 250 hard-court event, following defending champion Rafael Nadal’s first-round exit to qualifier Michael Berrer on Tuesday.
World number one Djokovic followed his Spanish rival out of the tournament in Doha, after Karlovic produced a stirring performance from one set down.
The big-serving Croatian eventually prevailed 6-7 (2-7) 7-6 (8-6) 6-4 to book his semi-final spot, where he will take on David Ferrer.
After the first set went with serve, Djokovic took the lead by clinching the tie-breaker.
The second set followed the same pattern, only this time it was Karlovic who prevailed in the extra game to set up a decider.
The match’s first break of serve eventually arrived in game seven, and seventh seed Karlovic was able to successfully consolidate his advantage, seeing out the victory by serving to love in game 10.
He will now face fourth seed Ferrer, who had little trouble in overcoming Dustin Brown 6-2 6-2.
In the other half of the draw, Tomas Berdych had a similarly straightforward passage as he breezed past Richard Gasquet 6-2 6-1.
Berdych’s next opponent is Andreas Seppi, due to the Italian beating Ivan Dodig 7-5 6-4 in the day’s final singles match.