Gauff, 16, took a see-sawing 61-minute first set on a tie-break, but was broken to love in the final and decisive game of the second.
But the American’s pressure told in the third as Belarusian second seed Sabelenka’s serve crumbled late on.
It is the third victory over a top-20 ranked player in Gauff’s career.
She has previously beaten Naomi Osaka at the Australian Open in January and Kiki Bertens in Linz in 2019, who were ranked fourth and eighth in the world respectively at the time of playing Gauff.
Gauff will play Tunisian world number 39 Ons Jabeur next in Kentucky after the eighth seed recovered from a set down to beat Belarus’ Olga Govortsova.
Gauff would have to win through to the final for any possibility of meeting either Serena or Venus Williams at the tournament.
The Williams sisters, on the opposite side of the draw to Gauff, meet in a last-16 match on Thursday.