Marshall Henderson scored 19 points for Ole Miss but made only 5 of 21 shots, including 2 of 16 3-pointers. Jarvis Summers led the Rebels with 26 points but missed a last-second jumper.
Georgia (19-12) will play Kentucky in the second SEC semifinal game on Saturday, following No. 1 Florida's game against Tennessee.
Kenny Gaines, a Whitefield Academy product, and Brandon Morris each had 12 points for the Bulldogs, who may need to win the SEC tournament to earn a spot in the NCAA field. J.J. Frazier added 11 points.
There were 51 fouls called in the game. Sebastian Saiz and Dwight Coleby fouled out for Ole Miss (19-14).
Mann made two free throws to give Georgia a 72-71 lead. Summers answered with a jumper with 45 seconds remaining to give the lead back to the Rebels.
Following a timeout, Gaines missed a 3-pointer, Morris missed a follow shot and Mann grabbed the rebound and scored on a layup to give Georgia the lead.
Henderson missed a jumper and fouled Marcus Thornton, who made 1 of 2 free throws for a 75-73 lead. Summers missed a last-second jumper following an Ole Miss timeout with 2.6 seconds remaining.
Georgia struggled with fouls in the first half. Thornton played only 5 minutes and Mann was limited to 7 minutes after each picked up two early fouls. Two more Georgia starters, Donte' Williams and Gaines, finished the half with two fouls.
Frazier, who took over for Mann, hit two 3-pointers and led Georgia with eight points in the half.
Ole Miss made only 1 of 11 3-pointers in the half. Henderson missed 6 of his 7 3s, but Summers' scoring filled the void as the Rebels led 38-36 at halftime.
Fouls became a bigger problem for Ole Miss in the second half. Coleby fouled out with 8 minutes remaining, and Saiz drew his fifth with 3:38 remaining.
Georgia coach Mark Fox was called for a technical foul when he disagreed with a foul called on Mann with 6:35 remaining. Fox continued to argue and pushed away assistant coach Jonas Hayes when Hayes tried to intervene.