The two sides shared nine tries – including one penalty try for the Irish side – in a frenetic encounter in Bloemfontein.
Ulster led 32-29 with seven minutes left as Johnny Stewart’s score was quickly followed by a penalty try – Charles Marais sin-binned for ill-discipline on the home side’s line.
Despite Marais’ absence the 14 men of Cheetahs came roaring back, Rabz Maxwane crossing for a try converted by Louis Fouche, whose subsequent long-range penalty looked to have sealed a triumph.
But Speight had the final say as the clock went past 80 minutes, powering over from the back of a maul, Billy Burns converting as the Conference B leaders avoided a first defeat after three successive wins.
Cardiff Blues’ disappointing start to the season came to an end as they eased past Munster.
They only led 14-13 at the break but scored 23 unanswered points in the second half, Gareth Anscombe kicking 13 of his 17 points after the restart.
In other news:
Sign up to our mailing list here and we’ll keep you up to the minute with weekly updates from the world of rugby.