The Orica-Greenedge rider crossed the line in 17th place, 18 seconds behind Australian Rohan Dennis (BMC Racing), who launched a lengthy sprint for the line and held off teammate Cadel Evans to win the 143km leg from Norwood to
Impey, who had climbed to fourth place overall after finishing second on stage two, dropped back six spots with three stages remaining.
The South African was 22 seconds behind Dennis, who took the overall lead with a gap of seven seconds over former Tour de France winner Evans.
Previous general classification leader Jack Bobridge (UniSA-Australia) dropped back to 25th position, lying 39 seconds off the pace.
The 145km fourth stage on Friday will take the peloton from Glenelg to Mount Barker.