Not sure this really has to do something with iCloud. If he was controlling the screen as well he must've started some kind of VNC server or rdesktop on your phone.
And yeah there are undocumented exploits ofcourse, so it's not impossible but those attacks are very rare.
Replacing your cell phone works ofcourse, but you could also factory reset it.