Ford Ka door mirror?

I need to change the door mirror on my "S" reg Ford Ka. It is a basic manually operated one. No Electrics. The cast metal fitting inside is broken and the mirror hangs down on the outside of the car. I took off the interior trim, took out a single screw through the plastic fittings, undid the large plastic Nut which was holding it together and undid the trim on the outside, but i still could not remove the Mirror. Can anyone tell me what i missed or why?