Normally they are held in by screws, but sometimes they just push in and are held by small "teeth" so they cant come back out. You could just cut them as close to the old socket as you can so as to leave as much to connect to the new socket.
But be very carefull you have turned all the power OFF
Jeff
September 2008