How do I get the tailgate switch housing out of my signum?

I have taken the rubber protector off the switch housing so that i could release the aluminium flap that opperates the tailgate micro switch. I have removed the trim from inside the tailgate, but i cant get the switch housing out so that i can replace the rubber protector, i dont want to force it and risk damaging it. Any ideas?