Car door lock not working ?

My Vauxhall Vectra 2004 Door lock will lock manualy by key but will not unlock manualy. The Fob stoped working a long time ago and have been opening and locking manualy since.
How do I remove the door panel to get to the lock from inside to see if anything has become loos as when I apply a bit of pressure on the key whilst turning to the unlock position the cylinder seems to move slighyly?