You wont get the olive off as you may have found.
Go to plumbing shop ask for fernox jointing seal its in a tube in a yellow box or get something similar. Turn off the water undo the joint collar pull it back wipe the joint dry then squeeze about quarter inch of sealant on finger and wipe around olive against the joint, pull pipe back slightly if it will move, then some around the back end that the collar fits against then re-tighten. same on other joint.
bob
January 2013