I found one of these the other day. The filling "loop" is a small fixed copper pipe with a tap at one end. This tap had as you described a small white bit in it that stopped the flow when the tap was shut. I would not recomend removing this as then there is nothing to stop the water pressure from increaseing and the pressure release opening up. Which I think is what you are describing is now happening. I think you will nead to replace the Filling loop valve Part No 801679
You will be able to get it from Plumb Center, http://www.keeptheheaton.com or any other supplier.
Peter
November 2009