I recently fitted a loo seat with similar results! I then checked to see whether the plastic 'buffers' were sitting flush (sorry!) against the toilet bowl and they were only just touching it - so when you sat on the seat it moved. With some experimenation (changing hinges around) I moved the seat forward so the buffers were squarely on the rim of the bowl - result a rock solid toilet seat! So - try taking the hinges off the 'axle' and turning them around 180 degrees before refitting (also site the seat where it should be before fitting and see where the hinges need to be in that position). A properly fitted seat is actually further forward than you would think. Good luck.
Matt.
November 2008