I have had this problem with my SNES, and as I own an N64, it could be - the wire between the console to the TV. Nintendo were cheap sods when it came to this wire - their pins are a hell of a lot weaker than in, say Segas or Sonys. If your wire shows any damage or wear - Get a new wire - only around £2.
Mr_Fusty
January 2008