Dirty steam?

As I'm starting to use the iron when I press for the steam, dirty brown steam runs out of the side of the iron plate, didn't know if it was the sponge thing that goes in it or not so I've tried to wash it but doesn't really help, this seems to have been happening since I got it, probably about the 3rd time I used it - please help!!