If the mastic is sound then just clean it with a bleach solution in warm water.
If it is pulling away then it can be removed using a metal scraper (but take care not to damage plastic window frames) or perhaps a cheese shaped wedge of hard wood.
Another way is to buy a mastic solvent (check small hidden area that solvent does not effect plasic) from DIY plumber merchant.
Once resealed, see if you can improve ventilation in this area or perhaps get a bit more heat into the room.
Maurice
November 2010