My car takes time to start 1.9cdti 2007 model sri vectra?

I have had the car for now over 2 months only done 80000 on the clock full main dealer service history and has an alternator changed on 75000 miles, the problem I am having is when I turn the key sometimes the car will not start this can be anytime of the day, but I have realised that if the car is not driven much then on that particular day it will not start immmediatly, I also notice when the car sometimes starts up the baterry light comes on, I then switch the engine off and restart it the battery light dissapears, this particular problem with the baterry light I have had about 6-7 times, can anyone please advise. I have been to the main dealers and they have advised that there is no problem, don'y worry about it.