You may be drilling into Lime Plaster often found in older houses (mine 1935 has it all over) concrete will (should not) "powder".
To fit curtain poles in my house I use a wooden batten fixed firmly with two or three 6" frame fixings right through the batten and into the brickwork...
http://www.toolstation.com/images/library/stock/webbig/61889.jpg
...and screw the pole brackets to the batten with woodscrews. Much better hold than trying to fix the brackets directly to the wall - even if it holds it will not be long before it all comes tumbling down as the lime plaster crumbles. Fixings well into the brickwork are much more permanent.
Good luck...
Peccavi
March 2010