Dryer roller on drum?

I replaced my dryers belt and Idler pulley. The rear drum rollers look good, but the one on the left does move back and forth a lot, keeping the drum from staying tight against the unit. The dryer spins and drys fine, however, you can still see the gap between the drum and the unit. Do I need to replace the felt for the drum and/or the rollers? Is there any way to tighten the roller so it does not move in and out? It is a Sears, Kenmore series 80, electric dryer.