The offending sleeve is most likely not the right one for the bike as they're a snug fit but can still easily be removed by hand. It sounds like your one was forced on. If there's enough room behind it, I'd use a three pronged pullers to remove it. I'd advise not bashing it or using levers to pry it off or you could end up damaging the main bearings.
Steo
July 2012