But when I put the idler pulley back on there is no clearance between the pulley and the engine. Am I missing a spacer or something?

the bolt is sticking out and I slip on the pulley put on the nut and I can see it rubs before even tightening it.

I really am not an idiot I know....