There should be a new set of spacers that are thinner. The front are different thickness from the back. Don't reuse the spacers that came the sissy bar.
The order working from outside in should be Grab Rail, Spacer, Saddlebag Bracket, Sissy bar then Bolt.
The right side saddlebag stay mounts from the inside of the bikes frame, opposite installation from the left side FYI. Mess this up and the right stay will stick out. I found out the hard way LOL.
I recommend some additional painters tape or cloth on the sides where the stay bolts to the bracket so the bolts don't scratch the paint.
Mount everything loose and tighten at the end.