The decorations have very small pegs/sticks protruding from them. I'm not sure what else they could attach to. I think that those pegs/sticks are even smaller than those which are on the ribbons that attach to the back of Princess dresses and the back of some dining chairs.
I haven't checked this out, though. I was too busy moving on to the next pony set, and once I took the decorative pieces off, I just put them into a container for safekeeping.