seat belt buckle

I noticed that my seat belt buckle end system on driver's seat is "loose" whereas the one on passenger's side is "stiff". I'm having hard time describing this but I hope you'll understand... :doh: I don't mean the locking mechanism but the whole assembly, the point where it's attached to the...