i'm and advocate of wearing it and do all the time. there are a few times where i have to have it off (sitting at the bank drive through, backing the car in the garage, etc), and the chime does get annoying. i've disabled mine using the below thread:
Disable seat belt chime (Threads Merged)