I did some looking into this, apparently similar problems have been reported by others. Most of the time the exhaust does not come right off, but gets very loose.
They think its because of the way its mounted, on my 02 there is a bolt right where the headers come into the main pipe, goes into the bottom of the engine. On the 03 apparently its only held on at the headers, and then at the can itself.