It would be a shame!

What is it with the words “it would be a shame”, that bothers people.

I saw a nice couple with a baby, I said what a lovely baby you have, it would be a shame…
I saw a old man pushing his wife in a wheelchair, I said what a nice wheelchair, it would be a shame.
I saw a guy with a nice hat on, I said what a nice hat you have, it would be a shame.
I was putting petrol in my car, a guy was filling up his motorbike, I said what a nice bike you have, it would be a shame.

All these people took serious offence, all because “it would be a shame”… so words do mean something to most, I think, anyway. It would be a shame