While there are naturally occurring levels of copper in our environment, putting pennies in water and leaving them there is not natural and may expose birds to too much copper, potentially causing harm to birds. Instead, rinse your bird bath a couple of times a week and clean once every week or so.