How guilty should I feel about the environmental impact of using AI?

Less guilty than you might about the means by which materials are sourced to make your electronics.

As guilty as you feel about your use of electricity and hydrocarbons in general. For me that is not at all. If society cared enough about climate change it would impose a carbon tax, but it doesn't, so instead we get mandates that don't reduce overall emissions.