Web devs, px or rem for sizing? Is there a general consensus?

I've always used px but started to use rem thinking setting the base font size would help limit the number of font-size declarations with media queries. But it never works out that way - the design requirements for font-size always vary so much that simply changing the base font size does not help at all. I still have to declare numerous sizes for various breakpoints. I don't know what rem helps with - since I still need numerous declarations for sizes, I might as well use px which is more comfortable to me.