For now, I have an encrypted disk volume (so everything is in one place and it's easy to remove from the machine) and a separate browser profile with an SSH tunnel as SOCKS proxy, but maybe using one of my personal machines via some remote desktop solution would be the cleanest solution to this.
I guess it could also depend on how severely restricted and monitored said company machine is.
When I do private stuff (happens sometimes), I only do it in private browser windows. And only if the HTTPS session is not cracked by the company proxy (I check the certificate chain).
That said, portable apps running off the encrypted disk might also help. Browser, email, etc.