Generally its better to ask. If theyre already a customer, there is a good chance someone is going to go to your site and see it.
Sometimes it gets wierd, we had a large fast food logo on our site. We had only sold to 5 or so franchises in our area... So it was true but only kindof.
(This excludes public brand usage guidelines published by companies for including their logos eg: Social media links for FB, Goog, LinkedIn, Instagram etc., and they are covered by the separate terms of use there.)
Check your agreement with your client. If it has such a clause related to marketing material (social proof would fall under that), seek their permission as prescribed in your agreement. It may help to get your client liaison on your side by letting them know first.
Then wait.
It is quite possible that the client may not get back to you. In that case, err on the side of caution and DO NOT use the logo on your website/social proof page.
As an alternative, when we made presentations to a small room (of prospective clients, potential investors etc.,) we may include a Top clients roster slide with the logos but mark that specific slide as confidential and show it only within the room and never use it for broader distribution. This practice is easier for your legal team to clear if the parties in the room are under non-disclosure agreements. It may be okay in other circumstances; read the room and use your discretion.
Not a lawyer.
Source: personal experience.
At best, it would be presumptuous, at worst, a cause for legal action.
I mean, I already do it, but should I also ask permission to do that? That would be weird since they had to pay font awesome to have their logo included in the font.
In some cases, the customer will expect a discount, knowing that they're a large customer and that their logo on your startup's website will instill some trust from potential customers and could lead to more sales.