Why isn't all the land south of the St. Lawrence River part of the USA?

When the American Revolution happened, none of the rebelling colonies had land on the St Lawrence River banks. Quebec already had the southern banks. People in the area sided with England, and never wanted to join the US afterwards. The US never saw fit to take over the area, and the US eventually became friends with Canada.

Edit: Shipping rights were never much of a concern, since the St Lawrence River has rapids[0] and isn't naturally navigable past Montreal. Besides, in 1825, the Erie Canal opened, and railroads were later built.

[0] https://en.wikipedia.org/wiki/Lachine_Rapids

Natural boundaries often don't make borders. Rivers are as much a highway as a barrier. Likewise, mountain ranges might seperate highlanders from lowlanders, instead of one side from the other.

Also, it wasn't for want of trying. King William's War, Queen Anne's War, King George's War, the French and Indian War, the American Revolution, the War of 1812. Not to mention the Fenian Raids.

It was part of the colony of Quebec at the time. Since that colony did not declare independence, one of the declaring colonies or later the new United States, would have had to annex and occupy that territory. That whole independence and self-government argument would start to sound a bit hollow.

More practically, the region was pretty well-defended. Upper Canada was strongly loyalist and Lower Canada had come to at least tolerate British rule (which was pretty hands-off about local rule and customs, as far as 18th century empires go). And of course the British had excellent transport by controlling the waters, a major disadvantage for the Americans.

Ultimately, see the War of 1812. It's the proposed scenario, more or less. Some have suggested the US fully intended to annex Canada, but there's no clear historical attestation by any leaders to that effect. In any case, at least officially, the Americans declared war following a string of British outrages, and preemptively invade Upper and Lower Canada. Sporadic fighting over two years results in no significant gains for either side and the whole front locked into a stalemale, until the war ended diplomatically.

Québec was settled by the French by sailing from the Atlantic up the St Laurence. So the river was not a natural boundary but the main pathway to expand inland on both sides.

Making the river a border means conflict and land grab by the southern neighbour...

How would it "make a lot of sense" if the two largest French settlements, Montreal and Quebec City, are right on the river?

The US did try to annex the province of Quebec in 1775 but retreated after failing to take Quebec city (which was the biggest city at the time and controlled access to the saint Lawrence river). That was probably the closest Quebec got to joining the USA. The other two attempts (1812 invasion and 1837 rebellions) were even less successful. https://en.wikipedia.org/wiki/Invasion_of_Quebec_%281775%29