Has anyone tried to make LLMs drive a car?

Question

Seems like LLMs are very good at generalizing to random tasks that they're not necessarily trained for. I wonder how GPT-4 would do at responding to inputs from LIDAR sensors on an autonomous vehicle.

rvz · Accepted Answer

Why?We already have to deal with Tesla FSD hallucinations and regulating it in serious and safety critical applications such as autonomous transportation.I don't think we need a LLM behind the wheel and hallucinating directions to end up leading the driver to drive off a cliff or running past a stop sign it got confused over.> Seems like LLMs are very good at generalizing to random tasks that they're not necessarily trained for.Some tools are useful for other applications, especially safety critical applications. LLMs are *absolutely* not useful for this use-case.

sharemywin · Answer

I don't think the reaction times would work.
Where it could add possible value is to dream up use cases.
give me some unusual and/or weird uses cases a driver or autonomous vehicle might come across while driving( on the high way, at a cross walk) etc.

Has anyone tried to make LLMs drive a car?

Seems like LLMs are very good at generalizing to random tasks that they're not necessarily trained for. I wonder how GPT-4 would do at responding to inputs from LIDAR sensors on an autonomous vehicle.

I don't think the reaction times would work.Where it could add possible value is to dream up use cases.give me some unusual and/or weird uses cases a driver or autonomous vehicle might come across while driving( on the high way, at a cross walk) etc.

I don't think the reaction times would work.
Where it could add possible value is to dream up use cases.
give me some unusual and/or weird uses cases a driver or autonomous vehicle might come across while driving( on the high way, at a cross walk) etc.