HACKER Q&A
📣 binibus

Which is your preferred language for ML deployment


At training time and experimentation, Python is king, but I've been searching for a while for the best programming language to use in production environments. The scope I'm thinking about is a microservice, maybe in a server maybe on the edge, with a pretty simple code (input conversion -> inference -> output conversion). Appreciated features: strict type system to avoid errors without too much testing, reasonable performance, ease of parallelization, and garbage collection (I find memory management overkill).

Some options I have in mind are go, rust, ocaml, haskell, erlang/elixir, and scala.

What is your experience in these cases?

  👤 yawaramin Accepted Answer ✓
At work I use Scala and it's of course rock-solid, but it feels like a drag to develop and deploy–taking minutes to iteratively develop, and tens of minutes to test and build an assembly jar or Docker image. For a side project I'm using OCaml and greatly enjoying the fast feedback and deploy experience. For parallelization, there's not really much choice than to use multiple processes for now; but in the near future true multicore support will be shipped. Performance is very reasonable–most of my responses are in low milliseconds. Take a look at https://aantron.github.io/dream/