HACKER Q&A
📣 version_five

Supply chain attacks in ML models


I'm on pytorch hub, which is a site where you can download ML models others have build and trained. Some of the organizations that have models on there I've never heard of, and have model pages that look more like a page for some dodgy amazon brand you've never heard of - it hotel my radar up a bit. My question is, has anyone encountered this as an attack vector before, distributing a pytorch (or I assume TF) model that runs arbitrary code, and having it do something malicious? Some of these have a lot of users and I doubt many are taking the time to review what they are actually running. Let me know if I can clarify, thanks!

  👤 masterchief1s1k Accepted Answer ✓
Unfortunately, the problem is the same with many compiled application out there, not just ML.

My rule of thumb in using models from these public model zoo is to always use a separate container instance (or Colab) to check the source code. Then either convert such model to some standardized format like ONNX, which might not be possible for some state-of-the-art models with custom ops afaik.

Or you could just load the torch model first, then load the model state dict separately. Avoid using command like torch.load() since it uses pickle module implicitly, since it is possible to construct malicious pickle data which will execute arbitrary code during unpickling.

👤 stevenminhhh
Unfortunately, the problem is the same with many compiled application out there, not just ML. My rule of thumb in using models from these public model zoo is to always use a separate container instance (or Colab) to check the source code.

Then either convert such model to some standardized format like ONNX, which might not be possible for some state-of-the-art models with custom ops afaik.

Or you could just load the torch model first, then load the model state dict separately. Avoid using command like torch.load() since it uses pickle module implicitly, since it is possible to construct malicious pickle data which will execute arbitrary code during unpickling.