I don’t know if ‘necessary components’ means a binary diff or something bigger. But if a security update requires rebuilding multiple components, that diff could still be large.
So, you’ve got either an entire image being written, or an image being patched. Then it’s all checksummed and verified. Then you have a phone reboot, with any necessary ‘run-on-update’ scripts being executed.
I think that’s why it takes so long.