It takes about 10 milliseconds to execute 1m modulus operations on Number.MAX_SAFE_INTEGER and about 5 milliseconds to execute 1m modulus operations on 100.
But I can't wrap my head around how.
How is it virtually just as fast to find the remainder of such a large and small number in Javascript?
Number.MAX_SAFE_INTEGER is 2^53 - 1 (9,007,199,254,740,991 or ~9 quadrillion).
In brief, if the value being modulo-ed by is a constant known in advance (which is the case you describe), a sequence of bit manipulation instructions that performs the equivalent of a generic modulo operation but faster can be substituted. The length of that sequence depends on the specific constant used.
If the value being modulo-ed by is not known in advance, I'd suspect that all the modulo operations would take the same time because the same sequence of instructions would be used for all cases. The register width for modern computers is 64-bits, meaning that the same sequence of operations on values would take the same time up for values up to a maximum value of 2^64 (unsigned).