A3 · Shipping and Operating AI
Batching and the Tail
Trade batch size against the wait, and read a system by its slowest users.
One person gets fifty tokens a second from the card. Thirty two people at once get about thirty six each, from the same card. Serving many costs barely more than serving one.