Handling Overload

http://ferd.ca/handling-overload.html

• идея статьи - из спора о необходимости иметь бесконечный mailbox для процесса (все равно ограничение - в размере оперативки)
• ?Критика flow control остается в preemptive языках (Go). Кооперативные языки (Akka, Node.js) имеют неявные механизмы (большая нагрузка на CPU рано блокируется раньше)
• naive system - tasks are done asynchronously. No control, you hope for the best
• Usually, one of the part is not parallelized -> bottleneck -> problem with the size of the mailbox
• backpressure - default way in most languages, because function call is synced (you do smth, and return smth when everything is done)
• all the servers have some limit of concurrent requests
• Erlang is asynced by default (multi-process communication) (???not true for OTP?)
• Naive backpressure - make all calls to bottleneck synced.

  

• Problem: limitation of incoming connection is still required. Add more workers - how to increase limits? Not all the requests have the same cost. 90% - 1ms, 5% - 5ms, 5% - 250ms. Pessimistic and safe <-> Optimistic and unsafe.
• pull-base control flow - the same as synced
• Example of ghost request when the sender see ‘sent and accepted TCP request’, while the receiver never logs it. (it didn’t receive it)
• bounded queues: fail fast if queue is full. Problem: queue process is also a limitation (copy message, copy from mailbox to heap, then shed)
• counter-protected process: counter to send request
• Probing for quality and adjusting behaviour:
  • queue time (sum of all the reqs - more avg stats)
  • randomly drop requests (from the queue - shedding, before queue - back-pressure)
  • sometimes you can’t loose messages
• Capacity probes (CPU?memory in system, send buffers of network get filled, etc)
  • circuit breaker - fail if a lot of fails for this service (3 failures during 10 secs - do not send more reqs)
  • drop specific operations (return error)