Reduce maxResponseDelay from 1000 ms to 100 ms.
When there are lots of polling requests and not enough downstream data
to write immediately to all of them, the ones that get starved of data
hold a TCP connection open for maxResponseDelay. I was getting
http: Accept error: accept tcp 127.0.0.1:8080: accept4: too many open files; retrying in 5ms
at high downstream rates.
Probably, what we should do is instead keep a map of client ID →
in-progress HTTP requests, and make it so that when a new request for
the same client ID arrives, we immediately close the in-progress
request. That way, we could set a nice long maxResponseDelay (for better
latency and less frequent polling), while at the same time limiting
ongoing connections.