monitor: switch to non-blocking HTTP device checks
commit333e0716fc80027d38d5323353bd9b2de2813886
authorEric Wong <normalperson@yhbt.net>
Tue, 4 Sep 2012 23:21:40 +0000 (4 23:21 +0000)
committerEric Wong <normalperson@yhbt.net>
Sat, 10 Aug 2013 07:03:23 +0000 (10 07:03 +0000)
treeecb345ed3b6c0dff555c115a16fcfa0783fe6a26
parentd8cd470b66ba015b0e55706a3d3ec5fbddc1c6e0
monitor: switch to non-blocking HTTP device checks

Net::HTTP::NB is usable with Danga::Socket and may be used to
make HTTP requests in parallel.

The new connection pool supports persistent connection pooling
similar to LWP::ConnCache.  Total connection capacity is
enforced to prevent out-of-FD situations on the workers.

Unlike LWP::ConnCache, MogileFS::ConnectionPool is designed for
use with concurrent, active connections.  It also supports
queueing (when any enforced capacity or system limits are
reached) and relies on Danga::Socket for scheduling queued
connections.

In addition to total capacity limits, MogileFS::ConnectionPool
also supports limiting concurrency on a per-destination basis to
avoid potentially overloading a single destination.

Currently, we limit ourselves to 20 connections from a single
worker (matching the old LWP limit) and also limit ourselves
to 20 connections to a single host (again matching our previous
LWP behavior).
MANIFEST
lib/MogileFS/Connection/HTTP.pm [new file with mode: 0644]
lib/MogileFS/Connection/Poolable.pm [new file with mode: 0644]
lib/MogileFS/ConnectionPool.pm [new file with mode: 0644]
lib/MogileFS/Host.pm
t/http.t [new file with mode: 0644]