2014-05-18 16:43:29 UTC
A while ago I asked this question on Stackoverflow (
) but didn't have much luck with getting any replies, so I thought I'd
ask on the mailing list. Here's the question:
I am writing an HTTP reverse-proxy in C using Libevent and I would like
to implement multithreading to make use of all available CPU cores. I
had a look at this example:
In this example it appears that one thread is used for the full duration
of a connection, but for HTTP 1.1 I don't think this will be the most
effective solution as connections are kept alive by default after each
request so that they can be reused later. I have noticed that even one
browser panel can open several connections to one server and keep them
open until the tab is closed which would immediately exhaust the thread
pool. For an HTTP 1.1 proxy there will be many open connections but only
very few of them actively transferring data at a given moment.
So I was thinking of an alternative, to have one event base for all
incoming connections and have the event callback functions delegate to
worker threads. This way we could have many open connections and make
use of a thread only when data arrives on a connection, returning it
back to the pool once the data has been dealt with.
My question is: is this a suitable implementation of threads with Libevent?
Specifically -- is there any need to have one event base per connection
as in the example or is one for all connections sufficient?
Also -- are there any other issues I should be aware of?
Currently the only problem I can see is with burstiness, when data is
received in many small chunks triggering many read events per HTTP
response which would lead to a lot of handing-off to worker threads.
Would this be a problem? If it would be, then it could be somewhat
negated using Libevent's watermarking, although I'm not sure how that
works if a request arrives in two chunks and the second chunk is
sufficiently small to leave the buffer size below the watermark. Would
it then stay there until more data arrives?
Also, I would need to implement scheduling so that a chunk is only sent
once the previous chunk has been fully sent.
The second problem I thought of is when the thread pool is exhausted,
i.e. all threads are currently doing something, and another read event
occurs -- this would lead to the read event callback blocking. Does that
matter? I thought of putting these into another queue, but surely that's
exactly what happens internally in the event base. On the other hand, a
second queue might be a good way to organise scheduling of the chunks
without blocking worker threads.