Discussion:
On which thread does bufferevent filter run, and when?
(too old to reply)
Yucong Sun
2014-01-26 07:58:21 UTC
Permalink
Raw Message
Hi there,

So I need to implement a zlib bufferevent filter to transparent
compress the data passing through a bufferevent. and but I want the
zlib work to be done in a separate thread to offload main loop.

I currently call bufferevent_write() on main thread, would the filter
code executed immediately? if I call bufferevent_write() from
another thread, would the code be executed over that thread instead?
(locking is all properly setuped).

Also.. how does this play out with flush? is the filter code executed
only when calling flush?

Cheers.
***********************************************************************
To unsubscribe, send an e-mail to ***@freehaven.net with
unsubscribe libevent-users in the body.
Nick Mathewson
2014-01-29 20:42:27 UTC
Permalink
Raw Message
Post by Yucong Sun
Hi there,
So I need to implement a zlib bufferevent filter to transparent
compress the data passing through a bufferevent. and but I want the
zlib work to be done in a separate thread to offload main loop.
I currently call bufferevent_write() on main thread, would the filter
code executed immediately? if I call bufferevent_write() from
another thread, would the code be executed over that thread instead?
(locking is all properly setuped).
The thread in which callbacks happen isn't currently specified by the
documentation. So it is allowed to change in future versions.

In practice right now, I believe that filters are greedy: they consume
all input as soon as you give it to them, and they process data from
the underlying bufferevent as soon as it's read. They do this in the
same thread where the they receive the data. (This is affected by
watermarks on the underlying bufferevent.)
Post by Yucong Sun
Also.. how does this play out with flush? is the filter code executed
only when calling flush?
flushing on a filtered buffferevent makes the filter get called
immediately with the provided mode. Ordinarily, you use it to finish
or clean-up a stream. The filter code is also called with
BEV_FINISHED or BEV_NORMAL as you add data to the buffer.


hope this helps,
--
Nick
***********************************************************************
To unsubscribe, send an e-mail to ***@freehaven.net with
unsubscribe libevent-users in the body.
Yucong Sun
2014-01-29 21:52:08 UTC
Permalink
Raw Message
Post by Nick Mathewson
Post by Yucong Sun
Hi there,
So I need to implement a zlib bufferevent filter to transparent
compress the data passing through a bufferevent. and but I want the
zlib work to be done in a separate thread to offload main loop.
I currently call bufferevent_write() on main thread, would the filter
code executed immediately? if I call bufferevent_write() from
another thread, would the code be executed over that thread instead?
(locking is all properly setuped).
The thread in which callbacks happen isn't currently specified by the
documentation. So it is allowed to change in future versions.
In practice right now, I believe that filters are greedy: they consume
all input as soon as you give it to them, and they process data from
the underlying bufferevent as soon as it's read. They do this in the
same thread where the they receive the data. (This is affected by
watermarks on the underlying bufferevent.)
I don't quite understand this, what "callback" were you referring to?
When I call bufferevent_write() on a filtered socket based
bufferevent, would the filters A) get executed recurisvely on the
current thread/stack ? Or would B) it internally add some more events
to the loop and executes the filter code on next callback?

A) is better for me since I can arrange my call to bufferent_write()
onto another thread, thus offloading main loop. But for B) I can't
think of a way of doing that.

Thanks for the reply!
Post by Nick Mathewson
Post by Yucong Sun
Also.. how does this play out with flush? is the filter code executed
only when calling flush?
flushing on a filtered buffferevent makes the filter get called
immediately with the provided mode. Ordinarily, you use it to finish
or clean-up a stream. The filter code is also called with
BEV_FINISHED or BEV_NORMAL as you add data to the buffer.
hope this helps,
--
Nick
***********************************************************************
unsubscribe libevent-users in the body.
***********************************************************************
To unsubscribe, send an e-mail to ***@freehaven.net with
unsubscribe libevent-users in the body.
Nick Mathewson
2014-02-03 15:21:44 UTC
Permalink
Raw Message
Post by Yucong Sun
Post by Nick Mathewson
Post by Yucong Sun
Hi there,
So I need to implement a zlib bufferevent filter to transparent
compress the data passing through a bufferevent. and but I want the
zlib work to be done in a separate thread to offload main loop.
I currently call bufferevent_write() on main thread, would the filter
code executed immediately? if I call bufferevent_write() from
another thread, would the code be executed over that thread instead?
(locking is all properly setuped).
The thread in which callbacks happen isn't currently specified by the
documentation. So it is allowed to change in future versions.
In practice right now, I believe that filters are greedy: they consume
all input as soon as you give it to them, and they process data from
the underlying bufferevent as soon as it's read. They do this in the
same thread where the they receive the data. (This is affected by
watermarks on the underlying bufferevent.)
I don't quite understand this, what "callback" were you referring to?
When I call bufferevent_write() on a filtered socket based
bufferevent, would the filters A) get executed recurisvely on the
current thread/stack ? Or would B) it internally add some more events
to the loop and executes the filter code on next callback?
I believe that, in the current code, the filter callbacks (that is,
the functions that you provided to implement the filter) get called on
the stack.
--
Nick
***********************************************************************
To unsubscribe, send an e-mail to ***@freehaven.net with
unsubscribe libevent-users in the body.
Loading...