egghelp/eggheads community

Non-blocking use of http::geturl returns results very slow ?

2008-03-09T15:58:58-04:00

@incith:
Errors generated in event-driven code is reported through the bgerror mechanism (since there is no "toplevel" caller which to return the error).

A very simple way of logging these errors would be something like this:

Code:

proc bgerror {message} { putlog "bgerror: $message" putlog " - errorInfo: $::errorInfo" putlog " - errorCode: $::errorCode"}

Also see http://www.tcl.tk/man/tcl8.0/TclCmd/bgerror.htm

Statistics: Posted by nml375 — Sun Mar 09, 2008 3:58 pm

Non-blocking use of http::geturl returns results very slow ?

2008-03-09T05:53:18-04:00

What is the cpu consumption of an eggdrop applied with that patch compared to an eggdrop without it applied?

Statistics: Posted by strikelight — Sun Mar 09, 2008 5:53 am

Non-blocking use of http::geturl returns results very slow ?

2008-03-09T03:25:30-04:00

This is why it is a good thing to apply the patch above to your eggdrops source.

@arfer: Only one is using callback in your example, the one with -command.

I have begun adding 'variable callback' to my scripts for those of you that do want to enable callback procedures. I did notice slowdowns still, but nothing too painful. For others, it is very simple and basically the same as above to add optional callback, something like:

Code:

set callback 0/1  # disable/enableif {$callback >= 1} {  set token [::http::geturl -command httpCallbackProcedure BlahBlahBlah]} else {  set token [::http::geturl BlahBlahBlah]  httpCallbackProcedure $token}

One problem I found was when errors occur in the callback, there is no output or logging. Even if you call non-existant variables, it just stops at the failing line of code, but doesn't let you know why it stopped.

Statistics: Posted by incith — Sun Mar 09, 2008 3:25 am

Non-blocking use of http::geturl returns results very slow ?

2008-03-09T01:37:13-04:00

Not to mention if you have other scripts loaded that require accurate timing or fast execution as well, that they will be unable to do so when the bot is executing that blocked code.

Statistics: Posted by strikelight — Sun Mar 09, 2008 1:37 am

Non-blocking use of http::geturl returns results very slow ?

2008-03-08T23:18:28-04:00

The difference in these two cases, are that in pFetch1, the call to ::http::geturl will block until the HTTP transaction is completed and token is ready, and not relying on Tcl_DoOneEvent() being called; whereas in pFetch2, the call to ::http::geturl will return immediately, and launch an event-driven job "in background".

What might be worth keeping in mind, is that with pFetch1 your eggdrop will become unresponsive until the HTTP transaction is completed. Be careful when selecting timeout values so your bot won't get disconnected (ping timeout)

Statistics: Posted by nml375 — Sat Mar 08, 2008 11:18 pm

Non-blocking use of http::geturl returns results very slow ?

2008-03-08T21:58:27-04:00

Perhaps this is relevent here

I found the same difficulty with execution times and as a solution took to declaring the callback proc outside of the geturl statement to speed things up considerably, as per the following :-

Two styles of using callback

pFetch1 proc with call to pCallback outside of the geturl statement executes quite quickly

pFetch2 proc with the call to pCallback within the geturl statement is comparatively slow to execute

no idea why

proc pFetch1 {address} {
set agent [::http::config -useragent "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"]
if {[catch {set token [::http::geturl $address -timeout 10000]}]} {
putlog "cant connect to $address"
} else {pCallback $token}
return 0
}

proc pFetch2 {address} {
set cmd pCallback
set agent [::http::config -useragent "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"]
if {[catch {set token [::http::geturl $address -command $cmd -timeout 10000]}]} {
putlog "cant connect to $address"
}
return 0
}

proc pCallback {token} {
# callback code here
}

Statistics: Posted by arfer — Sat Mar 08, 2008 9:58 pm

Non-blocking use of http::geturl returns results very slow ?

2008-03-07T23:49:35-04:00

I jumped the gun a bit, it appears he attached a patch file which does more than just undef USE_TCL_EVENTS. I will give it a go later on and see if it actually does make Tcl events work properly, and fast. Second post updated with links.

Update: The patch fixes everything properly.

Statistics: Posted by incith — Fri Mar 07, 2008 11:49 pm

Non-blocking use of http::geturl returns results very slow ?

2008-03-06T23:11:43-04:00

I'd be interested in knowing how egghttp.tcl compares to the initial poster's script.

Statistics: Posted by strikelight — Thu Mar 06, 2008 11:11 pm

Non-blocking use of http::geturl returns results very slow ?

2008-03-06T22:58:33-04:00

I would say that's a fair assumption. In my tests, the callback routine will get an incorrect token, which essentially breaks http again, despite improving/fixing the events as well in a way.

Basically, you will call ::http::geturl, the callback will get ::http::1 and then ::geturl returns ::http::2, which is really bizarre. After recompiling eggdrop back to normal this behavior disappeared. So applying the 'patch' above will both fix and break http callbacks. This lead to some strange things, such as the callback would be trying to work on the previous URL/token, because the ::http::geturl is returning a different token. I even tried to increment the token, but then the callback didn't have any data to parse because ::http::geturl hadn't actually returned the new token yet.

Well, hopefully this will be fixed one day as a whole and we can use fast non-blocking events in eggdrop, because the callback was pretty cool even while it was broken.

Statistics: Posted by incith — Thu Mar 06, 2008 10:58 pm

Non-blocking use of http::geturl returns results very slow ?

2008-03-06T13:15:52-04:00

I'm pretty sure by undefining that, you are essentially getting rid of non-blocking Tcl operation from eggdrop. Hence, you may as well just use the blocking method of the http::geturl command.

Statistics: Posted by strikelight — Thu Mar 06, 2008 1:15 pm

Non-blocking use of http::geturl returns results very slow ?

2008-12-28T08:03:24-04:00

It is related to this: http://www.eggheads.org/pipermail/bugs/ ... 00960.html

There is a patch which he attaches, located here: http://eggheads.org/pipermail/bugs/atta ... levent.bin (mirror)

Update: The patch above needs to be applied in order for things to work properly. After applying the above patch, non-blocking http gets work properly and normally.

Statistics: Posted by incith — Thu Mar 06, 2008 7:15 am

Non-blocking use of http::geturl returns results very slow ?

2007-05-06T01:51:17-04:00

Can anyone offer any suggestions or reasons why a non-blocking style http get (using the http package) ala demonds rssnews script would take such a long time in comparison to a regular get/wait retrieve ?

I've timed the non-blocking method at ~15-20 seconds whereas a regular (blocking) read/wait only takes ~2-3 seconds.

Not overly sure what causes the delay. Is this to be expected/normal ?

Statistics: Posted by madwoota — Sun May 06, 2007 1:51 am