egghelp/eggheads community

Parsing webpages made easy

2007-05-03T14:25:21-04:00

$text doesn't exist, replace 'args' with 'text'.

Statistics: Posted by Sir_Fz — Thu May 03, 2007 2:25 pm

Re: Parsing webpages made easy

2007-05-03T11:21:06-04:00

The Script works for my shell @ home. Now I want my eggdrop to post this Info by typing !test.

I Made this, but it doesnt work

Code:

#!/bin/sh# This line continues for Tcl, but is a single line for 'sh' \exec tclsh8.4 "$0" ${1+"$@"}package require tdompackage require httpset url "http://www.url.de/"set page [::http::data [::http::geturl $url]]set doc [dom parse -html $page]set root [$doc documentElement]set node [$root selectNodes {//table[@cellspacing=0]/tr[1]/td[1]}]set text [[[lindex $node 0] childNodes] nodeValue]bind pub - !test testerproc tester { nick uhost hand chan args } {putserv "PRIVMSG $chan : $text"}

I got following error msg:
Tcl error [tester]: can't read "text": no such variable

Probably the variable text is not committed,
can somebody help me please? thanks

EDIT:
got it,

Code:

bind pub - !test thefunctionproc thefunction { nick uhost hand chan rest } {package require tdompackage require httpset url "http://www.url.de/"set page [::http::data [::http::geturl $url]]set doc [dom parse -html $page]set root [$doc documentElement]set node [$root selectNodes {//table[@cellspacing=0]/tr[1]/td[1]}]set text [[[lindex $node 0] childNodes] nodeValue]putserv "PRIVMSG $chan : $text"}

Statistics: Posted by karodde — Thu May 03, 2007 11:21 am

Parsing webpages made easy

2007-04-01T17:51:58-04:00

some more working examples would be greatly appreciated

Statistics: Posted by johne — Sun Apr 01, 2007 5:51 pm

Parsing webpages made easy

2006-11-16T06:17:48-04:00

it's not tclDOM, it's tDOM; TclDOM is completely different package that indeed requires TclXML; tDOM does not require any XML package as it itself contains XML engine

Statistics: Posted by demond — Thu Nov 16, 2006 6:17 am

Parsing webpages made easy

2006-10-02T14:48:28-04:00

Assuming your shell and your local machine are running the same OS on a similar platform (eg, running freebsd on a 80868 based machine), you can compile the module statically if necessary. You can find out what libs it depends on when compiled dynamically (the usual/default method) by running 'ldd modulename' on it. If your shell isn't running on the same platform, you'll have to cross-compile, which is beyond my own experience, so I'm not much help there.

Statistics: Posted by rosc2112 — Mon Oct 02, 2006 2:48 pm

Parsing webpages made easy

2006-10-02T10:20:09-04:00

Hello all

I am interested in trying this tclDOM, but my shell doesnt support compiling..
So, is there anything I should know when I compile it on my own machine and then copy to the shell?

Like, should the shell have all those libxml, etc installed? And if yes, how can I find out which modules are installed?

Lots of questions and too few answers ( I checked many websites regarding this, but couldn't find, sort of, basic info )

Statistics: Posted by salkkus — Mon Oct 02, 2006 10:20 am

Parsing webpages made easy

2006-08-16T03:33:45-04:00

man thats awesome, can someone make a little how-to guide to finding out the nessery things to phrase something off a webpage? finding out table size td row div and such using dom inspector?

that would be your basic HTML primer

Statistics: Posted by demond — Wed Aug 16, 2006 3:33 am

Parsing webpages made easy

2006-06-13T19:24:38-04:00

man thats awesome, can someone make a little how-to guide to finding out the nessery things to phrase something off a webpage? finding out table size td row div and such using dom inspector?

Statistics: Posted by flashy — Tue Jun 13, 2006 7:24 pm

Parsing webpages made easy

2006-05-20T23:14:56-04:00

Not talking about executing any utimer event every second, or every few seconds. We are talking specifically about calling a utimer event that makes a call to the 'update' command. Update will have to process any and all info in the event queue (not just what you think may be there), and it will not return until that has been completed, again as I stated earlier, negating the purpose of async mode altogether. And no, newer versions of eggdrop would not be blocked as their event queue is updated every iteration of the main program loop, so having a utimer event on those bots would be pointless as the event queue would most certainly be empty already. I have tried this on versions that required manual updating, many a time, and it did indeed result in severe performance degredation.
Hence the production of the egghttp.tcl script, and also a patch for certain versions as an alternative solution.

Statistics: Posted by strikelight — Sat May 20, 2006 11:14 pm

Parsing webpages made easy

2006-05-20T11:31:03-04:00

The performance. Well. To call a proc/command-update every seconds doesn`t cause a that high cpu usage. I tested it. Important is there should only run one utimer at once. If this timer is started on evey script load or reash again and again this will cause a high cpu usage for sure.

Update is simple. Do i have work to do? No? -> Return. This doesn`t take that long. And if he has work to do this will block the bot just that long the newer versions of eggdrop blocked aswell too.

Statistics: Posted by sKy — Sat May 20, 2006 11:31 am

Parsing webpages made easy

2006-05-19T23:25:17-04:00

Yes, you could call a utimer event for 'update' every second for certain versions of eggdrop/tcl combination, however, you will be causing your bot to consume heavy cpu usage by doing so. However, also doing update every second will render your bot inoperable for extended amounts of time, negating the purpose of the async mode altogether.
Performance, needless to say, would be sluggish using such a method,
and not be very practical. Calling 'update' every second (or even every few seconds) is not a viable solution.

Statistics: Posted by strikelight — Fri May 19, 2006 11:25 pm

Parsing webpages made easy

2006-05-19T05:36:19-04:00

But you always could process the eventloop yourself. Call the update proc every second via utimer.

Statistics: Posted by sKy — Fri May 19, 2006 5:36 am

Parsing webpages made easy

2006-05-13T17:28:06-04:00

I didn't say the http package, I said older versions of Tcl itself and thus different eggdrop versions. (There is no event loop to facilitate the required callback mechanism of the http package in these versions)

Statistics: Posted by strikelight — Sat May 13, 2006 5:28 pm

Parsing webpages made easy

2006-05-13T13:24:24-04:00

refering to the header of http.tcl v2.5.1 from 2005/01/06, http supports callback (truely async) since version 2.1. I found 2.4.2 which was from 2004/04/05. You can probably guess how old 2.1 is...
The biggest diffrence between http and egghttp is, that egghttp uses connect and control and http uses of course socket and fileevent.

Statistics: Posted by De Kus — Sat May 13, 2006 1:24 pm

Parsing webpages made easy

2006-05-12T23:56:49-04:00

How can I do this with egghttp?
don't use egghttp, it is severily outdated; it had its use long time ago, when Tcl still didn't have the built-in http package, which is superior in any way to egghttp

No, egghttp was written even when the http package for Tcl existed.
However the http package is severely bloated for the function of simply grabbing a web page, not to mention it does not provide async connection across all Tcl versions, and thus all eggdrop versions. Egghttp is still very much a practical utility today.

Statistics: Posted by strikelight — Fri May 12, 2006 11:56 pm