egghelp/eggheads community

Parsing HTML

2003-10-09T00:08:45-04:00

You really don't want to use exec on anything slow. Use open and use a pipe. Then you can read from the fd that open returns.

e.g.:
set fd [open "|wget http://www.cnn.com/" r]

Statistics: Posted by BarkerJr — Thu Oct 09, 2003 12:08 am

Parsing HTML

2003-10-08T21:52:57-04:00

the problem with lynx is that there is no real timeout
i had some of my scripts with lynx too, but when the webpage to slow etc the exec just took some minutes -> bot times out....

what would be the best method here, does fetch or wget have a good timeout?

Statistics: Posted by Darkj — Wed Oct 08, 2003 9:52 pm

Parsing HTML

2003-10-05T14:39:49-04:00

Are there any good tutorials on the subject parsing files/html
???
if there is i would reallt like an url.
or if someone could help me that would be good too.
prefrably real time like chat on ICQ or IRC..

Statistics: Posted by Syntax — Sun Oct 05, 2003 2:39 pm

Parsing HTML

2003-10-04T00:50:32-04:00

Code:
proc status_callback {sock} {   global inifilestatus  set data [egghttp:data $sock]   egghttp:cleanup $sock   regsub -all "\n" $data "" data   regsub -all "
" $data "\n" data   foreach line [split $data \n] {     if {[string match "*Deception*Up*" $line]} {       set item [join [lindex [split $line] 1]]      ini_write $inifilestatus server status $item    }    if {[string match "*Deception*Down*" $line]} {       set item [join [lindex [split $line] 1]]      ini_write $inifilestatus server status $item    }    return 0  } } 
Ok so when it checks the status, its supposed to use my ini_write proc. But the problem is, is its not even getting the line, I'm not sure If its even getting the page properly.

To call it in another script, i do this:
Code:
set sock [egghttp:geturl chronicle.ubi.com/ status_callback]set server_status [ini_read $inifilestatus server status]
But thats totally wrong as I've never ever used this http stuff in a script before so i'm totally lost.

Because egghttp works in non-blocking mode (ie. it won't freeze up your bot), your 'set server_status' line will most likely be called before the callback proc gets called.. If you want to set anything, it will have to be within the callback proc itself... To verify it is working, add 'putlog' statements in your callback proc, and stay in your bot's partyline while the bot tries to connect to the site and watch what is going on.

Statistics: Posted by strikelight — Sat Oct 04, 2003 12:50 am

Parsing HTML

2003-10-03T18:39:18-04:00

anyone able to help?

Statistics: Posted by Darkj — Fri Oct 03, 2003 6:39 pm

Parsing HTML

2003-10-03T08:20:11-04:00

Code:

proc status_callback {sock} {   global inifilestatus  set data [egghttp:data $sock]   egghttp:cleanup $sock   regsub -all "\n" $data "" data   regsub -all "
" $data "\n" data   foreach line [split $data \n] {     if {[string match "*Deception*Up*" $line]} {       set item [join [lindex [split $line] 1]]      ini_write $inifilestatus server status $item    }    if {[string match "*Deception*Down*" $line]} {       set item [join [lindex [split $line] 1]]      ini_write $inifilestatus server status $item    }    return 0  } }

Ok so when it checks the status, its supposed to use my ini_write proc. But the problem is, is its not even getting the line, I'm not sure If its even getting the page properly.

To call it in another script, i do this:

Code:

set sock [egghttp:geturl chronicle.ubi.com/ status_callback]set server_status [ini_read $inifilestatus server status]

But thats totally wrong as I've never ever used this http stuff in a script before so i'm totally lost.

Statistics: Posted by Darkj — Fri Oct 03, 2003 8:20 am

Parsing HTML

2003-10-03T04:28:13-04:00

the problem with lynx is that there is no real timeout
i had some of my scripts with lynx too, but when the webpage to slow etc the exec just took some minutes -> bot times out....

Statistics: Posted by GodOfSuicide — Fri Oct 03, 2003 4:28 am

Parsing HTML

2003-10-03T03:34:02-04:00

Oh and this is the line on the page that I am trying to get

DeceptionUp

if you're on a *nix system with lynx, you can use the simple way I often use:

Code:

file delete -force $usertempset fs [open $usertemp w]puts $fs [exec $lynx -preparsed -dump $rub]close $fs

and you don't have any html code in $fs

Statistics: Posted by CrazyCat — Fri Oct 03, 2003 3:34 am

Parsing HTML

2003-10-02T22:21:15-04:00

Oh and this is the line on the page that I am trying to get

DeceptionUp

If that's the line you are trying to get, then your string match is invalid..

Code:

if {[string match "*Deception Up*" $line]} {

should be:

Code:

if {[string match "*Deceoption*Up*" $line]} {

Statistics: Posted by strikelight — Thu Oct 02, 2003 10:21 pm

Parsing HTML

2003-10-02T21:33:42-04:00

Oh and this is the line on the page that I am trying to get

DeceptionUp

Statistics: Posted by Darkj — Thu Oct 02, 2003 9:33 pm

Parsing HTML

2003-10-02T21:29:53-04:00

Code:

proc status_callback {sock} {   global inifilestatus  set data [[b]egghttp[/b]:data $sock]   [b]egghttp[/b]:cleanup $sock   regsub -all "\n" $data "" data   regsub -all "
" $data "\n" data   foreach line [split $data \n] {     if {[string match "*Deception Up*" $line]} {       set item [join [lindex [split $line] 1]]    }    return 0  } }

That probably looks totally wrong, i'm not sure it even works, or if I'm calling it right.

The page has multiple words "Deception" in it, so I'm hoping that *Deception Up* would work, then just write another if statement for Down servers.

But my problem is now, I can't get the data from that proc back to a var so I can use the !status command.

So now I just need help fixing that messy code, and setting the var in the !status proc. Thanks

Statistics: Posted by Darkj — Thu Oct 02, 2003 9:29 pm

Parsing HTML

2003-10-02T20:34:03-04:00

Ok, I can't even get this connect stuff down, how do I connect to the site, search for the line, take arg 1 from that line.

Statistics: Posted by Darkj — Thu Oct 02, 2003 8:34 pm

Parsing HTML

2003-10-02T11:10:56-04:00

If you catch the html page, so it's allright.
Just put each line in $line and make

Code:

set present [string first "Deception" $line]if {$present != 0} {# it's ok, you have your line} else {# it is not the good line}

Statistics: Posted by CrazyCat — Thu Oct 02, 2003 11:10 am

Parsing HTML

2003-10-02T08:20:40-04:00

Basically, I have zero idea how to even start this. I wanna make a server status script for my bot that would read off a webpage (http://chronicle.ubi.com/). Now if you view that webpage, you will see on the right about 10 servers. I would like to be able to get the status for the one server called Deception. So basically i just wanna see if beside Deception, if it says UP or Down.

I have got the http script loaded, but after trying the parsing I get totally lost, if someone can provide some insight, that would be greatly appreciated. Thanks.

Statistics: Posted by Darkj — Thu Oct 02, 2003 8:20 am