egghelp/eggheads community

Scraping pages while logged in

2009-05-02T14:37:00-04:00

remember about cookies, most of login "systems" uses session IDs on the server side and session cookies on the client side (web browser)

some hints,
http://wiki.tcl.tk/3828
http://wiki.tcl.tk/17611

Statistics: Posted by tomekk — Sat May 02, 2009 2:37 pm

Scraping pages while logged in

2009-05-01T18:31:50-04:00

Yes, it's just a regular web page, but unfortunately, you have to be logged in to see it. I can log in and read it just fine, but I wasn't sure if tcl could be used to do the same.

Statistics: Posted by pen — Fri May 01, 2009 6:31 pm

Scraping pages while logged in

2009-05-01T15:34:29-04:00

Most likely, yes.

I'm assuming you're talking 'bout scraping text off a webpage. "All" you'll have to do, is replicate the login procedure, and you should be all done. Since logons can be done in a multitude of different fashions, that's as good as the answer gets.

Statistics: Posted by nml375 — Fri May 01, 2009 3:34 pm

Scraping pages while logged in

2009-05-01T14:58:28-04:00

Hey, I'm trying to figure out how to scrape some text and output it to my irc channel. But the problem is that the text I want is on a page that's only accessible while I'm logged in. Is there a way to do this with tcl?

Statistics: Posted by pen — Fri May 01, 2009 2:58 pm