egghelp/eggheads community

http geturl 'limiting'?

2004-06-16T15:02:12-04:00

I changed 1.0 to 1.1 just like you and added puts $s "Connection: close" right below it.

Then I did:

set token [http::geturl http://localhost/test.html -headers

]
puts [http::data $token]

It worked for me.

Statistics: Posted by stdragon — Wed Jun 16, 2004 3:02 pm

http geturl 'limiting'?

2004-06-16T01:45:58-04:00

Ic, yeah looking at the http tcl it uses 1.0.. I changed 1 occurance of 1.0 to 1.1 in \lib\tcl8.4\http2.4\http.tcl

You might have to add a bit to it too, like the Host header (I think) and Connection: close.

Where would i change this in the http tcl?

What i did is this..
change: puts $s "$how $srvurl HTTP/1.0" to puts $s "$how $srvurl HTTP/1.1"
added line putlog "$key: $value" in the -headers
added line puts $s "Connection: close" immediately after the the foreach for the -headers, this succesfully send info to the consol, so i know its processing the range: bytes. However it still transfers the entire file....

Im totaly lost, is their another way? user mentioned using -bloacksize and -progress? (but as i said before i don't full understand how to implement -progress)

Statistics: Posted by DayCuts — Wed Jun 16, 2004 1:45 am

http geturl 'limiting'?

2004-06-15T23:58:56-04:00

I'm pretty sure Range will work with any file, but you have to be using http 1.1. Normally the http package uses http 1.0. So you have to modify it a bit. Find the 1.0 and change it to 1.1. You might have to add a bit to it too, like the Host header (I think) and Connection: close.

Statistics: Posted by stdragon — Tue Jun 15, 2004 11:58 pm

http geturl 'limiting'?

2004-06-15T16:47:27-04:00

It's -headers "Range bytes=0-10"
The http package will properly format it with a : when it sends to server

I tried this also. : )

As i mentioned in my first post, this is a .csv file i am accesing, not an average html page, this is probably why the html headers are ignored.

Statistics: Posted by DayCuts — Tue Jun 15, 2004 4:47 pm

http geturl 'limiting'?

2004-06-15T16:45:42-04:00

I tried using -headers "Range: bytes=0-10" in the http geturl but it was unsuccesful so i assume the server doesn't allow byte rangers.

It's -headers "Range bytes=0-10"
The http package will properly format it with a : when it sends to server

Statistics: Posted by strikelight — Tue Jun 15, 2004 4:45 pm

http geturl 'limiting'?

2004-06-15T16:42:38-04:00

yeah..that's a good idea as you'll also avoid wasting lots of bandwidth transfering data you don't want/need.

This is what i was wanting to do in the first place.. not transfer un-needed data.

I tried using -headers "Range: bytes=0-10" in the http geturl but it was unsuccesful so i assume the server doesn't allow byte rangers.

Also spent awhile playing with your method, user. It made almost no difference than my original code, accept that it organised the data a little different, still transfered the entire file. Which is what i am trying to avoid.

Code:

  if {$text == ""} {    set token [::http::geturl $itrurlg]    set content [::http::data $token]    ::http::cleanup $content    regsub -all -- {,} $content { } cont2    list $cont2    # for debug purposes    putlog "[lrange $cont2 0 end]"    return 0  } else {

Sample of content

Code:

Date,Rating,Latency,Loss13:25 6/15/2004,85,145,213:20 6/15/2004,85,144,2

the raw data is updated every 5 minutes, the only information i need is line no. 2

Statistics: Posted by DayCuts — Tue Jun 15, 2004 4:42 pm

http geturl 'limiting'?

2004-06-15T15:54:08-04:00

Another thing is to use the regular form and send the http byte range request header.

yeah..that's a good idea as you'll also avoid wasting lots of bandwidth transfering data you don't want/need.

Statistics: Posted by user — Tue Jun 15, 2004 3:54 pm

http geturl 'limiting'?

2004-06-15T15:36:06-04:00

Another thing is to use the regular form and send the http byte range request header.

It's like:

Range: bytes=0-100

Statistics: Posted by stdragon — Tue Jun 15, 2004 3:36 pm

http geturl 'limiting'?

2004-06-15T15:19:55-04:00

I took another look at that manual page and noticed another option that seems more suitable for your needs: -handler

Code:

package require httpproc yourCallback {sock tok} {while {[gets $sock line]==0} continue;# read untill we get a non-empty line or eofputs $line;# you probably want to do something else here :)http::reset $tok;# i'm not sure this one's needed, but I included it anyway :Phttp::cleanup $tok;# this one's mandatory}http::geturl http://www.your.url/ -handler yourCallback -command nah -timeout 20000

the -command thing is just a hack to avoid blocking (and nasty error messages because of the cleanup performed before geturl returns )

Statistics: Posted by user — Tue Jun 15, 2004 3:19 pm

http geturl 'limiting'?

2004-06-15T14:21:44-04:00

Hmm, can you give me an example of the syntax for this? This is what i had before my original post, however it still transfers the entire page of data. This is my first script using the http package so i am sure i do not understand the proper usage and syntax of it all.

Code:

set token [::http::geturl $itrurlg -blocksize 50]

I have read the -progress help, however i still don't quite understand proper use and syntax. So am a little lost on how to combine the two. The information after the first break is not at all needed, so stopping after the first block is what i am trying to do.

Statistics: Posted by DayCuts — Tue Jun 15, 2004 2:21 pm

http geturl 'limiting'?

2004-06-15T12:00:01-04:00

-blocksize combined with -progress should do the trick.

Statistics: Posted by user — Tue Jun 15, 2004 12:00 pm

http geturl 'limiting'?

2004-06-15T08:48:56-04:00

I am writing a tcl script to return certain information from a .csv on a website using the http package. Is there a way to limit the amount of bytes the script will read from the file. This file can become quite large, as such using normal http geturl can be slow to respond (due to reading the entire file). It would be much more efficient if i could just read the first XX bytes or X lines, since the most updated information that the script will be using is in the first few lines.

Any and all help much appreciated, thanks in advance.

Statistics: Posted by DayCuts — Tue Jun 15, 2004 8:48 am