egghelp/eggheads community

Problem with special characters ² ³ and °

2008-12-28T21:38:12-04:00

My solution would likewise work in a TCL script not confined to display within a non UTF-8 environment.

When eggdrop v1.6.20 is released, eggdrop should become a workable utf-8 environment finally. At that time I'll probably update my irc client, until then... work-around is the name of the game. ^_~

Statistics: Posted by speechles — Sun Dec 28, 2008 9:38 pm

Problem with special characters ² ³ and °

2008-12-28T21:31:51-04:00

Sorry my mistake.

The bot's partyline seems incapable of interpreting/displaying UTF-8 characters by default. No amount of encoding seems to change that, as your post confirms.

My original posts were using a public commands Tclsh and so done through a mIRC bot channel, hence my solution works because mIRC displays UTF-8 by default.

My solution would likewise work in a TCL script not confined to display within a non UTF-8 environment.

Statistics: Posted by arfer — Sun Dec 28, 2008 9:31 pm

Problem with special characters ² ³ and °

2008-12-28T21:15:22-04:00

Works fine for me using partyline Tcl.

My guess is you are not using a utf-8 compliant IRC client or you are in the bot's partyline via telnet.

Should work in DCC CHAT within mIRC or XChat (I'm using mIRC) providing they are set to display utf-8 by default.

Why does it matter what my irc client does? You aren't seeing the bigger picture. What you are trying to regsub, the encoding you set it to, and what you are regsubbing in, it's encoding both matter.

.tcl set mytest [encoding convertto "utf-8" "at xy³ + y² temperature is 14°"]
Tcl: at xyÂ³ + yÂ² temperature is 14Â°
.tcl set mytest2 [encoding convertto "utf-8" "\[°²³\]"]
Tcl: [Â°Â²Â³]
.tcl set testing [regsub -all -- "$mytest2" $mytest "\\\\&"]
Tcl: at xy\Â\³ + y\Â\² temperature is 14\Â\°

This is meant to demonstrate working outside of the bot's internal encoding or system encoding. You can get it to work, you just have to be explicit.

If you check out the unofficial incith google script, you can see this issue causes problems in several places and has numerous work arounds.

Statistics: Posted by speechles — Sun Dec 28, 2008 9:15 pm

Problem with special characters ² ³ and °

2008-12-28T21:06:57-04:00

Works fine for me using partyline Tcl.

My guess is you are not using a utf-8 compliant IRC client or you are in the bot's partyline via telnet.

Should work in DCC CHAT within mIRC or XChat (I'm using mIRC) providing they are set to display utf-8 by default.

Statistics: Posted by arfer — Sun Dec 28, 2008 9:06 pm

Problem with special characters ² ³ and °

2008-12-28T20:52:12-04:00

Probably has more to do with encodings, using eggdrop v1.6.17 and tcl 8.4

.tcl set mytest "at xy³ + y² temperature is 14°"
Tcl: at xy? + y? temperature is 14?
.tcl set testing [regsub -all -- {[°²³]} $mytest "\\\\&"]
Tcl: at xy\? + y\? temperature is 14\?

This is how it works as iso8859-1, those chars aren't represented correctly they get the question mark treatment. But apparently it has worked because the escapes are properly placed. But if we instead use "utf-8"...

.tcl set mytest [encoding convertto "utf-8" "at xy³ + y² temperature is 14°"]
Tcl: at xyÂ³ + yÂ² temperature is 14Â°
.tcl set testing [regsub -all -- {[°²³]} $mytest "\\\\&"]
Tcl: at xyÂ³ + yÂ² temperature is 14Â°

Fails, but I can see them clearly...
The work around of course is to use binary/octal/decimal/hex notation when referencing these characters or using the correct encoding to begin with...

When eggdrop finally supports utf-8, and latin charsets aren't confused with iso8859-1 representations.. Well, at that time all this stuff will probably not need work arounds any longer..

Statistics: Posted by speechles — Sun Dec 28, 2008 8:52 pm

Problem with special characters ² ³ and °

2008-12-28T19:41:16-04:00

Reference my post above, I have since done much reading and muttering under my breath in an attempt to understand the difference between ³ as per a copy/paste from say windows character map and a ³ as generated from the hex notation \xB3. I am non the wiser. Even binary scanning the character shows them to have the same underlying value.

At least I am 50% happy in that I have a solution. Simply build up the regsub pattern BOTH from an explicit copy/paste of the characters themself AND from their implicit hex equivalents.

use --> [regsub -all -- {[°²³\xB0\xB2\xB3]} $varname "\\\\&"]

This is the result when I build up the variable using windows character map for the special characters :-

% set mytest "at xy³ + y² temperature is 14°"
at xy³ + y² temperature is 14°
% return [regsub -all -- {[°²³\xB0\xB2\xB3]} $mytest "\\\\&"]
at xy\³ + y\² temperature is 14\°

This is the result when I build up the variable using hex notation for the special characters :-

% set mytest "at xy\xB3 + y\xB2 temperature is 14\xB0"
at xy³ + y² temperature is 14°
% return [regsub -all -- {[°²³\xB0\xB2\xB3]} $mytest "\\\\&"]
at xy\³ + y\² temperature is 14\°

Works for both.

My guess is that this is pretty much the dirty solution the original poster found.

Please, please, somebody explain what the difference is so that I may sleep peacefully.

/me wanders off threatening a terrible revenge on all descendants of Charles Babbage.

Statistics: Posted by arfer — Sun Dec 28, 2008 7:41 pm

Problem with special characters ² ³ and °

2008-12-26T09:37:02-04:00

An interesting subject.

I have long suffered the mystical art of fonts, code pages, encodings and the like.

Lets say I copy/paste from the character map tool in windows to build up a text string as follows :-

% set mytest "at xy³ + y² temperature is 14°"
at xy³ + y² temperature is 14°

An attempt to modify (add a backslash to) the special characters using their equivalent hex values and regsub does not work :-

% return [regsub -all -- {[\xB0\xB2\xB3]} $mytest "\\\\&"]
at xy³ + y² temperature is 14°

It should have worked. It seems clear that copy/paste from windows character map does not result in the expected hex characters in the Tcl interpreter I am using.

However, if we explicity use hex character notation to build up the exact same text string as follows :-

% set mytest "at xy\xB3 + y\xB2 temperature is 14\xB0"
at xy³ + y² temperature is 14°

It looks the same, but now using the same regsub statement :-

% return [regsub -all -- {[\xB0\xB2\xB3]} $mytest "\\\\&"]
at xy\³ + y\² temperature is 14\°

It works!

If it is any help to the original poster, if you are able to explicitly build up your string using hex notation then that would enable you to use hex notation in the regsub pattern successfully.

If not, then it remains a mystical art which will eventually result in insanity.

Statistics: Posted by arfer — Fri Dec 26, 2008 9:37 am

Problem with special characters ² ³ and °

2008-12-22T14:00:44-04:00

We found a dirty but working way to fix it
tnx all for help

Thank you for posting your solution.

Statistics: Posted by incith — Mon Dec 22, 2008 2:00 pm

Problem with special characters ² ³ and °

2008-12-11T17:13:19-04:00

We found a dirty but working way to fix it
tnx all for help

Statistics: Posted by tomcat — Thu Dec 11, 2008 5:13 pm

Problem with special characters ² ³ and °

2008-12-11T16:55:09-04:00

damn, I have no problem with this char on my bot

u can always try to change this char to its code number, and after that u can delete it from whole string or smth :>

do some trigger and simple IF, !test and after in test script if {$arg == "special_char"} { puts "its ok" }

cause we dont know what is current wrong
like user wrote, that can be smth with codepage ;]

Statistics: Posted by tomekk — Thu Dec 11, 2008 4:55 pm

Problem with special characters ² ³ and °

2008-12-11T16:38:48-04:00

I tried iso 8859 the other is not known and utf-8

it doesnt matter ² ³ ° still dont get replaced with \² \³ \°

Statistics: Posted by tomcat — Thu Dec 11, 2008 4:38 pm

Problem with special characters ² ³ and °

2008-12-11T16:11:06-04:00

8.5.5

try:

Code:

source -encoding iso8859-1 scripts/scanner.tcl

u mean 8859 not 8659? ;p

:>

Statistics: Posted by tomekk — Thu Dec 11, 2008 4:11 pm

Problem with special characters ² ³ and °

2008-12-11T16:07:09-04:00

8.5.5

Statistics: Posted by tomcat — Thu Dec 11, 2008 4:07 pm

Problem with special characters ² ³ and °

2008-12-11T15:32:16-04:00

source -encoding iso-8659-1 scripts/scanner.tcl
results in crash...

tcl version 8.5.x? because its option from 8.5

Statistics: Posted by tomekk — Thu Dec 11, 2008 3:32 pm

Problem with special characters ² ³ and °

2008-12-11T14:32:53-04:00

source -encoding iso-8659-1 scripts/scanner.tcl
results in crash...

Statistics: Posted by tomcat — Thu Dec 11, 2008 2:32 pm