egghelp/eggheads community

Multithreaded Eggdrop ?

2003-02-05T11:35:08-04:00

There are 2 methods in Tcl.

One is to use the exec module, which is available on the eggheads.org FTP server.

Second it is use the normal exec command.

Tcl's exec command will run applications in the background, however, this is not documented.

Proof of concept.

FILE: test1

Code:

#!/usr/bin/tclshputs stdout "Program 1 - test 1"after 1000puts stdout "program 1 - test 2"exec /root/test2 &puts stdout "program 1 - test 3"

FILE: test2

Code:

#!/usr/bin/tclshafter 1000puts stdout "Program 2 - test 1"

If exec was to block, we would expect to see

Program 1 - test 1
Program 1 - test 2
Program 2 - test 1
Program 1 - test 3

However, I get

[rootdev@services root]# ./test1
Program 1 - test 1
program 1 - test 2
program 1 - test 3
[rootdev@services root]# Program 2 - test 1

The mess with the output, is caused by the second program, doing output to the console, after control has been returned to my shell program.

The exec command should never use quotes, which can cause lot of issues. Any quotes that need to be directly passed to a program, need to be excaped.

The last character on the line, of a exec command needs to be the &, without any >, >>, <, << touching (which won't be needed anyway).

I used this teqnique for repair databases, which would collect about 1MB of data per hour. While not large, repair, sort and optimisation could take a long time.

Before launching background operations, my bot would timeout and not return for 2 mins for every 5 MBytes. Thus, you can imagine how long the bot was out for if the database was 50MBytes.

Statistics: Posted by ppslim — Wed Feb 05, 2003 11:35 am

Multithreaded Eggdrop ?

2003-02-05T10:50:36-04:00

A minor point, but in my experience tcl exec blocks until the external process quits.

Under C it is very easy to multithread (i've just done it in a module I'm writing, but not easy to get the output from the second task unless it specifically reads/writes from a file.) However, you can flag completion easily (see fork() execl() and wait() for more details.

So, if you can code c, I'd recomend a module. If not ... um...

Statistics: Posted by Photon — Wed Feb 05, 2003 10:50 am

Multithreaded Eggdrop ?

2003-02-05T09:52:22-04:00

Fantastic response, thanks.

Something to get my teeth into again

Thanks
Si

Statistics: Posted by Guest — Wed Feb 05, 2003 9:52 am

Multithreaded Eggdrop ?

2003-02-05T09:44:49-04:00

First off, I aint an expert in this area, so take the comments at face value.

I don't think C is multi-threaded, only C++.

That being said, back to juicy part.

You don't need a multi-threaded bot, to perform blocking tasks. Only a set of utilities to manage input/output from other programs.

If a script is going to block, then you are bound to have timeouts, and even more so when queries start queuing.

With a launch and monitor system, you can process queries independantly of the obt, and retreive the output for display.

IE.

1: Query comes in.

2: Process any inofmration, needed to make the query

3: Create a random filename, that can be used to store results

4: Launch/exec second application in the background, giving the random filename, and the PID of the bot.

5: Script exit.

The second application, should do it's job, and otuput any data to the file. Once complete, it should signal to eggdrop as such. This can be done with kill signals (not all of them kill the bot, and some are scriptable).

When eggdrop gets the kill signal. It check to see which files are complete, out of a list of query requests (this method in genral, will allow for multiple queries to operate at the same time).

It will then process/output the information, then delete the temporary file.

While this isn't multi-threading, it does do the job just as good.

Statistics: Posted by ppslim — Wed Feb 05, 2003 9:44 am

Multithreaded Eggdrop ?

2003-02-05T09:32:45-04:00

The bot I run is a service bot. What I mean by that is that it is in constant use reporting information from a database. When a user asks for a long report (we don't actually know how long teh report will be until its run), it can tie up the bot for a while with all other requests for information from the bot being queued up.

I fully understand the implications of mulithreaded bots and writing to DBs, with locks etc causing probs. But, what if its only reading from a DB, surely it doesn't matter, and can speed up access to the bot tremendously, esp for bots like mine which spend their time spewing out stats/info from a DB built up from cheats/players on a set of game servers for the admins in their IRC channel.

I would be really kewl if this was implemented as a "type" of function, so that in your TCL you would essentially be saying "this function can run in multithreaded mode as its only reading from a DB", that way you could implement DB write functions in non-multithreaded mode, hence avoiding the need for extra locks etc.

Hope that makes sense.

Also, it would be nice to be able to "interupt" a bots output somehow. Not sure if this can already be done, i've looked and can't see how.

Cheers
Si

Statistics: Posted by Guest — Wed Feb 05, 2003 9:32 am