Macrobyte Resources Queue Suite
Queue Processing Daemon
for Radio UserLand and UserLand Frontier

Documentation




 

What is a Queue?

The function of the queue suite is fairly obvious, if you're at all familiar with queues. In case you're not familiar with queues, the following is a definition from The Free On-line Dictionary of Computing at Dictionary.com.

A first-in first-out data structure used to sequence multiple demands for a resource such as a printer, processor or communications channel. Objects are added to the tail of the queue and taken off the head.

A typical use of queues in an operating system involves a user command which places something on a queue, e.g. a file on a printer queue or a job on a job queue, and a background process or "demon" which takes things off and processes them (e.g. prints or executes them). Another common use is to pass data between an interrupt handler and a user process.

In other words, if you have multiple threads producing or receiving data (such as when running on a web server), but the data must be processed "single file" and in the order in which it was received/created, then you need a queue.

Working with the definition above, The Queue Suite is a general implementation of a queue processing demon.

How Does The Queue Suite Work?

First, you must create a new queue. Queue's run in their own threads, and you can run as many queues as you want (the actual limit depends on your system resources, mostly on RAM). Create a queue like this:

local ( queueId ); queueId = queue.new( 2 )

The parameter supplied to new() is the amount of time that the queue thread will sleep when the queue is empty. (After sleeping that amount of time, it checks the queue again.)

You MUST keep the queueId reference somewhere: either in a local variable, or in a global variable (somewhere in the ODB). You'll need this id number to add items to the queue, to stop the queue, to flush the queue... all queue verbs require the queue id that is returned from queue.new().

queue.new() accepts a number of optional parameters. See the section below, Optional Functionality, for more information.

Second, you add items to the queue with queue.add(). Here's an overly-simple example:

local ( queueId ); queueId = queue.new( 2 ); queue.add( queueId, "the queued item", @dialog.notify ); queue.add( queueId, "the second queued item", @dialog.alert ); queue.kill( queueId )

queue.add() takes four parameters.

The first parameter is the queue id, as was explained above.

The second parameter is the item to be added to the queue: it can literally be ANYTHING. A string, a table, an address, an outline, a filespec, an object specifier, ANYTHING. Of course, you're generally better off not moving large objects like big tables or very long strings: if you want to add something big to the queue, try to pass an address to the object.

The third parameter is an address of a script to process the item in the queue. In the example given, the first queued item uses dialog.notify, which conveniently takes one parameter: a string. The script whose address is passed must take one parameter (it may take additional optional parameters, but the queue won't ever call them). When the script is called, the queued item will be passed as the only parameter.

The fourth parameter is optional. See the section below, Optional Functionality -> queue.add, for more information.

WARNING: calling queue.add() (as with almost any queue verb) with a queueId that has already been killed, will generate a script error.

One thing you might consider for the third parameter, if you are creating a queue which might contain many types of elements: pass the address of a dispatch script. The dispatch script would receive the parameter, decide which script should process it (based on type), and then run that script. A simple example:

on dispatch( param ) { case typeof( param ) { stringtype { dialog.notify( param )}; addresstype { dialog.alert( param^ )}}}; local ( queueId ); queueId = queue.new( 2 ); queue.add( queueId, "this item is a string", @scratchpad.dispatch ); bundle { « add an address to the queue scratchpad.QItem = "this item passed as an address"; queue.add( queueId, @scratchpad.QItem, @scratchpad.dispatch )}; queue.kill( queueId )

To run this example, paste it into a script at scratchpad.dispatch, then click the Run button.

Finally, when you're sure that you're done putting items in the queue, call queue.kill( ). This adds a special item to the queue (items are always added to the end of the queue). When the queue reaches this special item, it deletes the queue table and kills the queue thread. This way, it's safe to call queue.kill(), because items which are already in the queue will be processed before the queue is stopped. Notice that in the examples above, we killed the queue immediately after adding items, but those items were processed anyway.

If you absolutely must stop the queue immediately, call queue.killNow( ). This kills the queue thread immediately, and deletes the queue from memory. Unprocessed items in the queue are lost.

queue.killNow( queueId )

Optional Functionality

queue.new

queue.new allows you to specify some optional parameters.

Queue Storage Location

The optional parameter storageLocation allows you to specify where the queue should be kept. If you supply this parameter, then it must be an address pointing to a table. Items added to the queue will be placed here, rather than in the default location.

The default location is temp.queues.

You can change the default storage location for new queues, by creating user.queue.storage named storage. The value of user.queue.storage should be the address of a table.

When you create a new queue, the queue suite will first check for the storageLocation parameter in the call to queue.new. If you haven't supplied that, then it checks user.queue.storage. If user.queue.storage doesn't exist or can't be used for some reason, then the default location (as described above) is used.

Saving The Queue

Setting the optional parameter flSaveOnAdd to true, when calling queue.new, will cause the file containing the queue (frontier.root, or a guest database) to be saved every time an item is queued or removed from the queue. Here's an example to show how to call it:

local ( qId ); qId = queue.new( 10, flSaveOnAdd: true )

Logging Queue Activity

Using two optional parameters, you can have all queue activity logged using Frontier's standard logging features.

The first optional, logging-related parameter, flLogging, is a boolean (true or false) that acts like a switch. If flLogging is set to true when you call queue.new, then most queue-processing actions will be logged.

Note that logs are written according to the preferences you've set in user.log.prefs.

The second parameter, queueName, is used in the log to identify the queue generating the log items. For example, if you set queueName to "dispatch," then all log items will be added to a log named "queue dispatch". If you don't provide the queueName parameter, and flLogging is set to true, then the queueName will be set to the qId. For example, if the qId is 234, then the log entries will be added to a log named "queue 234."

If the flLogging parameter is false, then the queueName parameter is ignored.

Logging is intended as a debugging tool, to help you write your scripts that work with queues. You may choose to use logging in your finished scripts, but keep in mind that it adds some overhead (time and memory) to your queues.

Example

Here's the dispatch script, from above, with all options activated:

on dispatch( param ) { case typeof( param ) { stringtype { dialog.notify( param )}; addresstype { dialog.alert( param^ )}}}; local ( queueId ); queueId = queue.new( 2, storageLocation: @workspace.queues,\ flSaveOnAdd: true, flLogging: true, queueName: "dispatch" );

queue.add( queueId, "item is a string", @scratchpad.dispatch ); bundle { « add an address to the queue scratchpad.QItem = "item passed as address"; queue.add( queueId, @scratchpad.QItem, @scratchpad.dispatch )}; queue.kill( queueId )

queue.add

queue.add has one optional parameter, adrErrorCallback. If this parameter is supplied to queue.add, and the item being added generates an error when it is processed, then this script will be called. It must take three parameters, and it must return either true or false.

The three parameters are:

  • the queueId for the current queue
  • the address of the item that generated the error
  • the error string (tryError).

The response from the error callback determines what is done with the item that generated the error. If the response is true, then the item is re-queued. If the response is false, then the item is deleted.

If the adrErrorCallback parameter is not supplied, then the queued item which generated the error is deleted (this is the default behavior).

The callback script can do anything: it might choose to requeue the item itself, and then return false. It might choose to modify the object somehow (remember: the address is being passed, not the object itself) before returning true. The possibilities are endless!

Here is a very simple example of an error callback script:

on queueErrorCallback( qId, adrQItem, error ) { if ( error contains "not yet ready" ) { return true } else { return false } }

Miscellaneous Verbs

There are some other useful scripts in the suite, for management of a queue.

  • Use queue.flush( queueId ) to completely empty out the contents of a queue. Will generate an error if the queue has already been killed.
  • queue.setSleepTime( queueId, newSleepTime ) can be used to change the amount of time that a queue thread sleeps for, when its queue is empty.
  • queue.getSleepTime( queueId ) returns a number representing the number of seconds that the queue thread will sleep for, when its queue is empty.

The remainder of the scripts in suites.queue are for internal use: don't use them, they're not guaranteed to stay the same from version to version. Think of the queue suite (and individual queues) as a mostly-opaque black box, and it'll work fine.

Last update: 5/14/2002; 11:05 PM



 
© 2002 Macrobyte Resources. All rights reserved.