Readers not queued?

When multiple readers are waiting on a message from the same queue, I would expect that the reader that has been waiting the longest would be the first to pick up a message. However, I'm shocked to discover that the opposite seems to be true; in my tests I'm showing that the reader that has been waiting the least time picks up a message first! This seems totally counter-intuitive, and I'd like to know why it's working this way. This implementation will cause a lot more reader timeouts to occur than a properly queued method. For instance, assume that I have two readers, each using a one minute timeout. Reader #1 starts waiting, and reader #2 is busy for another 20 seconds before it starts waiting. 39 seconds later a message comes in and reader #2 will pick it up, leaving reader #1 to time out one second later! I would much rather have reader #1 pick up the message and reader #2 continue to wait for 20 more seconds.

I'm considering filing a bug on Connect about this, but I thought I'd post here first and see if I can get an answer...

Following is the script I'm using to test:

Setup / Window #1

--
CREATE DATABASE SimpleSSB
GO

USE SimpleSSB
GO

--Create a database master key
CREATE MASTER KEY
ENCRYPTION BY PASSWORD = 'onteuhoeu'
GO

--Create a message type
CREATE MESSAGE TYPE Simple_Msg
VALIDATION = EMPTY
GO

--Create a contract based on the message type
CREATE CONTRACT Simple_Contract
(Simple_Msg SENT BY INITIATOR)
GO

--create a queue
CREATE QUEUE Simple_Queue
GO

--Create a service
CREATE SERVICE Simple_Service
ON QUEUE Simple_Queue
(Simple_Contract)
GO
--

Go start the other windows now

Readers: Windows #2-n

--
USE SimpleSSB
GO

WAITFOR
(
RECEIVE *
FROM Simple_Queue
), TIMEOUT 300000

--

Start at least two readers, then do
--

--send a message...
DECLARE @.h UNIQUEIDENTIFIER

BEGIN DIALOG CONVERSATION @.h
FROM SERVICE Simple_Service
TO SERVICE 'Simple_Service'
ON CONTRACT Simple_Contract
WITH ENCRYPTION=OFF;

SEND ON CONVERSATION @.h
MESSAGE TYPE Simple_Msg
GO

--

... the last reader you've started will pick up the message first. Note I'm testing on 9.0.3033, in case that matters.

Thanks!

Hi Adam,

The bevior you see is intentional. The purpose is to allow activated tasks to exit after a spike.

Suppose a traffic spike happens and suddenly all queue readers are activated to drain the queue. After the queue is drained, ideally the extra readers should go away. If we'd honor the longest waiting, then potentially all queue readers will stay active because each one will eventually get a message before the WAITFOR times out. With the behavior you see one reader will steal all messages, causing the other activated tasks to time out and exit. The same thinking goes for external readers as well.

HTH,
~ Remus

|||Hi Remus,

Thanks for the response. I agree that the behavior you're describing makes sense for activation, but in the scenario I'm working on--a farm of readers waiting on messages--it doesn't really work. I want to balance the activity across the farm so if one reader is working, the one waiting the longest will be the next to pick up a message. Any ideas?

Thanks,
Adam|||

You can't change the way WAITFOR(RECEIVE...) is notified.

If a reader is working, then it won't have any RECEIVE posted, so any available work will go to the next reader. The existing mode of operation will cause the first machine in the farm to take any load up to it's max capacity, then the next machine start picking up load until it max out, then the next and so on. What you describe would cause all machines to take an even workload and all grow simultanously until they all reach the max. Why is the first described behavior unacceptable?

An alternative to consider would be to use the broker built-in load balancing capabilities. If the same service is declared in multiple databases, then dialog targeting the said service will spread evenly across all instances of the service. Maybe you can use this feature to spread the workload.

HTH,
~ Remus

|||Hi Remus,

The reason it's unacceptable is that the readers in this case will be doing other work in addition to processing the queues. Basically, the scenario I'm working with right now is an application that already uses a load balanced server farm for doing some work, and the idea is to add some additional functionality queued via SSB. Obviously since the farm is already balanced it would be nice to also balance the SSB stuff--otherwise the "first" server in the farm is going to overload and not be able to handle the rest of its duties.

I was not aware of the load balancing feature you mention... I'll try it out and see how it fits into the scenario.

Thanks again! Your replies have been very helpful.
Adam