This feature was introduced in Qpid Dispatch 0.6. It is a significant improvement on an earlier somewhat experimental feature called Waypoints.
Auto-link is a feature of Qpid Dispatch Router that enables a router to actively attach a link to a node on an external AMQP container. The obvious application for this feature is to route messages through a queue on a broker, but other applications are possible as well.
An auto-link manages the lifecycle of one AMQP link. If messages are to be routed to and from a queue on a broker, then two auto-links are needed: one for sending messages to the queue and another for receiving messages from the queue. The container to which an auto-link attempts to attach may be identified in one of two ways:
- The name of the connector/listener that resulted in the connection of the container, or
- The AMQP container-id of the remote container.
Here is an example configuration for routing messages deliveries through a pair of queues on a broker:
connector {
name: broker
role: route-container
host: <hostname>
port: <port>
sasl-mechanisms: ANONYMOUS
}
address {
prefix: queue
waypoint: yes
}
autoLink {
addr: queue.first
dir: in
connection: broker
}
autoLink {
addr: queue.first
dir: out
connection: broker
}
autoLink {
addr: queue.second
dir: in
connection: broker
}
autoLink {
addr: queue.second
dir: out
connection: broker
}
The address entity identifies a namespace (queue.*) that will be used for routing messages through queues via autolinks. The four autoLink entities identify the head and tail of two queues on the broker that will be connected via auto-links.
If there is no broker connected, the auto-links shall remain inactive. This can be observed by using the qdstat tool:
$ qdstat --autolinks
AutoLinks
addr dir phase link status lastErr
===================================================
queue.first in 1 inactive
queue.first out 0 inactive
queue.second in 1 inactive
queue.second out 0 inactive
If a broker comes online with a queue called queue.first, the auto-links will attempt to activate:
$ qdstat --autolinks
AutoLinks
addr dir phase link status lastErr
======================================================================
queue.first in 1 6 active
queue.first out 0 7 active
queue.second in 1 failed Node not found: queue.second
queue.second out 0 failed Node not found: queue.second
Note that two of the auto-links are in failed state because the queue does not exist on the broker.
If we now use the Qpid Proton example application simple_send to send three messages to queue.first via the router:
$ python simple_send.py -a 127.0.0.1/queue.first -m3
all messages confirmed
and then look at the address statistics on the router:
$ qdstat -a
Router Addresses
class addr phs distrib in-proc local remote cntnr in out thru to-proc from-proc
========================================================================================================
mobile queue.first 1 balanced 0 0 0 0 0 0 0 0 0
mobile queue.first 0 balanced 0 1 0 0 3 3 0 0 0
we see that queue.first appears twice in the list of addresses. The phs, or phase column shows that there are two phases for the address. Phase ‘0’ is for routing message deliveries from producers to the tail of the queue (the out auto-link associated with the queue). Phase ‘1’ is for routing deliveries from the head of the queue to subscribed consumers.
Note that three deliveries have been counted in the “in” and “out” columns for phase ‘0’. The “in” column represents the three messages that arrived from simple_send and the “out” column represents the three deliveries to the queue on the broker.
If we now use simple_recv to receive three messages from this address:
$ python simple_recv_noignore.py -a 127.0.0.1:5672/queue.first -m3
{u'sequence': int32(1)}
{u'sequence': int32(2)}
{u'sequence': int32(3)}
We receive the three queued messages. Looking at the addresses again, we see that phase ‘1’ was used to deliver those messages from the queue to the consumer.
$ qdstat -a
Router Addresses
class addr phs distrib in-proc local remote cntnr in out thru to-proc from-proc
========================================================================================================
mobile queue.first 1 balanced 0 0 0 0 3 3 0 0 0
mobile queue.first 0 balanced 0 1 0 0 3 3 0 0 0
Note that even in a multi-router network, and with multiple producers and consumers for queue.first, all deliveries will be routed through the queue on the connected broker.
Here is an extension of the above example to illustrate how Qpid Dispatch Router can be used to create a distributed queue in which multiple brokers share the message-queueing load.
connector {
name: broker1
role: route-container
host: <hostname>
port: <port>
sasl-mechanisms: ANONYMOUS
}
connector {
name: broker2
role: route-container
host: <hostname>
port: <port>
sasl-mechanisms: ANONYMOUS
}
address {
prefix: queue
waypoint: yes
}
autoLink {
addr: queue.first
dir: in
connection: broker1
}
autoLink {
addr: queue.first
dir: out
connection: broker1
}
autoLink {
addr: queue.first
dir: in
connection: broker2
}
autoLink {
addr: queue.first
dir: out
connection: broker2
}
In the above configuration, there are two instances of queue.first on brokers 1 and 2. Message traffic from producers to address queue.first shall be balanced between the two instance and messages from the queues shall be balanced across the collection of subscribers to the same address.
Since configurable entities in the router can also be accessed via the management protocol, we can remotely add a shard to the above example using qdmanage:
qdmanage create --type org.apache.qpid.dispatch.connector host=<host> port=<port> name=broker3
qdmanage create --type org.apache.qpid.dispatch.router.config.autoLink addr=queue.first dir=in connection=broker3
qdmanage create --type org.apache.qpid.dispatch.router.config.autoLink addr=queue.first dir=out connection=broker3