DeliveryMode should also here be a constant. Though, why are this set to PERSISTENT, when it elsewhere is set to NON_PERSISTENT? This means, that it is initially set to PERSISTENT in this function, which creates the producer, but the method which retrieves this producer through this function sets it to NON_PERSISTENT. Setting it to PERSISTENT is clearly overridden by the NON_PERSISTENT afterwards.
I think the test would be more readable if the test sequence code was explicit, instead of hidden in a TestSingleMessage construction. The test code should reflect the business process being tested, which again is what is stated in the test description. You could use the addStep(String, String) method to make this really verbose.
It seems this class is supposed to be used for a round-robin selection of connections from a number of potential connections. It is the responsibility of the caller to detect a dead connection and request a new one. This should at the very least be documented in the class documentation. However, it seems to me that it would be a better idea to make it the responsibility of this class to find a working connection. However, I think that fallbacks etc. should perhaps be handled outside java, since the message bus software should possibly be capable of replicating instances and providing fallbacks in a manner specific for that type of connection