Fix for not finding client(key)

The connectioninfo brought in using the connectioninfo as the key for the clients. The application using the sb only have access to the remote ip and port so the key didn't match if it was using the remote ip:port and local ip:port.

Change-Id: If61faac9bb2be571c08faa95f3beb5190499a54b
Signed-off-by: Sam Hague <shague@redhat.com>