Exchange UM Cert and Skype for Business

Don’t you just hate it when you’ve been doing things the right way for so long that you actually forget or don’t realize why it is “the right way”.  The Exchange UM Certificate is one such example.  I’ve been deploying OCS, Lync, Skype for 10+ years now, and Exchange before that since the 5.0 days (frack I’m old)…  99% of the time when integrating Skype with Exchange Unified Messaging, the UM role has either been left out or simply not configured.  I’m not going into the configuration of UM, there are a tonne of articles to that effect, but one item of extreme importance that is not always clear; The Subject Name for the Certificate used for the UM role(s) MUST be the FQDN of the server itself.

Most of the time because the certs used on Exchange servers have been deployed for a while, and a lot of the time Public Certs are already deployed on Exchange servers, BUT Public Certs now must use valid Top Level Domain Names e.g. can’t use .local .lan .domain.  So you create a internal CA cert with the FQDN of the server and assign it, no problems.  But, every now and then you come across a customer with internal and external domains matching, and they have added the server FQDN to the SAN list.  Internal calls to the the Subscriber Access line work, great, moving on…  Opps, did someone forgot to test externally, through the Edge, wait what, media failure, dang.

Ok, story time over, time to look in the weeds at the technical mumbo jumbo.

Symptom 1 – Call connects, but no bing bong “To access your mailbox…” when a User is working remotely and connected to their Edge server and trying to connect to Voicemail.

Symptom 2  – Not every time, but Event ID 32263 LS User Services error shows up in the Lync Event Log on the Frontends
Error code: 0xC3E93C2F(SIPPROXY_E_UNKNOWN_USER_OR_EPID), Error identifier: NotifySingle.Notify
Cause: Possible issues with the other front end or with this front end.

Symptom 3 – Call Detail Reports, Diagnostic ID 21 – “Call failed to establish due to a media connectivity failure where one endpoint is of unknown type”

Now for the “why”.  I’m such a GenX, I need the “why”.  Extensive logging, and reviewing of logs, and yes, support with Microsoft, I kept noticing something odd in the SIP Trace.  After the 180 Ringing and 200 OK, the ACK switched to, then delay, and then BYE.  MS went down a few other roads regarding networking, PowerShell commands, ports, port settings, and about 6 Gb more worth of logging, but this Autodiscover thing kept bugging me.

Finally we looked at the cert on the UM servers (Exchange 2010, two dedicated UM role servers), to check and make sure all the roots, intermediates and what not (FYI, this Cert checklist link, is handy have).  First thing that popped up to me, the Subject name of the Cert assigned to the UM role is yes,  The SAN list did contain the FQDN of both Servers.

Of course this seems silly, no where have I seen it documented that the Subject Name MUST be the FQDN of the UM server, but lets give that a try.  Two new internal CA certs were generated, SN and SAN being the server FQDN, assigned to the UM role, and services restarted.

Calls work.  Mothersmurfer…

I must reiterate, we made absolutely no other changes other than to replace the cert.

I did notice that in the autodiscover sip trace that there is a field, but this was insufficient for connecting calls through the Edge, even with name resolution for autodiscover hard codes in the HOSTS file to point to a UM server.  Somewhere in the code, the SIP call appears to switch to the Subject Name of the Cert for both resolution and MTLS, and it must be this way for Edge users to connect to Voice mail, and Auto Attendants too I suspect, but did not have one to test or work with.

An additional link of interest, a link to narrowing the audio port range of your Exchange UM servers and set for QoS.

Group Series 6.1.5 update

The Group Series 6.1.5 firmware updates were released today bringing some useful functionality to the systems, mainly VBSS.

  • Video-based Screen Sharing for Skype for Business
  • Managing Skype for Business Calls
  • System Upgrade or Downgrade Through Skype for Business Server
  • Org ID Authentication
  • Conference Recording with RealPresence Touch
  • Dialing through ISDN Gateway
  • RealPresence Collaboration Server (RMX) Call Escalation
  • Displaying Call Participant Names
  • New and Modified API Commands
Release notes and downloads can be found at Polycom.
(The Polycom Trio’s also came out with VBSS support with their December update, but came out with a new firmware with additional field fixes and can be found here.)
Some items that are resolved with 6.1.5 may be of interest:
  • In a Skype for Business environment, when a PSTN endpoint places a call to another PSTN endpoint registered to RealPresence Group Series system, the caller receives a loud audio.
  • When RealPresence Group Series system joins an AVMCU  conference, the Remote Desktop Protocol (RDP) content drops
  • When RealPresence Group Series system dials in to a Skype for Business client for Mac in a point-to-point call, Skype for Business client crashes
  • When RealPresence Group Series system registered to Skype for Business server performs a Skype for Business call with other RealPresence Group Series system and SfB clients, far-end
    video drops and system restarts
  • In an AVMCU call with Skype for Business client, the Group Series system does not receive video content occasionally

Happy updating.

VBSS and QoS

A glitch in the matrix that’s been bugging me for a while but just haven’t sat down to think it through… What to do with Video Based Screen Sharing and QoS.  VBSS came to us midstream for Skype for Business in June 2016, as a nice improvement to update speed and smoothness for Application/Desktop sharing, utilizing video and UDP instead of RDP and TCP.  Of course if you have a older client or if you start Recording the session, it reverts back to RDP/TCP.

What many have not realized though is that when a client is using VBSS in a Skype Meeting (not talking peer to peer here), that the Skype client selects an Application Port for the Source, but the Skype Server actually selects a Video port.  For the purpose of this post I’m using my typical QoS settings:
Audio Server: 49152-57500 Audio Client: 50040-50079
Video Server: 57501-65535 Video Client: 58000-58039
App Server: 40803-49151 App Client: 42000-42039
DSCP: Audio=46, Video=34, Application Sharing=24

The screenshot below is a Wireshark capture on the FE server.  As you can see, the Client source port is 42004, and the Destination port on the Server is 62345.  And in this particular network where this sample is taken, there are source/destination port restrictions set for the DSCP tags.  In this case 42000-40039 <–> 40803-49151.  Result is that not only is the traffic using both App and Video ports, it’s not even left tagged at all.

QoS Problem #1 – assuming the network does not have Src/Dst restrictions, you would end up with a DSCP mismatch; traffic from the server tagged with 34, traffic from the client tagged with 24.  Server Video traffic is then getting mixed with Server VBSS traffic.

QoS Problem #2 – with network Src/Dst port restrictions in place, traffic is deprecated into the lowest bucket 0, or worse.

In my opinion, the VBSS ports, both server and client, should have remained in the Application port ranges, but obviously this is not the case.  Surprisingly the solution is not to difficult.  We need to modify the QoS Policy on the Frontend servers by adding one new QoS policy.

New-NetQosPolicy -Name “Skype Server VBSS” -IPProtocol Both -IPSrcPortStart 57501 -IPSrcPortEnd 65535 -IPDstPortStart 42000 -IPDstPortEnd 42039 -DSCPValue 24

This new policy filters for network traffic matching both the Source port and the Destination port to tag it with a 24.  From my observations, because the “Skype Server VBSS” has this additional criteria, this makes it more restrictive than the “Skype Server Video” policy and it gets applied and the tagging works as desired.

Look at those happy little tags.  .101 is the Server in this screen shot and the .119 is a Trio.

If it is a particular concern, or just really want to locked down the policy, you can use this “Skype Server Video” policy.  My concern about adding this policy restriction is the traffic with the Edge for external users would not be prioritized on the network.  Edge traffic to the front end is going to be 443 or 3478 and would meet either Video QoS policy criteria:
New-NetQosPolicy -Name “Skype Server Video” -IPProtocol Both -IPSrcPortStart 57501 -IPSrcPortEnd 65535 -IPDstPortStart 58000 -IPDstPortEnd 58039 -DSCPValue 34

To address QoS Problem #2, the network will need to be reconfigured to allow:
42000-40039 <–> 40803-49151 with DSCP=24
42000-40039 <–> 57501-65535 with DSCP=24

Of course another solution is to modify your App and Video ports for client and server to all live in the 57501-65535 space with both using DSCP=34, but then you have no delineation between VBSS and Camera Video streaming.  A less then ideal solution.

Or disable VBSS… yuck.


For posterity sake, the other Net QoS Policy settings I use are:

Frontend Only:
New-NetQosPolicy -Name “Skype Server Audio” -IPProtocol Both -IPSrcPortStart 49152 -IPSrcPortEnd 57500 -DSCPValue 46

New-NetQosPolicy -Name “Skype Server Video” -IPProtocol Both -IPSrcPortStart 57501 -IPSrcPortEnd 65535 -DSCPValue 34

New-NetQosPolicy -Name “Skype Server App” -IPProtocol Both -IPSrcPortStart 40803 -IPSrcPortEnd 49151 -DSCPValue 24