Local SQL Services Startup issue

I’ve come across this issue a number of times over the years, rebooting a Lync or Skype server, specifically Std deployment type, and one of the three SQL Express Instances just doesn’t start up in time before the RTCSRV Frontend Service starts to start.  “Sometimes” it goes away with patching the local SQL instance up to SP2, but not often enough.  Could be a resourcing problem, but still, for a lab even, 4 processors and 8 Gb should be enough to start services up.

Enough is enough.  Why on earth what I’m about to show you isn’t done by default, is beyond me.  Why is the RTCSRV service on a Standard Edition Deployment, or for that matter on an Enterprise Deployment, though I’ve not encountered this on Enterprise Frontend Servers.

Using the SC command from an CMD Prompt (does not work from PowerShell) use the following command to make sure no other dependencies have already been made:  SC QC RTCSRV

Second last line you can see the current Dependencies, i.e. KeyIso in this case.

When modifying the dependency list, you have to include the existing, plus what ever you are wanting to add, separated by a forward slash (/).

 

sc config RtcSrv depend= KeyIso/MSSQL$LYNCLOCAL/MSSQL$RTC/MSSQL$RTCLOCAL

Re-run:  “SC QC RTCSRV”  to confirm your new dependencies have been successfully added.

Notice the change in the list of Dependencies.  You can also view this in the Services:

And if you want to get really nerdy, go into the Registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\RtcSrv
DependOnService

I’ve found that with this change, the RTCSRV service starts up nice and clean, and instead of taking about 25 minutes before crapping out, all the services have started up within 5 minutes of initiating a normal Reboot of the server.

I haven’t had need to try this on an Enterprise Server but the command should be similar, just minus the MSSQL$RTC.

sc config RtcSrv depend= KeyIso/MSSQL$LYNCLOCAL/MSSQL$RTCLOCAL

I still install at least SP2 for SQL 2014.  There were some performance issues in RTM that were resolved around CU7 or CU8, so SP1 should be your minimum anyway, that should help speed things up.  SQL Download Table

Magic CsDatabase Command

Hopefully the use of this will be rare for most, but its come up a couple times of late and magically solved a number of glitches, hiccups and downright headaches.  I’ll save the suspense and give it to you first, then the scenario’s.

Install-CsDatabase -Update -LocalDatabases

A) I needed to add a new SIP trunk to the Topology of a Lync 2013 environment, done it a thousand times with out issue.  Unfortunately this time the databases weren’t happy or in a properly updated state as the RTCSRV service crashed after running the Deployment Wizard, which also crashed, leaving the system in an in operable state.  With similar or the same error messages below, in scenario B.  In this case the CPSDyn and RTCDyn were irreparably damaged and un-mountable.  Fortunately from past experience with Lync 2010 database updates, I knew Dyn databases can be toasted without loss of life.  CPSDyn came back no problem with the Install-CsDatabase -ConfiguredDatabases -SqlServerFqdn, but the RTCDyn database did not.  Lync 2013 had “deprecated” the use/need for the -update switch, so I didn’t even think about it and must have tried 15 other methods or variations of getting this Dyn database back.  @RealTimeUC and I both seemed to have the same divine inspiration, as I was typing, he also suggested “Try with the -Update switch”.  Low and behold, it worked.  RTCDyn database was brought back to life, and the Frontend RTCSRV service can finally start again.

B) A recent Standard Deployment of Skype for Business resulted in the starting of the RTCSRV, Call Park Server and RGS services.

LS Call Park Service Event ID 31008
Connection: Data Source=(local)\rtc;Initial Catalog=cpsdyn;Integrated Security=True
Message: Cannot open database “cpsdyn” requested by the login. The login failed.
Login failed for user ‘NT AUTHORITY\NETWORK SERVICE’.

LS Response Group Service Event ID 31201
Exception: System.Data.SqlClient.SqlException – Cannot open database “rgsconfig” requested by the login. The login failed.

Login failed for user ‘NT AUTHORITY\NETWORK SERVICE’.

LS User Services Event ID 32134
Back-end Database: rtcxds Connection string of:
driver={SQL Server Native Client 11.0};Trusted_Connection=yes;AutoTranslate=no;server=(local)\rtc;database=rtcxds;
Cause: Possible issues with back-end database.

I checked, and the 3 SQL Services were running (LYNCLOCAL, RTC and RTCLocal).  I also have a neat trick for these databases I’ll show in my next blog.

A coworker had seen this before and suggested using the SQL Mgmt tools and compare the permissions of the databases to a working one.

Forget that, I have a magic PS command.  I figured if it can bring back a missing database, why not correct the database permissions.  I ran the same command successfully, and the system came to life.

Summary

I don’t believe in short cuts through life, but work smarter, not necessarily harder.  PS is your friend.

Group Series Skype Meeting Join Issue

I would like to think I had a pretty darn good OCS/Lync mentor when I started transitioning from mainly an Exchange focus to more with actual Unified Communications.  One of his mantra’s, “If you set things up right, they just work”, case in point, Polycom’s Group Series.

Client/users were complaining that they were having troubles joining scheduled Skype for Business meetings from the console.  They were experiencing odd voice messages like “That number is not in service”, “That call can not be completed as dialed”, or just ring 1-2 dozen times and disconnect…  Yup, that ain’t right, and yes, those very much sound like PSTN dialing issues.

This is exactly why you deploy Monitor Reports.  Pulling up the activity for the meeting room in question, one can see that the GS500 in this case was in fact dialing the Skype Conference ID’s, some were 6 digit and some were 7, making a phone number that either didn’t normalize at all, or managed to resolve to International dialing rules.

Eventually the user has learned to just drag the GS device in to the meeting, which is the case in row 3 above.

Client was running version 6.0, and both 6.0 and 6.1 are supported with Skype for Business.  We provided the client with James Arber’s post for setting up Group Series devices.  It’s more about 5.x version, but most of the settings are still relevant.

In the scenario we were troubleshooting, Skype for Business was running CU272, but looking at the history of the devices, this behaviour was not recent and occurred with previous CU’s.  Updating the GS to 6.1 did not resolve the issue.

The critical setting that was required to make the system work was under Admin Settings | Network | Dialing Preference, and specifically the Voice and Video Dialing order.  Here the client had left it defaulted to H.323 instead of changing to SIP.  Changing this setting on other systems still running 6.0 didn’t have the desired affect, 6.1 and prioritizing SIP seems to be the magic trick here.  This environment also had no Gatekeepers or RMX type systems, strictly a Skype for Business environment, otherwise you may need to test out various scenarios if you are running a combined environment.

Group Series updates can be found here:  http://support.polycom.com/content/support/North_America/USA/en/support/video/group_series.html

 

Shared Line Appearance and SimRing

This would be my own version of a “Fast Publish”, and haven’t fully tested out all the different scenario’s, so provide feedback if you like.

A client was experimenting with with Skype’s Shared Line Appearance (SLA) and was experiencing a problem where the shared line would ring once and go straight to voice mail of the -Target.  In this particular scenario, the -Target was also a Delegate, but that shouldn’t be an issue.

After checking 82 other things, I then installed the Skype Resource Kit, configured Sefautil and pulled out Johan’s SefaUtil GUI script, and reviewed the settings of the delegates.  Turned out that the Target had their SimRing enabled.  As soon as it was off, SLA worked as expected.

A scenario I’ll have to try some time, is there the same issue when just a Delegate has SimRing enabled.  I don’t beleive this will be the case.

I do suspect that the Target object will not be able to use Call Forwarding either.

Skype4b Online SIPAddress/UPN Mismatch

A recent troubleshoot case for which I did not have enough alcohol for to deal with.  A client was enabled in Office 365 for Skype Online, but had issues with Signing into the Skype client.  Checked all the usual suspects like DNS, licensing, PEBCAK, all without success.  Everything in the Portal looked correct.

PowerShell time:
Import-Module SkypeOnlineConnector
$credential = Get-Credential
$session = New-CsOnlineSession -Credential $credential
Import-PSSession $session -AllowClobber

Get-CsOnlineUser -Identity “Bob Smith” | fl *voice*,*PSTN*,*lineuri*,ObjectID,*SIP*,UserPrincipalName

In my output, the the SipAddress and the SipProxyAddress was a combination of the users ObjectID and UserPrinciplaName.  Armed with this new information, the user was able to use the combined SIP address as the Sign in Address, then use his UPN name for the User Name, and then the password.  It worked, but not acceptable.

Unfortunately we are provided with few tools to affect changes.  I am more accustomed to On Premises deployments where I can play god, but in Skype Online, I just feel more like Perseus or Hercules, some god-like powers, but outside looking in and a little bitter…

Set-CsUser is very limited in the Skype Online PowerShell, and you definitely can not modify the SIP Address, as per Article ID: 2909916.  So, very frustrating.

Technically, the SIP address is supposed to be populated with the User Principal Name (UPN), but in some odd cases, this does not happen.  After trying way to many ways to list to correct this, the “only” way currently (April 4, 2017, it’s O365, things change monthly) is to change the users UPN attribute via PowerShell.  You can try Option A and see if it works.  This particular case required Option B, much more extreme and time consuming.

Option A)
1) Start Windows PowerShell
2) run: Install-Module -Name MSOnline
3) after installation is complete, run: Connect-MsolService and enter credentials.
4) Run: Get-MsolUser -UserPrincipalName “bob.smith@company.com” | fl to confirm successful connection.
5) To change the UPN: Set-MsolUserPrincipalName -UserPrincipalName “bob.smith@company.com” -NewUserPrincipalName “bob.smith@company.onmicrosoft.com”

Now wait and O365 Minute (15-60 minutes), and observe that the users Account changes in the Portal.  Their Sign in address as well as their default SMTP address change.  Using your SkypeOnline powershell, recheck the SIP address.

Option A is actually very similar to a Online user name change, where you would set the UPN to their new name in essence, and once changes have completed, you would just stop there.  In this situation, we want to change back to bob.smith@company.com, and assuming the above process worked,  run the following command and wait another O365 Minute:

6) Run: Set-MsolUserPrincipalName -UserPrincipalName “bob.smith@company.onmicrosoft.com” -NewUserPrincipalName “bob.smith@company.com”

Bob wasn’t so lucky in his case.

Option B)
1)  Make note of the user’s current UPN, and any other Skype properties like LineURI, ect:  Get-CsOnlineUser -Identity bob.smith@company.com | fl *voice*, *PSTN*, *lineuri*, ObjectID, *SIP*, UserPrincipalName
2) Skype Online disable the user by removing all Skype licensing including but not limited to: Skype for Business Cloud PBX, Skype for Business PSTN Conferencing, Skype for Business Online, Skype for Business PSTN Consumption
3) Wait a O365 minute for the users Skype properties to fall off
4) Follow steps 1-5 in Option A using PowerShell to modify the UPN and wait the O365 Minute.  Additionally check the Users Exchange email address properties.  Here you can also see the incorrect SIP address prior to changing the UPN. It should disappear and the EUM address should also change.  IF the default SMTP address has changed to the new UPN, but the EUM has not, manually change it by just removing the ObjectID from the address, and then remove the bad SIP address.
5) Run step 6 of Option A, wait the O365 Minute for the Primary email address and user name to change back.  Check the Email Addresses again, see that the EUM has also changed to the correct UPN/email address and not the ObjectID
6) Re-enable the Skype Licensing that you previously disabled, and wait for Skype Online Provisioning to take place again for the user.  (O365 minute)

Assuming everything went well, the SIP Address should end up populating with the UPN address once again.  Verify the Email Address properties, PS or Portal, see that EUM and SIP are correctly populated.

Site wide validation couldn’t hurt right, try this Skype Online PS command:

get-csonlineuser | where {$_.SipAddress -ne “sip:”+ $_.userprincipalname} | fl DisplayName,UserPrincipalName,Sip*

Oh, and no explanation as to why this might happen, though the internal thinking is it’s DirSync related, BUT this client never had a Domain let alone DirSync.