SBA Fails to install. Due to applications not being defined in topology

Ran into an interesting one for those that still install SBA’s

I know the mentality these days is to get redundant network links to highly available pools, but when a business is located over 600km (370+ mi) away from the nearest CBD completely independent, affordable, high quality network links tend to be hard to come by. So SBA’s still have a place esp when you just want calling to work!

TL;DR: If your just here for the solution, check that the topology is actually publishing properly. Try running enable-cstopology and if you get errors, resolve those before continuing.

The symptoms and troubleshooting

The new AudioCodes installer originally complained that the topology hadn’t been published correctly and that the SBA was not part of the topology, despite it appearing in topology builder from a fresh topology download.

This SBA is not part of topology. Please add the SBA to topology and click ‘Refresh’ button

I initially thought this was because I forgot to create the computer object before publishing the topology. A bit of digging into AudioCodes powershell scripts and I found a the computer object was not a part of the RTCSBAUniversalServices so manually added this and continued with the install.

This all worked fine until the Activating Services part of the SBA install, the SBA would run Test-CsComputer and the test would fail. Citing that application objects were not defined with the error

“The server applications document is already defined with “0” applications”

Manually trying to start the RTC service on the SBA failed with API call failures suggesting the UCMA was corrupt or missing.

event id # 30236:

The UserServices module was not found in the Application List.
Application Uri: ‘http://www.microsoft.com/LCS/UserServices’
Cause: The server configuration requires the UserServices module to be registered and active. However the configuration information is not found in WMI.
Resolution:
UserServices configuration is installed in WMI at setup time. Check the application list in MMC and ensure that it is present. If it is not present, reinstall the server. If this problem occurred after you installed a new application, contact the application vendor to rectify the problem.

Event  id # 30211:

Failed to initialize the API subsystem.
Cause: Review earlier events to determine the specific reason APIEM failed to initialize.

Event id # 12308

A component could not be started.  The service has to stop.
Component: Live Communications Applications Module  Error code: C3F2765F (APIEM_E_ESBASE_MISSING)

(unfortunately I don’t have screenshots of these errors, as I re-flashed the SBA with them on)

This led to several re-installs of the Skype4B components, UCMA, Windows fabric and even sending an engineer onsite to reflash the SBA all to no avail.

The actual issue

This customer has brought me in to help stabilize their Skype4B platform. As part of the cleanup I noticed I would get an error when publishing the topology relating to a trusted application being in AD twice. I raised it to the client to discuss with that vendor and left it at that.

Error: An error occurred: “System.IO.InvalidDataException” “Multiple Active Directory entries were found for type “ms-RTC-SIP-TrustedService” with ID “appserver.www.UcMadScientist.com,Appname”.”

 

However, Topology builder would still list the publish as successful “Completed with warnings” in topology builder.

 

This was in fact, far from the truth. Whilst the Topology was published it wasn’t enabled correctly.

Looking at the logs for publishing the topology, you can see this error in the core site is causing the topology enablement to abort midway through the changes. Never actually enabling the new SBA in the CMS.

With the error you can see the topology stops publishing after only a few sites.

After the duplicate AD entry issue was resolved. You can see the additional sites getting enabled.

And the topology getting published properly

So how did I find the offending AD entry?

Well in this case, the customer has been using Skype4B since the OCS2003 days, but I used a method similar to what Paul Bloem (The Visio Stencil guy!) used in his blog post except I used LDIFDE to export the two relevant sections to a notepad document

C:\Windows\system32>LDIFDE -D “CN=Trusted Services, CN=RTC Service, CN=Microsoft, CN=System, DC=ucmadcientist, DC=com” -s ad01.www.UcMadScientist.com -f “c:\ucmadscientist\Trusted App exports\adsi2.txt”

Note: This would typically be in “CN=Trusted Services, CN=RTC Service, CN=Microsoft, CN=Configuration, DC=ucmadcientist, DC=com” but this is a very very very old install that wasn’t migrated correctly from OCS to lync 2010

Once the offending entries were found, it was a simple matter of deleting them via ADSI edit and re-publishing the topology.