***Hybrid Apps and Services can no longer be activated for new accounts. These features are only supported for accounts that already have these features enabled/activated.***
The information in this article is intended to enable ST14.2 customers (system administrators and associated IT personnel), who have activated HYBRID services (for ST14.2), to investigate and resolve issues with the syncing of their HYBRID services.
Before utilizing the troubleshooting information in this article, do the following:
See the Viewing Sky-Connect Sync Status on the Diagnostics Monitoring Page in Director section of the HYBRID Enablement Guide.
For an overview of the entire process of creating a MiCloud account, synchronizing user data from Director to the Account Portal, and managing MiCloud services, see the entire HYBRID Provisioning Process (for ST14.2) article.
After a Premise system administrator accesses HYBRID Services (for ST14.2 GA8 build 19.43.7902.0 and greater) via a link in Director to create a new HYBRID account, the system administrator becomes a Decision Maker with access to the Account Portal to manage HYBRID Services. When the system administrator (Decision Maker) accesses the HYBRID Services screen (via a link in the Administration menu of Director) and enters the Account ID and Account Token, the Connect Sync process is initiated to sync (import) data from Director to the Account Portal. However, there is no immediate indication to determine that the Account ID and Account Token were processed successfully (or unsuccessfully). See the Connect Sync Times section of this article for default processing times associated with syncing data.
The following image shows the HYBRID Services screen in Director immediately after an Account ID and Account Token were entered.
The information that follows provides instructions and suggested guidelines for troubleshooting and resolving issues with the Connect Sync process.
Warning Message
If there are any errors or other issues with your HYBRID Service, the HYBRID Services screen (in Director) displays the following Warning message with a link to access Quick Look, as shown in the following image.
"Warning! There is an error with Connect Services. Click Quicklook to find out the details"
Quick Look
The Quick Look area of Director should be used first to verify the success of Connect Services processing, and to investigate and resolve processing errors.
Connect Sync Validation in Quick Look
To verify that the Account ID and Account Token entered in the Connect HYBRID Services screen were validated, and to verify that the Connect Sync process to synchronize user profile data from Director to the Account Portal was successful, perform the following steps:
Click the Quick Look link in the Maintenance menu of Director, as shown in the following image
In the Quick Look screen that opens, click the Headquarters link under Server/Appliance, which opens the Main Server Maintenance screen, as shown in the following image.
In the Status area of the screen, if the indicator to the left of Connect Token is green and the status to the right shows OK, the Account ID and Account Token were accepted by the Account Portal and user data synchronization should process successfully.
In the Services area of the screen, if the indicator to the left of Connect Sync is green and the status to the right does not show an error message, user data was successfully synced from Director to the Account Portal.
If the Status or Services indicators are not green, continue reading the sections below to find useful error status, troubleshooting, and logging information.
Quick Look Error Status
The following table shows the possible Quick Look status values that are associated with the "Connect Token" and "Connect Sync" processes. If the information in the Possible Resolution column does not resolve the issue you are experiencing, please see the Server Windows Event Logs and Connect Sync Logs sections of this article.
Color
Quick Look Display in Director
Status Description
Possible Resolution
Green
OK
Account ID and Account Token validated
The Connect Token credential validation process was successful.
Yellow
Internal Error
Invalid Parameter found
Possible causes are:
Email address already exists - In Director, navigate to Administration > Users > Individual Users, and verify that each user has a valid, unique Email Address.
Phone number is not a valid DID - In Director, navigate to Administration > Users > Individual Users and verify that each user has a valid DID Number. See the "TN" requirements in the Connect Sync User Profile Data section of this article.
TN already in use - In some instances, a TN needs to be reassigned. See the Reassigning a TN section of this article for instructions.
Yellow
Internal Error
Error with Database on Portal
Database connection timeout. This can occur when system resources are temporarily down. The system will continue trying to initiate the Connect Sync process every five minutes. Wait to see if this automatically resolves the issue. If this error persists for more than approx. 30 minutes, see the Server Windows Event Logs and Connect Sync Logs sections of this article to learn how to find more detailed information about this processing error.
Token is most likely incorrect or contains an extra space. See the Copy-Paste Token Error section of this article.
Red
Lost Communication
Portal Connection error
Reached the Account Portal, but could not contact the server. This can occur when system resources are temporarily down. The system will continue trying to initiate the Connect Sync process every five minutes. Wait to see if this automatically resolves the issue. If this error persists for more than approx. 30 minutes, see the Server Windows Event Logs and Connect Sync Logs sections of this article to learn how to find more detailed information about this processing error.
Red
Unknown
Unknown error
Could not reach the server (server was shut down or experienced a possible power outage). This can occur when system resources are temporarily down. The system will continue trying to initiate the Connect Sync process every five minutes. Wait to see if this automatically resolves the issue. If this error persists for more than approx. 30 minutes, see the Server Windows Event Logs and Connect Sync Logs sections of this article to learn how to find more detailed information about this processing error.
It is best to use Quick Look first, and then (as needed) obtain more details about issues using the Troubleshooting Errors with Connect Services section below, before contacting Support for help with resolving the issues related to your Connect HYBRID Services.
Troubleshooting Errors with Connect Services
See the "Viewing Sky-Connect Sync Status on the Diagnostics Monitoring Page in Director" section of the Connect HYBRID Enablement Guide. Then use the sections of this article that are listed below to identify and resolve errors associated with the Connect Sync process:
When the system administrator accesses Director to sync user data with the Account Portal, if the Account ID and Account Token are entered and the token is incorrect, the indicator to the left of Connect Token is red and the Status displays Invalid Login, as shown in the following image. This occurs when the Token is rejected by the Account Portal, which prevents the Connect Sync process from running.
The copy-paste action that is used to capture and enter the token can easily introduce an extra space or character. If you experience an Invalid Login status message for the Connect Token, make sure there are no spaces or extra characters when the Account Token is copied and pasted. It is recommended that you manually type the characters in the token. In a future release, logic will be added to remove any extra spaces before or after the string of characters in the Account Token.
Server Windows Event Logs
The Server Windows Event logs contain detailed information about issues with events related to the Connect Sync process, which can be used to troubleshoot and resolve errors.
Open the Event Viewer window
Navigate to menu > Windows Logs > Application
The log files include the Event IDs for warnings, errors, and informational events described in the table below
Note: In Windows Logs, you can find ConnectSync listed in the Task Category column of the Windows Application log, as shown in the following image:
The following table shows each of the possible Event IDs with the associated Event Type, Event Description, and Possible Resolution.
Event Type
Event ID
Event Description
Possible Resolution
Warning
5000
The Account ID or Token was rejected as invalid by the Account Portal.
Incorrect Account ID or Account Token. Carefully re-enter these credentials. See the Copy-Paste Token Error section of this article for more information.
Error
5001
There was an problem connecting to the Account Portal. Retrying in 5 minutes.
Account Portal connection issue. The system will continue trying to initiate the Connect Sync process every five minutes. Wait to see if this automatically resolves the issue. If this error persists for more than approx. 30 minutes, see the Connect Sync Logs section of this article before contacting Support.
Information
5002
The Account ID and Token was validated by Account Portal
Successful validation of Account information!
Information
5003
PushAll succeeded.
Successfully synced all valid user profiles in Director with the Account Portal!
Error
5004
Error in GetPartition HTTP Response
Error during the initial synchronization with the database in the Account Portal.
This can occur when system resources are temporarily down. The system will continue trying to initiate the Connect Sync process every five minutes. Wait to see if this automatically resolves the issue. If this error persists for more than approx. 30 minutes, see the Connect Sync Logs section of this article or contact Support.
Error
5005
Error in GetProfiles HTTP Response
Account Portal has returned an error when attempting to get stored user profiles in Director. Find the error details in the Details tab of the ConnectSyncEvents interface. If the details do not provide information to resolve this issue, see the Connect Sync Logs section of this article before contacting Support.
Error
5006
The following Profiles had errors while trying to sync with the Account Portal
This is the most common sync issue. Find the error details in the Details tab of the ConnectSyncEvents interface. The two most common causes and solutions are listed below:
Duplicate email address - In Director, navigate to Administration > Users > Individual Users, and update the user profile associated with the error to have a valid, unique Email Address.
TN already in use - In some instances, a TN needs to be reassigned. See the Reassigning a TN section of this article for instructions.
Error
5007
The following Profile Batch had errors while trying to sync with the Account Portal
Account Portal has returned errors on profile batches that were sent to Director. Find the error details in the Details tab of the ConnectSyncEvents interface. If the details do not provide information to resolve this issue, see the Connect Sync Logs section of this article before contacting Support.
If the instructions listed in the table above do not provide enough information to resolve the issue(s) with your Connect Services, see the Connect Sync Logs section of this article.
The following image shows an example of the information displayed in the General tab of the ConnectSync Events window for Event ID 5006:
The following image shows an example of the information displayed in the Details tab of the ConnectSync Events window for Event ID 5006:
Connect Sync Logs
This section of the article includes the following topics:
Connect Sync logs are located on the HQ (Headquarters) server at: *C:\Shoreline Data\Logs (* a drive other than "C" may be in use)
The log file format is: ConnectSync-YYMMDD-xxxxxx.log
If you experience any issues with the syncing of Connect Services, find and open the most recent Connect Sync log file, review the contents to identify the possible reasons for the issue, and make notes of any specific TNs, email addresses, etc. that you find. The following image shows an example of accessing a ConnectSync log file:
The following table shows a list of the Connect Sync log errors that can be resolved by a system administrator along with an explanation and possible resolution for each error:
Connect Sync Error Message
Error Explanation / Possible Resolution
There is already a Profile for this TN (Duplicate TN)
System administrator may have synced a user with this TN, then disabled that user in Director, and subsequently reassigned the TN to a different user.
The following are two possible solutions that can be performed to resolve this error:
In Director, navigate to Administration > Users > Individual Users, and select a different TN for the user associated with this error, then wait a minimum of 5 minutes for the Connect-Sync process to run again
In some instances, a TN needs to be reassigned. See the Reassigning a TN section of this article for instructions.
The requested TN is already in use
This error can be caused by one of the following: there is already a profile with the TN, the TN is already assigned to another account, or the TN is in a pool of reserved phone numbers.
To resolve this error, in Director, navigate to Administration > Users > Individual Users and select a different TN for the user associated with this error. In some instances, a TN needs to be reassigned. See the Reassigning a TN section of this article for instructions.
TN is required for Profile creation
Attempted to sync a profile with no TN field set.
To resolve this error, in Director, navigate to Administration > Users > Individual Users, and add a TN to the user associated with this error, then wait a minimum of 5 minutes for the Connect-Sync process to run again
Scribe creation failed: Not enough minutes available for Scribe.
Profile (XXXXXX) has exceeded the monthly allotment for the Restricted plan. Month-to-date usage: XX messages / 00:XX:XX (hh:mm:ss).
This means that Account Portal records show the user is not provisioned for Scribe, but Director shows the user is enabled for Scribe, so there is a mismatch of records. If the system administrator disabled the user for Connect HYBRID Scribe in the Account Portal, this enablement status is synced with Director every 8 hours, but if the network connection or server was down at that time, this could cause the system to be out of sync.
To resolve the issue, verify and/or update the following email settings in Director:
Administration > Users > Personal Options > Escalation Profiles and Other Mailbox Options (the "Deliver Message as Email" setting must be set to "Attach Wav File")
Administration > Users > Individual Users (Email Address must be unique and valid)
Wait until the Connect Enablement status (updated every 8 hours) is processed again by the Connect-Sync process. If the issue is not resolved after this period of time, contact Support to restart the Connect Sync Service.
If the instructions above did not resolve the issue, check with your IT department to see if a server port is blocked, which would prevent outbound messages from being sent.
Profile does not exist in the Account Portal, but does exist in Director.
Technically, this error should not occur, but if it does, access the Server Windows Event Logs, to identify the user profile, then disable the user's TN in Director, and wait a minimum of 5 minutes for the Connect-Sync process to run. If this does not resolve the error, contact Support to perform the following:
Restart the Connect Sync Service to sync data between Director and the Account Portal.
There is already an active Profile with this ForeignId
Director is sending multiple user profiles with the same ForeignId.
Delete offending profiles in Director, recreate the profiles, then wait for the Connect Sync process to run, which occurs every five minutes. If this error persists for more than approx. 30 minutes, contact Support.
Premise Profile can only be created on the Premise cluster.
This is an Account configuration issue that can occur when trying to sync users with a non-Connect account partition.
Technically, this error should not occur, but if it does, the system administrator in Director needs to verify that the correct Account ID and Token were entered in the Connect HYBRID Services screen in Director. See Copy-Paste Token Error for details. Otherwise, contact Support.
The following table shows a list of Connect Sync log errors that require contacting Support to resolve.
Connect Sync Error Message
Error Explanation
Only Premise Profiles can be created at this point
Not relevant, attempted to create a non-Connect Profile.
Only Premise profiles can be removed
Attempted to delete a non-Connect profile.
This profile cannot have Scribe provisioned
This is a Account Portal Issue.
This profile is not configured for Scribe
This is a Account Portal Issue.
Error retrieving ForeignId:{0} - There does not seem to be a record with a matching foreign id value.
This is a Account Portal Issue.
The requested profile type is not a Premise type
Error trying to query for a non-Connect user.
Missing Data in Scribe Request
This means the Premise system did not send enough data to fill in the request. That should not happen unless data was corrupted.
Cannot be changed after profile creation - Multiple fields cannot be modified after profile was already created.
This error means that a profile was created and synched to the Account Portal, and then changed. The following properties cannot be changed on a synced profile: TN, ForeignId, ProfileType, PartitionId, AccountId.
Find the offending profile in Director and make a note of the user data associated with the error.
Any other error messages that indicate an internal technical problem (disk space, database connection, etc.).
Make a note of the offending error message.
Reassigning a TN
If you experience an error indicating "TN already in use," "Duplicate TN," or "there is already a Profile for this TN," you may need to reassign a TN for the Connect HYBRID user who no longer needs that phone number. To reassign a phone number, perform the following:
In Director, navigate to Administration > Users > Individual Users
Find the desired user who no longer needs the TN
Remove the check mark next to "DID Range"
Save the changes
Wait a minimum of 5 minutes for the Connect-Sync process to run again.
Log into the Account Portal
Navigate to Phone System > Users, which opens the Users screen
Search for the user who was edited in Director, and verify that the user is not listed in the Users screen
The TN is now available to be assigned to a different user.
Troubleshooting the Scribe Service
The following sections provide information related to troubleshooting the Scribe service.
The log file format is Vmail-YYMMDD-xxxxxx.log
Use the following guidelines and search tips (shown in bold) when accessing voicemail logs to troubleshoot Scribe-related issues:
To determine if a user is enabled for the Scribe service, search the voicemail log for the email address of the user in question
If the following message is found in the voicemail log, then VMAIL is aware that the Mailbox is enabled for Scribe "Mailbox is configured for Scribe, sending Scribe request"
If the Scribe request failed, search for the following message: MS:Scribe failure, service returned
If authentication failed because of invalid Account ID and Token, search for the following message: ER: The Voice Mail Application failed to authenticate with the Scribe service
In Director, if Administration > Users > Personal Options > Escalation Profiles and Other Mailbox Options > Deliver Message is set to Attach Wav file, and communication with the Scribe server fails, search for the following message: The system was unable to transcribe this voice message
Scribe Settings in Director
To ensure that the Connect HYBRID Scribe service has the required settings in Director, for each user, do the following:
Navigate to Administration > Users > Personal Options > Escalation Profiles and Other Mailbox Options
Specify a valid Email Address, and set Deliver Message as Email to "Attach Wav File" (not "Disabled").
Note: If Deliver Message as Email is set to "Disabled," the PBX will not send the voicemail audio file to the Scribe Server.
When a user is enabled for Scribe, the user's Escalation Profiles and Other Mailbox Options screen displays "This mailbox is enabled for Scribe" as shown in the following image:
Scribe Settings in the Account Portal
If the PBX (Director) shows that all users are configured for Scribe, as explained in the Scribe Settings in Director section above, do the following to verify that Scribe is enabled in the Account Portal:
The Decision Maker (system administrator) logs in to Account Portal with email address and user password
Navigate to Phone System > Add-on Features
In the Add-on Features screen that appears, if you see two buttons, Settings and Manage, next to Connect HYBRID Scribe, the service has been activated. If you see the Activate button, the service has not been activated. See the Connect HYBRID Scribe (for ST14.2) article for complete instructions for managing this service.
Click the Manage button next to Connect HYBRID Scribe to see the list of users who were previously added to this service.
If the users who were added to the service are not receiving voicemail transcriptions when voicemail messages are left on the user's Sky phone, see the following possible reasons:
Incoming email server is blocked by IT
Contact your IT department to investigate and resolve.
This can occur when system resources are temporarily down. The system will continue trying to initiate the Connect Sync process every five minutes. Wait to see if this automatically resolves the issue. If this error persists for more than approx. 30 minutes, see the Server Windows Event Logs and Connect Sync Logs sections of this article before contacting Support.
Resolving the "No Minutes Remaining" Error for Scribe
To resolve this issue:
Log into the Account Portal and navigate to Phone System > Add-on Features
Click the Manage button next to Connect HYBRID Scribe
Right-click within the row of the user profile associated with the error
Select Update Hybrid Scribe
Set the Service Type to Unrestricted and click OK
Wait a minimum of 5 minutes for the Connect-Sync process to run.
To prevent this issue from occurring for new users added to this service in the future:
From the Add-on Features screen, click the Settings button next to Connect HYBRID Scribe
Set the Service Type to Unrestricted and click OK
Additional Connect Services Information
The following sections provide additional information that can be helpful in identifying and resolving issues with Connect HYBRID Services:
Creating Event Filters in Director
In Director, system administrators can use the Event Filter to receive emails when a Connect Sync Event takes place. These emails can be useful in detecting and resolving processing errors. To setup an Event Filter, navigate to Maintenance > Event Filters > Add New, and select or enter the values listed next to each of the fields listed below:
Server: Headquarters
Source: Select Services and Connect Sync Service from the drop-down menu
Category: Any
Event ID: Any
Type: All
Target E-mail Address: Enter the email address of the system administrator
The following image shows the Event Filter screen in Director:
Connect Sync User Profile Data
The following table shows the data fields imported for each user. Note that no initial services are created for imported users and there is no billing for the associated phone profiles. Decision Makers must continue to add, change, and remove user profile data in Director and then sync the data with the Connect database in the Account Portal.
User Profile Field
Data Example
Data Description
ForeignId
3ef4bc0e-0403-45cc-a52c-f752b372b2fa
Unique ID for each profile in Director and Account Portal
Extension
200
User extension on premise
* TN
4081231234
Unique 10-digit DID in the NPA-NXX-xxxx (NANP) number format. * A valid non-international DID phone number is required for Connect services.
* Email
johndoe@ringcentral.com
User email address * A valid, unique email address is required for Connect services.
FirstName
john
User first name
LastName
doe
User last name
GroupName
Staff
Group the user belongs to on premise
ForeignUsername
johndoe@ringcentral.com
User login name * A valid, unique email address is required for Connect services.
ForeignPassword
123456
User SIP password
SecondaryUsername
john_doe_200
User Mobility login name
Connect Sync Times
The following table shows the default time values that are set in Director to control the default Connect Sync processing times.
Default Times for Connect Sync Processes
Description of Process
User Data Changes = Every 5 Minutes
If any user data was changed in Director, sync the changes with the Account Portal.
Enablement Status = Every 8 Hours
Wake up, do a GET of Connect enablement status and save to the Director database. If any user data was changed in Director, sync the changes with the Account Portal.
Enablement Status + Version = Every 24 Hours
Wake up, do a GET of enablement status and version. If any user data was changed in Director, or the version in Director does not match the version in the Account Portal, sync the changes with the Account Portal.
