Troubleshooting License Service Reporter

Prerequisites

• Linux or OSX console
• Access to cluster with kubectl where IBM License Service and IBM License Service Reporter is installed.
• kubectl
• jq
• gzip
• base64
• xargs

Important:

• Before you follow the procedure, generate audit snapshot from License Service Reporter (if you already deployed the License Service Reporter) and for each License Service instance. Auditing period should cover time since the first deployment for each instance.

• Additionally, note the user-defined threshold values if the value has been set. You will have to manually recover them later.

Steps

1. Backup License Service Reporter token and licensing data:

   1. Define the namespace where License Service Reporter is deployed.

   LICENSING_NAMESPACE=<enter the namespace of the License Service Reporter>
   

   2. Backup token to the file on your computer.

   kubectl get secret ibm-license-service-reporter-token -n ${LICENSING_NAMESPACE} -o=jsonpath='{.data.token}' > token.txt
   

   3. Generate an audit snapshot from License Service Reporter for the backup purpose.

2. Clean License Service Reporter database by deleting the License Service Reporter instance:

   1. Log in to your OpenShift cluster console.

   2. Go to Operators > Installed Operators.

   3. Set the project to All Projects.

   4. Find and select the IBM Licensing.

   5. In the navigation bar, go to IBM License Service Reporter tab and delete existing instance.

   

   6. Confirm that you want to delete the License Service Reporter.

   7. Wait until the License Service Reporter pod disappears.

3. Deploy the newest License Service Reporter. Wait until License Service Reporter pod is up and running.

4. Optional: Restore backed up token.

   Note: Skip this step if you deploy License Service Reporter for the first time.

   1. Apply saved value to the secret on the cluster, where the License Service reporter is deployed.

   cat token.txt | xargs -I {} kubectl patch secret ibm-license-service-reporter-token -n ${LICENSING_NAMESPACE} -p '{"data":{"token":"{}"}}'
   

   2. Restart License Service Reporter pod.

   lsrPodName=$(kubectl get pods -n ${LICENSING_NAMESPACE} | grep ibm-license-service-reporter-instance | awk '{print $1}')
   kubectl delete pod ${lsrPodName} -n ${LICENSING_NAMESPACE}
   

   3. Restart License Service pod on the same cluster where License Service Reporter is deployed.

   lsPodName=$(kubectl get pods -n ${LICENSING_NAMESPACE} | grep ibm-licensing-service-instance | awk '{print $1}')
   kubectl delete pod ${lsPodName} -n ${LICENSING_NAMESPACE}
   

5. Optional: Configure each cluster that was configured to feed data to License Advisor as a data source for License Service Reporter. For more information, see Configuring data sources for License Service Reporter.

   See the following notes:

   • Skip this step if you are resetting previously deployed License Service Reporter.

   • For the cluster where you deployed License Service Reporter, the License Service instance is automatically configured. Therefore, you do not need to perform any configuration for this cluster.

6. Synchronize License Service historical data.

   Note: This step synchronizes the license usage data only. The user-defined threshold values are not synchronized, and the values need to be set manually afterwards.

   Important: Perform this step on each cluster with License Service that is configured as License Service Reporter datasource, including License Service instance on the cluster with License Service Reporter.

   1. Upgrade License Service instance to the latest version. Wait until the pod is running.

   2. Define the synchronization start date. The start data determines the time starts when the License Service data is sent to the License Service Reporter. Provide the date in the [year,month,day] format. By default, the date is set to License Service first release date.

      START_SYNC_DATE=[2020,4,6]
      

   3. Define the namespace where License Service is deployed.

      LICENSING_NAMESPACE=ibm-licensing
      

      Note: The ibm-licensing namespace is the default namespace of the License Service. If you installed License Service in a custom namespace in your cluster, replace ibm-licensing with your custom namespace in the command.

   4. Apply the synchronization start date by running the following command:

      For OSX:

      kubectl get cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -o jsonpath="{.binaryData.data}" | base64 -d | gzip -d | jq -c ".lastHubSynchronizationDate = ${START_SYNC_DATE}" | gzip | base64 | xargs -I {} kubectl patch cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -p '{"binaryData":{"data":"{}"}}'
      

      For Linux:

      kubectl get cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -o jsonpath="{.binaryData.data}" | base64 -d | gzip -d | jq -c ".lastHubSynchronizationDate = ${START_SYNC_DATE}" | gzip | base64 -w 0 | xargs -I {} kubectl patch cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -p '{"binaryData":{"data":"{}"}}'
      

7. Wait until your data is migrated to License Service Reporter. It takes from 12 to 24 hours to migrate the data from all the clusters and show the data on the License Service Reporter UI.

8. Access the Licensing dashboard, the License Service Reporter UI. For more information, see Viewing license usage on the Licensing dashboard.

9. Recover the threshold values manually. For more information, see Setting the license usage threshold.

Unable to sign in with the credentials using basic authentication

You can sign in to the License Service Reporter console with the credentials retrieved from the secret. If 401: Invalid Username or Password error comes into view, complete the steps that follow:

1. Restart the operator pod with the following command. Update the namespace with the installation namespace of the License Service Reporter:

      kubectl delete pods -l app=app.kubernetes.io/instance=ibm-license-service-reporter-operator -n $NAMESPACE
   

2. Wait for sometime until the new operator pod is created.

3. Check the logs with the following command. Update the namespace with the installation namespace of the License Service Reporter:

      kubectl logs -l app.kubernetes.io/instance=ibm-license-service-reporter-operator -n $NAMESPACE
   

   The similar log entry is displayed:

      2023-06-29T11:52:01Z INFO Reconcile all done {"controller": "ibmlicenseservicereporter", "controllerGroup": "operator.ibm.com", "controllerKind": "IBMLicenseServiceReporter", "IBMLicenseServiceReporter": {"name":"instance","namespace":"ibm-licensing"}, "namespace": "ibm-licensing", "name": "instance", "reconcileID": "2f3eb70d-56c0-4042-b194-0f7408e8fd75", "resourceKind": "*v1alpha1.IBMLicenseServiceReporter", "resourceName": "instance"}
   

4. Delete the License Service Reporter pod to restart the application with the following command. Update the namespace with the installation namespace of the License Service Reporter:

     kubectl delete pods -l app=ibm-license-service-reporter-instance -n $NAMESPACE
   

5. After the application is active, you can log in with the credentials.

Unable to sign in with IBM Identity Management v4.x

IBM Identity Management does not support authorization. To sign in with IBM Identity Management, remove --allowed-group and --allowed-role parameters from the Custom Resource of the License Service Reporter.

Unauthorized - 403 return code

• Make sure that the proper role is assigned for the user with --allowed-group or --allowed-role parameter.
• Make sure that the user belongs to the correct email domain with --email-domain parameter.

Error logs

For more details on the authentication issues, check the logs of the authentication container in the License Service Reporter deployment.

Unable to connect to the IAM IDP issuer_url because it points to the localhost

To resolve this issue, complete the following steps:

1. For the OIDC_ISSUER_URL parameter, change the localhost to the cp-console route address in the IAM platform-auth-idp configuration map.
2. Restart the IAM platform-auth-service pod and retry with the configuration.