I0319 18:04:37.045623 1 cmd.go:253] Using service-serving-cert provided certificates I0319 18:04:37.046072 1 observer_polling.go:159] Starting file observer I0319 18:04:38.418283 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0319 18:04:38.418477 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0319 18:04:38.419020 1 secure_serving.go:57] Forcing use of http/1.1 only W0319 18:04:38.419042 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0319 18:04:38.419048 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0319 18:04:38.419052 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0319 18:04:38.419056 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0319 18:04:38.419059 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0319 18:04:38.419062 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0319 18:04:38.419113 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0319 18:04:38.423129 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0319 18:04:38.423029 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"7c9166ba-cfaa-401e-92f5-fad6de28c9ab", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0319 18:04:38.423978 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0319 18:04:38.423996 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0319 18:04:38.424036 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0319 18:04:38.424055 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0319 18:04:38.424079 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0319 18:04:38.424086 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0319 18:04:38.424521 1 secure_serving.go:213] Serving securely on [::]:8443 I0319 18:04:38.424610 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0319 18:04:38.424639 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0319 18:04:38.429127 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0319 18:04:38.429152 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0319 18:04:38.429184 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0319 18:04:38.433867 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0319 18:04:38.433884 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0319 18:04:38.437978 1 secretconfigobserver.go:119] support secret does not exist I0319 18:04:38.442157 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0319 18:04:38.445927 1 secretconfigobserver.go:119] support secret does not exist I0319 18:04:38.449039 1 recorder.go:176] Pruning old reports every 6h36m16s, max age is 288h0m0s I0319 18:04:38.454849 1 operator.go:356] The last pod state is unhealthy I0319 18:04:38.524943 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0319 18:04:38.524960 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0319 18:04:38.524972 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0319 18:04:38.530147 1 base_controller.go:82] Caches are synced for ConfigController I0319 18:04:38.530160 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0319 18:04:38.534600 1 configmapobserver.go:84] configmaps "insights-config" not found I0319 18:04:58.455981 1 operator.go:356] The last pod state is unhealthy I0319 18:05:18.455274 1 operator.go:356] The last pod state is unhealthy I0319 18:05:38.455329 1 operator.go:356] The last pod state is unhealthy I0319 18:05:58.456726 1 operator.go:356] The last pod state is unhealthy I0319 18:06:18.464721 1 operator.go:356] The last pod state is unhealthy I0319 18:06:38.455777 1 operator.go:356] The last pod state is unhealthy I0319 18:06:58.458507 1 operator.go:356] The last pod state is unhealthy I0319 18:07:18.459827 1 operator.go:356] The last pod state is unhealthy I0319 18:07:38.455102 1 operator.go:356] The last pod state is unhealthy I0319 18:07:58.456646 1 operator.go:356] The last pod state is unhealthy I0319 18:08:18.455689 1 operator.go:356] The last pod state is unhealthy I0319 18:08:38.456071 1 operator.go:356] The last pod state is unhealthy I0319 18:08:58.457844 1 operator.go:356] The last pod state is unhealthy I0319 18:09:18.456005 1 operator.go:356] The last pod state is unhealthy I0319 18:09:38.452196 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0319 18:09:38.454840 1 operator.go:356] The last pod state is unhealthy I0319 18:09:38.456197 1 secretconfigobserver.go:119] support secret does not exist I0319 18:09:58.456341 1 operator.go:356] The last pod state is unhealthy I0319 18:10:02.570152 1 operator.go:288] Unable to check insights-operator pod status. Setting initial delay to 11m58.797556525s I0319 18:10:02.570226 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0319 18:10:02.570245 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 13m28s I0319 18:10:02.570285 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0319 18:10:02.570302 1 insightsreport.go:296] Starting report retriever I0319 18:10:02.570309 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0319 18:10:02.576222 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0319 18:10:02.576243 1 controller.go:368] The initial operator extension status is healthy I0319 18:10:02.576250 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:10:02.576255 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:10:02.576259 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:10:02.576276 1 controller.go:531] The operator is still being initialized I0319 18:10:02.576282 1 controller.go:554] The operator is healthy I0319 18:10:02.576356 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0319 18:10:02.576376 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0319 18:10:02.576564 1 operator.go:328] started I0319 18:10:02.576585 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0319 18:10:02.583736 1 controller.go:254] Source scaController *sca.Controller is not ready I0319 18:10:02.583748 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0319 18:10:02.583753 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:10:02.583756 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:10:02.583758 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:10:02.583776 1 controller.go:531] The operator is still being initialized I0319 18:10:02.583782 1 controller.go:554] The operator is healthy I0319 18:10:02.598819 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0319 18:10:02.677635 1 base_controller.go:82] Caches are synced for LoggingSyncer I0319 18:10:02.677666 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0319 18:10:02.684928 1 cluster_transfer.go:102] no available accepted cluster transfer I0319 18:10:02.684940 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0319 18:10:02.860627 1 sca.go:266] etc-pki-entitlement secret successfully created I0319 18:10:02.860645 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0319 18:12:02.584204 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:12:02.584223 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:12:02.584228 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:12:02.584255 1 controller.go:531] The operator is still being initialized I0319 18:12:02.584262 1 controller.go:554] The operator is healthy I0319 18:14:02.583141 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:14:02.583158 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:14:02.583161 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:14:02.583184 1 controller.go:554] The operator is healthy I0319 18:14:38.444220 1 configmapobserver.go:84] configmaps "insights-config" not found I0319 18:14:38.457246 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0319 18:14:38.462062 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0319 18:14:38.462079 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0319 18:14:38.466269 1 secretconfigobserver.go:119] support secret does not exist I0319 18:16:02.587087 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:16:02.587106 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:16:02.587111 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:16:02.587141 1 controller.go:554] The operator is healthy I0319 18:18:02.582977 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:18:02.582995 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:18:02.583000 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:18:02.583030 1 controller.go:554] The operator is healthy I0319 18:19:38.466330 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0319 18:19:38.471133 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0319 18:19:38.471151 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0319 18:19:38.475396 1 secretconfigobserver.go:119] support secret does not exist I0319 18:20:02.583478 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:20:02.583499 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:20:02.583504 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:20:02.583534 1 controller.go:554] The operator is healthy I0319 18:22:01.368067 1 periodic.go:216] Running clusterconfig gatherer I0319 18:22:01.368131 1 tasks_processing.go:45] number of workers: 32 I0319 18:22:01.368168 1 tasks_processing.go:69] worker 31 listening for tasks. I0319 18:22:01.368173 1 tasks_processing.go:71] worker 31 working on networks task. I0319 18:22:01.368166 1 tasks_processing.go:69] worker 0 listening for tasks. I0319 18:22:01.368175 1 tasks_processing.go:69] worker 11 listening for tasks. I0319 18:22:01.368192 1 tasks_processing.go:69] worker 2 listening for tasks. I0319 18:22:01.368191 1 tasks_processing.go:69] worker 21 listening for tasks. I0319 18:22:01.368187 1 tasks_processing.go:69] worker 1 listening for tasks. I0319 18:22:01.368207 1 tasks_processing.go:69] worker 18 listening for tasks. I0319 18:22:01.368208 1 tasks_processing.go:69] worker 16 listening for tasks. I0319 18:22:01.368206 1 tasks_processing.go:69] worker 3 listening for tasks. I0319 18:22:01.368217 1 tasks_processing.go:69] worker 19 listening for tasks. I0319 18:22:01.368222 1 tasks_processing.go:69] worker 17 listening for tasks. I0319 18:22:01.368225 1 tasks_processing.go:69] worker 20 listening for tasks. I0319 18:22:01.368217 1 tasks_processing.go:69] worker 4 listening for tasks. I0319 18:22:01.368230 1 tasks_processing.go:69] worker 15 listening for tasks. I0319 18:22:01.368235 1 tasks_processing.go:69] worker 8 listening for tasks. I0319 18:22:01.368224 1 tasks_processing.go:69] worker 5 listening for tasks. I0319 18:22:01.368232 1 tasks_processing.go:69] worker 13 listening for tasks. I0319 18:22:01.368244 1 tasks_processing.go:69] worker 7 listening for tasks. I0319 18:22:01.368231 1 tasks_processing.go:69] worker 6 listening for tasks. I0319 18:22:01.368238 1 tasks_processing.go:69] worker 9 listening for tasks. I0319 18:22:01.368249 1 tasks_processing.go:69] worker 28 listening for tasks. I0319 18:22:01.368247 1 tasks_processing.go:69] worker 14 listening for tasks. I0319 18:22:01.368254 1 tasks_processing.go:69] worker 29 listening for tasks. I0319 18:22:01.368256 1 tasks_processing.go:69] worker 27 listening for tasks. I0319 18:22:01.368260 1 tasks_processing.go:69] worker 24 listening for tasks. I0319 18:22:01.368263 1 tasks_processing.go:69] worker 23 listening for tasks. I0319 18:22:01.368240 1 tasks_processing.go:69] worker 12 listening for tasks. I0319 18:22:01.368263 1 tasks_processing.go:69] worker 25 listening for tasks. I0319 18:22:01.368247 1 tasks_processing.go:69] worker 26 listening for tasks. I0319 18:22:01.368269 1 tasks_processing.go:71] worker 0 working on cluster_apiserver task. I0319 18:22:01.368275 1 tasks_processing.go:71] worker 25 working on image_registries task. I0319 18:22:01.368276 1 tasks_processing.go:71] worker 11 working on tsdb_status task. I0319 18:22:01.368277 1 tasks_processing.go:71] worker 2 working on install_plans task. I0319 18:22:01.368282 1 tasks_processing.go:71] worker 26 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0319 18:22:01.368277 1 tasks_processing.go:71] worker 12 working on operators_pods_and_events task. I0319 18:22:01.368345 1 tasks_processing.go:71] worker 1 working on machine_healthchecks task. I0319 18:22:01.368349 1 tasks_processing.go:71] worker 5 working on storage_cluster task. I0319 18:22:01.368246 1 tasks_processing.go:69] worker 10 listening for tasks. I0319 18:22:01.368495 1 tasks_processing.go:71] worker 10 working on cost_management_metrics_configs task. I0319 18:22:01.368521 1 tasks_processing.go:71] worker 13 working on container_runtime_configs task. I0319 18:22:01.368545 1 tasks_processing.go:71] worker 18 working on service_accounts task. I0319 18:22:01.368669 1 tasks_processing.go:71] worker 7 working on proxies task. I0319 18:22:01.368253 1 tasks_processing.go:69] worker 22 listening for tasks. I0319 18:22:01.368271 1 tasks_processing.go:71] worker 21 working on certificate_signing_requests task. I0319 18:22:01.368791 1 tasks_processing.go:71] worker 17 working on aggregated_monitoring_cr_names task. I0319 18:22:01.368806 1 tasks_processing.go:71] worker 6 working on machine_configs task. I0319 18:22:01.368831 1 tasks_processing.go:71] worker 29 working on version task. I0319 18:22:01.368977 1 tasks_processing.go:71] worker 3 working on olm_operators task. I0319 18:22:01.369007 1 tasks_processing.go:71] worker 20 working on monitoring_persistent_volumes task. I0319 18:22:01.369041 1 tasks_processing.go:71] worker 19 working on clusterroles task. I0319 18:22:01.368259 1 tasks_processing.go:69] worker 30 listening for tasks. I0319 18:22:01.369108 1 tasks_processing.go:71] worker 30 working on overlapping_namespace_uids task. I0319 18:22:01.369116 1 tasks_processing.go:71] worker 27 working on machines task. I0319 18:22:01.369155 1 tasks_processing.go:71] worker 16 working on node_features task. I0319 18:22:01.369200 1 tasks_processing.go:71] worker 28 working on qemu_kubevirt_launcher_logs task. I0319 18:22:01.369307 1 tasks_processing.go:71] worker 14 working on validating_webhook_configurations task. I0319 18:22:01.369312 1 tasks_processing.go:71] worker 24 working on pdbs task. I0319 18:22:01.369357 1 tasks_processing.go:71] worker 23 working on ingress_certificates task. I0319 18:22:01.369384 1 tasks_processing.go:71] worker 9 working on pod_network_connectivity_checks task. I0319 18:22:01.369398 1 tasks_processing.go:71] worker 15 working on nodenetworkstates task. I0319 18:22:01.369450 1 tasks_processing.go:71] worker 22 working on operators task. I0319 18:22:01.369489 1 tasks_processing.go:71] worker 8 working on container_images task. I0319 18:22:01.369527 1 tasks_processing.go:71] worker 4 working on mutating_webhook_configurations task. I0319 18:22:01.374769 1 tasks_processing.go:71] worker 13 working on ingress task. I0319 18:22:01.374781 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 6.229893ms to process 0 records I0319 18:22:01.375738 1 tasks_processing.go:71] worker 5 working on nodenetworkconfigurationpolicies task. I0319 18:22:01.375752 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 7.376154ms to process 0 records I0319 18:22:01.376010 1 tasks_processing.go:71] worker 15 working on feature_gates task. I0319 18:22:01.376017 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 6.600324ms to process 0 records I0319 18:22:01.376308 1 tasks_processing.go:71] worker 27 working on lokistack task. E0319 18:22:01.376319 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0319 18:22:01.376332 1 gather.go:177] gatherer "clusterconfig" function "machines" took 7.179572ms to process 0 records I0319 18:22:01.376369 1 gather_lokistack.go:58] Start LokiStack gathering E0319 18:22:01.376520 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0319 18:22:01.376533 1 tasks_processing.go:71] worker 16 working on config_maps task. I0319 18:22:01.376542 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 7.363876ms to process 0 records I0319 18:22:01.377276 1 tasks_processing.go:71] worker 10 working on active_alerts task. I0319 18:22:01.377285 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 8.767746ms to process 0 records I0319 18:22:01.377565 1 tasks_processing.go:71] worker 1 working on openshift_logging task. E0319 18:22:01.377623 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0319 18:22:01.377639 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 9.21149ms to process 0 records I0319 18:22:01.379555 1 tasks_processing.go:71] worker 0 working on metrics task. I0319 18:22:01.379750 1 recorder.go:75] Recording config/apiserver with fingerprint=db9689824aef6c136c4b6eb19793da6b798c638574c32c1cdb41875bec3269d2 I0319 18:22:01.379789 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 11.266962ms to process 1 records I0319 18:22:01.383618 1 tasks_processing.go:71] worker 7 working on infrastructures task. I0319 18:22:01.383711 1 recorder.go:75] Recording config/proxy with fingerprint=6eb140ae5d2c8d44bb0848be867b574d048e0c3eedf35594146df58d63597b56 I0319 18:22:01.383731 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 14.926681ms to process 1 records I0319 18:22:01.383948 1 tasks_processing.go:71] worker 27 working on storage_classes task. I0319 18:22:01.383967 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 7.628277ms to process 0 records I0319 18:22:01.383990 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 14.948775ms to process 0 records I0319 18:22:01.384003 1 tasks_processing.go:71] worker 20 working on crds task. I0319 18:22:01.384368 1 tasks_processing.go:71] worker 14 working on openshift_machine_api_events task. I0319 18:22:01.384638 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=194d99b35866a9fd6211bcf08dd2e433851909bb030edbad596af9faa4ab0acf I0319 18:22:01.384731 1 recorder.go:75] Recording config/validatingwebhookconfigurations/cert-manager-webhook with fingerprint=165709bfc430d31466bcc5c79c5863e96764946be94e6f6f01e3221594c783f6 I0319 18:22:01.384804 1 recorder.go:75] Recording config/validatingwebhookconfigurations/config.webhook.operator.tekton.dev with fingerprint=6b9cd86e9d1341aa0593e520a7c372e51afdcf6ba095ce0d603eef92533fae6d I0319 18:22:01.384850 1 recorder.go:75] Recording config/validatingwebhookconfigurations/config.webhook.pipeline.tekton.dev with fingerprint=8c0ed4071b0b999a6ed7030ddd6bdf5165331f7c6f72e9c989ca609b972cb1e8 I0319 18:22:01.384881 1 recorder.go:75] Recording config/validatingwebhookconfigurations/config.webhook.triggers.tekton.dev with fingerprint=2a23906c93864c7086d30fa6d474f876681561d9e8eb1473fdb6fd1b583c80cc I0319 18:22:01.384963 1 recorder.go:75] Recording config/validatingwebhookconfigurations/integration-service-validating-webhook-configuration with fingerprint=81c6e90119cd817f1e9c1d736041139927d30c73ce924f636611d304b68da5c4 I0319 18:22:01.385051 1 recorder.go:75] Recording config/validatingwebhookconfigurations/monitoringconfigmaps.openshift.io with fingerprint=e0b31d9878caa145d91650aded3882b3a36963b5bcb20370bebc26ab9394b368 I0319 18:22:01.385152 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=4d1f9db6b932fcdd6be740b44774ee9ba7c32adce807afbfd3416c693125d93a I0319 18:22:01.385195 1 recorder.go:75] Recording config/validatingwebhookconfigurations/namespace.operator.tekton.dev with fingerprint=539083918dd16c1f254932958893109324995a20f64172960245d12bba533bfd I0319 18:22:01.385328 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=241872ad310f953546f36f578ef034502a7e446d58ccb2c7fec7b5c453699517 I0319 18:22:01.385418 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=ba173fbbd0a6bcfac8a00ffed59ed1b235768f397b07cb3424a284e320577a11 I0319 18:22:01.385459 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=01ef069332e5ffa80019602c1ac2438be99a09368c7d66d769eabb48f33bd01a I0319 18:22:01.385523 1 recorder.go:75] Recording config/validatingwebhookconfigurations/release-service-validating-webhook-configuration with fingerprint=f843e9320961a57c9cc2e3d036889264965b57dac35f38a504463f62e2af6533 I0319 18:22:01.385559 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=fb25a5b5727c7c092982bc40b87651808d84861ec358a9ee753d948d9a573d5d I0319 18:22:01.385589 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=62dce52a92343bcabc6e010a5ec7a00d80160718ec6cfe9f14b52005bf751bb8 I0319 18:22:01.385614 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=d74799f8e5bfda85a01969aa03728bb9671a0d728f3d89847ada796ce38621ea I0319 18:22:01.385637 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=5616670a2103ec0c029a23772208115d32f77498465dbce1238698e074ba4e10 I0319 18:22:01.385680 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=867440b396b834768f5eac8a800d9826c905142fa7f28d5f6d7a192eeba6ee98 I0319 18:22:01.385715 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=112e94386fb639cd3073829741d77c10e6880d80828660853df1edb00dc8142a I0319 18:22:01.385751 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=4dd09c0a5d9f403dec98efe225186721f0e910caf782a50c0a730b0f7e2cd81c I0319 18:22:01.385798 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=4c8887f6294b0d5f3c2757cb25346ac977ddf796ec2c4488a8255a6bf0a231b4 I0319 18:22:01.385831 1 recorder.go:75] Recording config/validatingwebhookconfigurations/validation.pipelinesascode.tekton.dev with fingerprint=74696363ba56702cd19bae323b391fdb78a36d7c9c623602fce9bc14686c4c24 I0319 18:22:01.385874 1 recorder.go:75] Recording config/validatingwebhookconfigurations/validation.webhook.operator.tekton.dev with fingerprint=54a6d2c4021ced0af1ab90364f4e2a6a93aeb8e55bd374ad7ec5ab1899131d72 I0319 18:22:01.385939 1 recorder.go:75] Recording config/validatingwebhookconfigurations/validation.webhook.pipeline.tekton.dev with fingerprint=d5189d69a6dd7d26cdb8b7fafda9993c5f1b33ba7e2004c40fb3e0b3adda5281 I0319 18:22:01.386001 1 recorder.go:75] Recording config/validatingwebhookconfigurations/validation.webhook.triggers.tekton.dev with fingerprint=e986af86c625321b815495c6737ef1faa10d88ee9c6d3a368fd96da0b1d0151f I0319 18:22:01.386015 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 15.04388ms to process 25 records I0319 18:22:01.386176 1 tasks_processing.go:71] worker 31 working on support_secret task. I0319 18:22:01.386601 1 recorder.go:75] Recording config/network with fingerprint=cc5b8134d5d7774e61d02c170dcae6531a764a4c527fec5abfbce0f3bf1f31b1 I0319 18:22:01.386662 1 gather.go:177] gatherer "clusterconfig" function "networks" took 17.023639ms to process 1 records I0319 18:22:01.387185 1 tasks_processing.go:71] worker 25 working on ceph_cluster task. I0319 18:22:01.387238 1 gather_logs.go:145] no pods in namespace were found I0319 18:22:01.388139 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=7237c6e66595df5519417d4cf744ef0f0695de0f322a0925d879d4bd8f14213b I0319 18:22:01.388225 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 17.320175ms to process 1 records I0319 18:22:01.388386 1 tasks_processing.go:71] worker 15 working on image_pruners task. I0319 18:22:01.388424 1 recorder.go:75] Recording config/featuregate with fingerprint=9998c72139e137c492fea284e0aa85cc17ab3d8357b1eee57732f0952df45826 I0319 18:22:01.388773 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 9.703916ms to process 1 records I0319 18:22:01.388831 1 recorder.go:75] Recording config/podnetworkconnectivitychecks with fingerprint=44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a I0319 18:22:01.388861 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 16.334938ms to process 1 records E0319 18:22:01.388890 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0319 18:22:01.388978 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0319 18:22:01.389009 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 16.941939ms to process 1 records I0319 18:22:01.389048 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 10.016694ms to process 0 records I0319 18:22:01.389080 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 8.524722ms to process 0 records I0319 18:22:01.389108 1 tasks_processing.go:71] worker 5 working on node_logs task. I0319 18:22:01.389115 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 18.080367ms to process 0 records I0319 18:22:01.389093 1 tasks_processing.go:71] worker 9 working on openstack_controlplanes task. I0319 18:22:01.390444 1 tasks_processing.go:71] worker 13 working on authentication task. I0319 18:22:01.390542 1 recorder.go:75] Recording config/ingress with fingerprint=202cd7ba7ce2b1b13fdc8f97a098353f29c9d2e5cb0a8627855ad1b27775a390 I0319 18:22:01.390599 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 13.475672ms to process 1 records I0319 18:22:01.389086 1 tasks_processing.go:71] worker 6 working on nodes task. I0319 18:22:01.390757 1 tasks_processing.go:71] worker 24 working on openstack_version task. I0319 18:22:01.390769 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=436671105a8439e7e27f8644ee77b9d33300bc169fa780af66566b2eb1ad342e I0319 18:22:01.390797 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=27d9735ddb32e738ee43b45c06657913f97c55503d4affd97faf44e53629511e I0319 18:22:01.389130 1 tasks_processing.go:71] worker 1 working on dvo_metrics task. I0319 18:22:01.390823 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=24f5adb26ee3df0065ad7978b920396cd00e6d204b4e2eebc0555613ef66e71b I0319 18:22:01.390851 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=42093ccdaf5b3c5710b4e2a9475516a91704fd0501760801505b44d79f629d69 I0319 18:22:01.390883 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=91eefbf177837ed745477d7567799e694f376cd98530370587413be6304496e7 I0319 18:22:01.390928 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=3f4cc99af885c8733497700060cf1aa372e61b2b4d069201004012a2f4a80e31 I0319 18:22:01.390953 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=e97ff531434ed6ec8777da42c51b1269fb479ef14e5ddd4a476e4aa0fddab9c9 I0319 18:22:01.391022 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=cf5f4534eb1c371469455d18773e01ea47c00af8a7bf07a5d039c2fb55d8fd0b I0319 18:22:01.391069 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=e10a9994b8de39dc2a6e69d5c55d0b0a04e9fd73af3fed82bfe9ecb6008ba9ee I0319 18:22:01.391123 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=0a7e521f0d87c831808ad3db7d28b3e2b0f45876c64e3a5365ad148a68ebbeb7 I0319 18:22:01.391161 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=e7e26f4ce5d9753f3607288322714adf522977148c740a852b55695b50da4848 I0319 18:22:01.391172 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 19.521224ms to process 11 records I0319 18:22:01.391183 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 21.423266ms to process 0 records I0319 18:22:01.391198 1 tasks_processing.go:71] worker 21 working on schedulers task. I0319 18:22:01.389143 1 tasks_processing.go:71] worker 28 working on machine_autoscalers task. I0319 18:22:01.392567 1 tasks_processing.go:71] worker 7 working on jaegers task. I0319 18:22:01.393203 1 recorder.go:75] Recording config/infrastructure with fingerprint=97f8ad4129efd47afa6fff22045d3257e74ae02e8d9ea1e86fc16ed9aec0c724 I0319 18:22:01.393222 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 8.92775ms to process 1 records I0319 18:22:01.398760 1 tasks_processing.go:71] worker 27 working on sap_pods task. I0319 18:22:01.398930 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=01306b8aa1d3bb8516f5d597feba293a82eb9a9ed0878c34cf3e94f474f46f7d I0319 18:22:01.398952 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=5233f5cd84261755c9cd79755916169ae4de754d45ea4b1de52a35abf3ea163b I0319 18:22:01.398958 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 14.796199ms to process 2 records I0319 18:22:01.399048 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/annotation.operator.tekton.dev with fingerprint=630f95af2b330022d539c3b0ac9d6297f189ed7b8638d2a2c73db698234b50db I0319 18:22:01.399123 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=670a02ed77b1e47b13d75ad19a9140bb4654ae7415bd7dceee9e50816effe427 I0319 18:22:01.399148 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/cert-manager-webhook with fingerprint=9ee7824282251097609cdb7f04596d1ed29ca330b2c9b718382f67a2a158702a I0319 18:22:01.399183 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/integration-service-mutating-webhook-configuration with fingerprint=efbbd37044f2c9623713d66c41ef633ebea30a8c55d5ea69cd0951af70bd0ba4 I0319 18:22:01.399215 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/proxy.operator.tekton.dev with fingerprint=900049686e86c1b384b64d6532a80b91758a52416e0db2f76bef5ec239af7eee I0319 18:22:01.399275 1 tasks_processing.go:71] worker 4 working on sap_config task. I0319 18:22:01.399278 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/release-service-mutating-webhook-configuration with fingerprint=3eddcf16feada605689a18632f9e9c0389f79cc67c753d7ce8f0789aa9e1d863 I0319 18:22:01.399498 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=ad0f50fa17a6989bc9f8f7c283c66e596058e02c81a39d57465db6acf8f09578 I0319 18:22:01.399546 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=4113b9b2bd117018074ee738b0f1d4369f00b08a29e69448df666b867d835bdc I0319 18:22:01.399599 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/webhook.operator.tekton.dev with fingerprint=6245a938c454f2a60214085adcc917a11cc69b97be2b9f86484a643a5a597f6b I0319 18:22:01.399669 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/webhook.pipeline.tekton.dev with fingerprint=25251fabee077fc814b07f6152b54b22009ed57e65f512e365a5af708e8acd87 I0319 18:22:01.399754 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/webhook.triggers.tekton.dev with fingerprint=386a0fca0493048d8807d8c46e002ad2f48e8fd9811ed44382cf57a8420c5495 I0319 18:22:01.399767 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 29.28587ms to process 11 records I0319 18:22:01.401055 1 tasks_processing.go:71] worker 9 working on machine_config_pools task. I0319 18:22:01.401393 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 10.701751ms to process 0 records I0319 18:22:01.401974 1 tasks_processing.go:71] worker 30 working on image task. I0319 18:22:01.402263 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0319 18:22:01.402324 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 32.852976ms to process 1 records I0319 18:22:01.403380 1 gather_cluster_operator_pods_and_events.go:121] Found 0 pods with 0 containers I0319 18:22:01.403438 1 tasks_processing.go:71] worker 12 working on openstack_dataplanenodesets task. I0319 18:22:01.403572 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 35.100919ms to process 0 records I0319 18:22:01.403622 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 11.026889ms to process 0 records I0319 18:22:01.403730 1 tasks_processing.go:71] worker 7 working on machine_sets task. I0319 18:22:01.403739 1 tasks_processing.go:71] worker 28 working on sap_datahubs task. I0319 18:22:01.404339 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 12.286245ms to process 0 records I0319 18:22:01.404488 1 tasks_processing.go:71] worker 29 working on oauths task. I0319 18:22:01.404838 1 recorder.go:75] Recording config/version with fingerprint=071f9dae31b3f364bfc295d55b515547a8dc9fd70e086e74ee1fb1405277431d I0319 18:22:01.404863 1 recorder.go:75] Recording config/id with fingerprint=7fd8f39dae63200289d00d1cfee99136f8f6d07283ed33fd38f9d48d93183b4d I0319 18:22:01.404872 1 gather.go:177] gatherer "clusterconfig" function "version" took 35.193787ms to process 2 records I0319 18:22:01.404883 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 13.635758ms to process 0 records I0319 18:22:01.404890 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 17.173231ms to process 0 records I0319 18:22:01.404895 1 tasks_processing.go:71] worker 24 working on openstack_dataplanedeployments task. I0319 18:22:01.404936 1 tasks_processing.go:71] worker 25 working on silenced_alerts task. I0319 18:22:01.409158 1 tasks_processing.go:74] worker 27 stopped. I0319 18:22:01.409203 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 10.334614ms to process 0 records I0319 18:22:01.409240 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 9.860477ms to process 0 records E0319 18:22:01.409254 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0319 18:22:01.409262 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 22.87014ms to process 0 records I0319 18:22:01.409266 1 tasks_processing.go:74] worker 4 stopped. I0319 18:22:01.409270 1 tasks_processing.go:74] worker 31 stopped. I0319 18:22:01.409571 1 tasks_processing.go:74] worker 14 stopped. I0319 18:22:01.409586 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 25.179471ms to process 0 records I0319 18:22:01.410382 1 tasks_processing.go:74] worker 9 stopped. I0319 18:22:01.410405 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 9.305065ms to process 0 records I0319 18:22:01.410616 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 6.714502ms to process 0 records I0319 18:22:01.410647 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 7.037815ms to process 0 records I0319 18:22:01.410675 1 tasks_processing.go:74] worker 12 stopped. I0319 18:22:01.410698 1 tasks_processing.go:74] worker 21 stopped. I0319 18:22:01.410701 1 tasks_processing.go:74] worker 7 stopped. I0319 18:22:01.410789 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=b014422f16d1821d1c54d311f6464a83ce4d5016dc2807049b40c5e2c05d7c7a I0319 18:22:01.410806 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 19.491932ms to process 1 records I0319 18:22:01.411010 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=c2ee9b82f56ef151198c7c525268c0077d232328a64534614d6d681254b83756 I0319 18:22:01.411073 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 22.282652ms to process 1 records I0319 18:22:01.411052 1 tasks_processing.go:74] worker 15 stopped. I0319 18:22:01.411111 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 6.984776ms to process 0 records I0319 18:22:01.411215 1 tasks_processing.go:74] worker 28 stopped. I0319 18:22:01.411262 1 tasks_processing.go:74] worker 13 stopped. I0319 18:22:01.411416 1 recorder.go:75] Recording config/authentication with fingerprint=d84d5562da495ae12bca9f73c385165c3b98e529fe3281c61ff0a56f0fa5f1f2 I0319 18:22:01.411430 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 20.291239ms to process 1 records I0319 18:22:01.411438 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 20.798777ms to process 0 records I0319 18:22:01.411518 1 tasks_processing.go:74] worker 5 stopped. I0319 18:22:01.411551 1 recorder.go:75] Recording config/image with fingerprint=d8eb33b6118dddb054a4530d7048f3fe55f49c17e7b54ea86702b471fd5d73cb I0319 18:22:01.411562 1 gather.go:177] gatherer "clusterconfig" function "image" took 9.162964ms to process 1 records I0319 18:22:01.411613 1 tasks_processing.go:74] worker 30 stopped. I0319 18:22:01.411761 1 tasks_processing.go:74] worker 19 stopped. I0319 18:22:01.412216 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=157ba7aaab6b4497e0c4b4d6d366335ac27ef10b3f429e177903308ba28b94d6 I0319 18:22:01.412688 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=af43ca10e6480d316326ed8401336d0ce35eea116d4877ba27aa4744616981f9 I0319 18:22:01.412752 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 42.489482ms to process 2 records I0319 18:22:01.412871 1 tasks_processing.go:74] worker 20 stopped. I0319 18:22:01.413346 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=4d9f616bcf7e14e7cd3aa4c57a9cd3b9dedee830ddd321ce0638e9ef632ee571 I0319 18:22:01.413659 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=a0586fb9fa4338762094a60160ed5f6551c9cdd4d8f296812cf6dc7dd8cd12d7 I0319 18:22:01.413680 1 gather.go:177] gatherer "clusterconfig" function "crds" took 27.884575ms to process 2 records I0319 18:22:01.413696 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 7.437534ms to process 0 records I0319 18:22:01.413702 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 43.909343ms to process 0 records I0319 18:22:01.413736 1 tasks_processing.go:74] worker 24 stopped. I0319 18:22:01.413751 1 tasks_processing.go:74] worker 17 stopped. I0319 18:22:01.413820 1 tasks_processing.go:74] worker 6 stopped. I0319 18:22:01.414078 1 recorder.go:75] Recording config/node/ip-10-0-0-199.ec2.internal with fingerprint=fe695857a052d8987179d8d46871d822b3674cfad4281a3d1f0c6b033aae962e I0319 18:22:01.414250 1 recorder.go:75] Recording config/node/ip-10-0-1-10.ec2.internal with fingerprint=3c4a344dd3d72387ecf235f105538010e872efbe3c496311d5b478b4cc31220d I0319 18:22:01.414340 1 recorder.go:75] Recording config/node/ip-10-0-2-8.ec2.internal with fingerprint=0f708378d0eab08645fb1320eafdb280d7cb285bbe3d17145a54e273e7b3ca5c I0319 18:22:01.414355 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 22.022393ms to process 3 records I0319 18:22:01.414461 1 tasks_processing.go:74] worker 29 stopped. I0319 18:22:01.414669 1 recorder.go:75] Recording config/oauth with fingerprint=a83528022b2643f1db2458a7f41cd8278cb8da80a3d6f979f865365969884c48 I0319 18:22:01.414685 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 8.261714ms to process 1 records I0319 18:22:01.440218 1 tasks_processing.go:74] worker 23 stopped. E0319 18:22:01.440244 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0319 18:22:01.440253 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2p4uhd2dagqt1ot08javpdvdjktnsnmq-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2p4uhd2dagqt1ot08javpdvdjktnsnmq-primary-cert-bundle-secret" not found I0319 18:22:01.440348 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=0f8380852e62f85157786d293ace06661768a14f74dc47d725c4425c8407d3cd I0319 18:22:01.440419 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 70.847604ms to process 1 records I0319 18:22:01.441716 1 tasks_processing.go:74] worker 25 stopped. I0319 18:22:01.441736 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0319 18:22:01.441747 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 36.762853ms to process 1 records I0319 18:22:01.441803 1 tasks_processing.go:74] worker 10 stopped. I0319 18:22:01.441863 1 recorder.go:75] Recording config/alerts with fingerprint=50a20572af400ce4e2a27e3d0798bf92edc012e69837f1406020e1bbada6390e I0319 18:22:01.441877 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 64.507817ms to process 1 records I0319 18:22:01.444040 1 tasks_processing.go:74] worker 16 stopped. E0319 18:22:01.444058 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0319 18:22:01.444065 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0319 18:22:01.444071 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0319 18:22:01.444083 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=c8a9805405aae7d04b7f03c09c51097942070877009b6daebd5573ef8127446e I0319 18:22:01.444146 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0319 18:22:01.444155 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0319 18:22:01.444165 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=3b70b9787487b548204d806ebebf90b0c2d460145e34cb33c0d8d3a7a011d907 I0319 18:22:01.444181 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0319 18:22:01.444245 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0319 18:22:01.444260 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0319 18:22:01.444268 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 67.490241ms to process 7 records I0319 18:22:01.448864 1 tasks_processing.go:74] worker 26 stopped. I0319 18:22:01.448890 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 80.569888ms to process 0 records I0319 18:22:01.452104 1 tasks_processing.go:74] worker 8 stopped. I0319 18:22:01.453475 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-5b5d9b4544-zdg82 with fingerprint=68e69a3d89c721c9d5a57d26098bf777b20f433989182eaf4fbf3d5e67936c1b I0319 18:22:01.453757 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-4lbnc with fingerprint=28fc561917ac624328ecf20f621546ab87a8e64671ea9fef2f79e9e9ce6b8625 I0319 18:22:01.454018 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-s89qg with fingerprint=6d83b222b1146af06869863d85535b82b215e41ee6e8c285ad1f4961608d629b I0319 18:22:01.454149 1 recorder.go:75] Recording config/running_containers with fingerprint=c3cfcb834054e4893b51ddc3b24e8d4a2436ef0ed1cde8617e112c86fc2e8378 I0319 18:22:01.454162 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 82.594113ms to process 4 records I0319 18:22:01.478975 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0319 18:22:01.482134 1 tasks_processing.go:74] worker 3 stopped. I0319 18:22:01.482335 1 recorder.go:75] Recording config/olm_operators with fingerprint=fada90ee3b45013fbcc995d4fe6a42b37b6016c1cae590139e5567416856e6f4 I0319 18:22:01.482349 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 113.132853ms to process 1 records I0319 18:22:01.482805 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0319 18:22:02.415039 1 tasks_processing.go:74] worker 0 stopped. I0319 18:22:02.415130 1 recorder.go:75] Recording config/metrics with fingerprint=164a2b6316b0f959afeb9a1a251e922fc601f5ef2b5dce0a73dae95d90280170 I0319 18:22:02.415147 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 1.035461196s to process 1 records I0319 18:22:02.420811 1 tasks_processing.go:74] worker 11 stopped. I0319 18:22:02.420833 1 recorder.go:75] Recording config/tsdb.json with fingerprint=ff0d31990f786eaea885859c24e817f9764b9a7935973059381278248722cdc5 I0319 18:22:02.420844 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 1.05252168s to process 1 records I0319 18:22:02.581807 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0319 18:22:02.581824 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0319 18:22:02.581827 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0319 18:22:02.581850 1 controller.go:554] The operator is healthy I0319 18:22:03.846011 1 tasks_processing.go:74] worker 22 stopped. I0319 18:22:03.846084 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=14d2c6f941ff13e00f40d72069e852351741f026244e8d57c7e5b12a09498128 I0319 18:22:03.846141 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0319 18:22:03.846174 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=499b37fa7ce906ed65ec6b07c49d05d6af203c48aa376bafbfb6977943620d34 I0319 18:22:03.846185 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0319 18:22:03.846207 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=2de4c888904af8ff68826099c47a3f2bda6eeb808b41f81afe4cfafba32f529c I0319 18:22:03.846224 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0319 18:22:03.846246 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=31b3dd4e8a58d7695edfccfb989364f8c99c58882ed4631fd4868d6525ff90bf I0319 18:22:03.846272 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=a95bceff169696ee83c3662be75f8ce2190c44974033c166e24fea4859512d34 I0319 18:22:03.846296 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=98d6c42729861beecb9269deaf1d9c7750b75369359e5d26ec2b78c28a5eb575 I0319 18:22:03.846326 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=1695c2ad31c051a5da93c6c87f0ff03c66a8d081352bba7407a79df15ddace6e I0319 18:22:03.847307 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=4fe3f89db4b88bf7aab0f7b35796b3896ed4a4525448ec9f8f6b8d3f718d3cf8 I0319 18:22:03.847384 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=3477fbd390c1cf0aaf9197427c35d9f5f55c3021bf5bdbde7e1bcfc75dc72ceb I0319 18:22:03.847412 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0319 18:22:03.847439 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=d568ad884aaa92b8d9eda62b31d1ec9e8ab5538702506d2f5f9e5bde61ab0e7f I0319 18:22:03.847454 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0319 18:22:03.847502 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=9c09353352f032cf8f0fd36ff4d3cce41eda5fa2fe7bd8fef13d601d018992f6 I0319 18:22:03.847522 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0319 18:22:03.847565 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=3fb089f5361693a1d44d30ab89943263cc945408d8b72609f47ce2c5fcba7a21 I0319 18:22:03.847585 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0319 18:22:03.847631 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=29c09b8ff48a72a816b0fd4a90ef75b30647c3663c827a24e82951b94e01bf13 I0319 18:22:03.847813 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=65843b096f530a5be2ae648058adadfc193d5a33f25cdc16c8b6c808346c89f5 I0319 18:22:03.847830 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0319 18:22:03.847844 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0319 18:22:03.847986 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0319 18:22:03.848038 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=3c885dd345eaded8856a19fa83e2818482ca712052d3e4b3362c6d0f6d860a94 I0319 18:22:03.848080 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=8135842f7a95d06bd8323dca0c483304502d94a41e68f83ecfc5dbde0e6a890b I0319 18:22:03.848097 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0319 18:22:03.848131 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=23466677f225092e34788cb449121f2fa46340bc3cd78433ecdfdd2642a9504d I0319 18:22:03.848147 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0319 18:22:03.848180 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=4751b2b1660d01548dcd7827ec40dbfc67915d5939db439e10ad90e7c95fd398 I0319 18:22:03.848203 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0319 18:22:03.848231 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=c343c97218803ccd46e3c7fb3552aac638d2833b595ba85b53f5cb8ad642f073 I0319 18:22:03.848260 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=122606b3181b55a56438529ffb9d35eb7212f710e40f89519e80ce72eb28ca29 I0319 18:22:03.848296 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=5dd7af0f69085c536e8e679ea031f1c6e83c7cc07aafbebc0c83289c7abfd410 I0319 18:22:03.848338 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=624d728d0d654e9fd438897e5f7f4e78a573a5aa055dc9c4ae0a7ab66a71490d I0319 18:22:03.848356 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0319 18:22:03.848416 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=f33d8e58d811e78a968f9193452276ee874a500cc882e17a92ca7405675582ae I0319 18:22:03.848448 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0319 18:22:03.848472 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0319 18:22:03.848487 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.476536611s to process 39 records I0319 18:22:04.023957 1 tasks_processing.go:74] worker 1 stopped. I0319 18:22:04.024331 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=08dbf2fd71cd723242b4f7d312e376a3ed5971e6566c37669e8cb14946e22583 I0319 18:22:04.024374 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 2.633133128s to process 1 records I0319 18:22:14.399888 1 tasks_processing.go:74] worker 2 stopped. I0319 18:22:14.399954 1 recorder.go:75] Recording config/installplans with fingerprint=7d8a656b78a9095bdaf60fc32ff4a9c86f335f4474e6edb0433b580daa979abe I0319 18:22:14.399967 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.031593605s to process 1 records I0319 18:22:15.174937 1 tasks_processing.go:74] worker 18 stopped. I0319 18:22:15.175291 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=74b727b232cc45b882d26cb89407f543533046358d1731eed12d9232af57da52 I0319 18:22:15.175309 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.806369184s to process 1 records E0319 18:22:15.175371 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.807s with: function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machine_configs\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error" I0319 18:22:15.176477 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "machine_configs" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error I0319 18:22:15.176492 1 periodic.go:216] Running workloads gatherer I0319 18:22:15.176509 1 tasks_processing.go:45] number of workers: 2 I0319 18:22:15.176521 1 tasks_processing.go:69] worker 1 listening for tasks. I0319 18:22:15.176527 1 tasks_processing.go:71] worker 1 working on workload_info task. I0319 18:22:15.176530 1 tasks_processing.go:69] worker 0 listening for tasks. I0319 18:22:15.176554 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0319 18:22:15.184343 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-199.ec2.internal... I0319 18:22:15.185682 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-8.ec2.internal... I0319 18:22:15.185716 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-10.ec2.internal... I0319 18:22:15.197502 1 gather_helm_info.go:102] unable to get HelmChart from services on tsf-keycloak from keycloak. I0319 18:22:15.197515 1 gather_helm_info.go:102] unable to get HelmChart from services on tsf-keycloak from keycloak-pgsql. I0319 18:22:15.197524 1 gather_helm_info.go:102] unable to get HelmChart from services on tsf-tpa from server. I0319 18:22:15.197532 1 gather_helm_info.go:102] unable to get HelmChart from services on tsf-tpa from tpa-pgsql. I0319 18:22:15.205203 1 gather_helm_info.go:102] unable to get HelmChart from deployments on tsf-keycloak from keycloak-pgsql-bee. I0319 18:22:15.205219 1 gather_helm_info.go:102] unable to get HelmChart from deployments on tsf-tpa from importer. I0319 18:22:15.205225 1 gather_helm_info.go:102] unable to get HelmChart from deployments on tsf-tpa from server. I0319 18:22:15.205231 1 gather_helm_info.go:102] unable to get HelmChart from deployments on tsf-tpa from tpa-pgsql-bee. I0319 18:22:15.205243 1 tasks_processing.go:74] worker 0 stopped. I0319 18:22:15.205255 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 28.664511ms to process 0 records I0319 18:22:23.800708 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 9s I0319 18:22:23.839735 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 31s for image data I0319 18:22:23.850407 1 gather_workloads_info.go:387] No image sha256:00fa6a8075723e279b420c76d13105c1650b1d46db51c7018801c558c0a876c6 (12ms) I0319 18:22:23.860929 1 gather_workloads_info.go:387] No image sha256:57a1aea49d7cc275b37b8f52d602a3a9d1601ec6a21a3268dd4903566cb2e335 (11ms) I0319 18:22:23.871720 1 gather_workloads_info.go:387] No image sha256:09c857f0c20721d6b447f5f567182befc1ca6157128225849117a5c830feab23 (11ms) I0319 18:22:23.880090 1 gather_workloads_info.go:387] No image sha256:17b8e04c7e711f5f9cb35a55644df5fc44f1eea5e5cd92c499d35ac81e86ed83 (8ms) I0319 18:22:23.889492 1 gather_workloads_info.go:387] No image sha256:f4e92bf3f35d86fe895a2e3225098b3d4d9dae720ef1d45e9efcf23dec8242b6 (9ms) I0319 18:22:23.897883 1 gather_workloads_info.go:387] No image sha256:d339ecf788d1681ba9945a2ea63ca675caec863ec75f497ef916103f4c898003 (8ms) I0319 18:22:23.905806 1 gather_workloads_info.go:387] No image sha256:a99f154140ac77b56e50ab7b98b1137d15085d7cfb603ee49bf366feaf80c49c (8ms) I0319 18:22:23.914080 1 gather_workloads_info.go:387] No image sha256:6521242b8805e70936f8e49c7d50a319adaeb73b4e5dec4693b0f93a34e397c5 (8ms) I0319 18:22:23.926034 1 gather_workloads_info.go:387] No image sha256:7dd9584d291967387f2a1ade991ad3bf8565bb59f49b013217951399e8273498 (12ms) I0319 18:22:23.935072 1 gather_workloads_info.go:387] No image sha256:12e6985890d3c549f7cc9881892793b27277f72e2739cb910aaf9f4dd7215df1 (9ms) I0319 18:22:23.947314 1 gather_workloads_info.go:387] No image sha256:59b844637eca688cc22c7841bc4480a26d1384444bf9b0161261a182f87a1262 (12ms) I0319 18:22:24.048950 1 gather_workloads_info.go:387] No image sha256:0b2cc6aadb1abc1d35a55da0c9884eb9426d7ff7a97174daeb09bb1e1a23cfae (102ms) I0319 18:22:24.147958 1 gather_workloads_info.go:387] No image sha256:ada1a963ce2c45d2bba08051c1e0e2db88b200e9d616fab8fde10b698101fc69 (99ms) I0319 18:22:24.249835 1 gather_workloads_info.go:387] No image sha256:07789f6e6e3c3a5a974c106b43c2e8bc60edb3321def1f0883f3fcd35a238ed0 (102ms) I0319 18:22:24.355887 1 gather_workloads_info.go:387] No image sha256:54440fa6cb1b01a6b8952f700677832d7b7d3b1b58934957f504ca885730844a (106ms) I0319 18:22:24.447690 1 gather_workloads_info.go:387] No image sha256:9746960bbc79e0ecf82a0ee12f878e90e202247dcaeb046bdd11db48a52ccb90 (92ms) I0319 18:22:24.552937 1 gather_workloads_info.go:387] No image sha256:d6af2e6496cdd7a3a8d49bc029fde9530e11df443f12ce8da6fb7f1ae8a6ec0b (105ms) I0319 18:22:24.648292 1 gather_workloads_info.go:387] No image sha256:07f14370b57046dc2ac49be933bb2248aaf5943e8fb56ce9703eaa7c673b999f (95ms) I0319 18:22:24.751473 1 gather_workloads_info.go:387] No image sha256:862d5befeba504d5058264cf65f16fc307eea4eb3914fbbbf40fbe61714b0958 (103ms) I0319 18:22:24.867284 1 gather_workloads_info.go:387] No image sha256:6ca67bb30984d15232085f580f46bf540f4e3b0ee25970641b912fbd28bd0754 (116ms) I0319 18:22:24.948308 1 gather_workloads_info.go:387] No image sha256:124f1b391b0f376a5deffd05b29ef0896ca0912e1f67b3a2051ed6678c54a57e (81ms) I0319 18:22:25.048632 1 gather_workloads_info.go:387] No image sha256:4f86f5c692a497442b54c435e68b0cbf961591a00ebc3a73c364a19da673e34d (100ms) I0319 18:22:25.147124 1 gather_workloads_info.go:387] No image sha256:41e0dcd2f91df9234249f168b3fbe6d55cfdb3ebd2e598d5edfa47f905c046cb (98ms) I0319 18:22:25.249560 1 gather_workloads_info.go:387] No image sha256:3971738912069448174202486b61ed384153ca18af3e8430a55795a6e65eb58d (102ms) I0319 18:22:25.350576 1 gather_workloads_info.go:387] No image sha256:7ea1ff613255c1ee67b34ec82b68ec6f5a4a6d283620a8e0ad142d4a31eb9211 (101ms) I0319 18:22:25.446988 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (96ms) I0319 18:22:25.549817 1 gather_workloads_info.go:387] No image sha256:04ee35aa6f87e954d386228808017cf2938c5a96a052059c6767078d5f3ad8d6 (103ms) I0319 18:22:25.650284 1 gather_workloads_info.go:387] No image sha256:2c4ec0dfd62375ed45d5758798b1069e16a0a6d1986439d23c61591c4dfb9297 (100ms) I0319 18:22:25.749287 1 gather_workloads_info.go:387] No image sha256:3f586d6fd6139bbad131716168590b12db0cdfe4dcbb15ccee65b080d5594b6e (99ms) I0319 18:22:25.848113 1 gather_workloads_info.go:387] No image sha256:a8493aa6857ea8d73dedb7f6891e6fd31cf91528d2ee12bd77495feb383bb4ad (99ms) I0319 18:22:25.947841 1 gather_workloads_info.go:387] No image sha256:8f13a1e673c882ee926e97270249a20d3c40797a8c7c5abb2fda9c98c24a2eb4 (100ms) I0319 18:22:26.049799 1 gather_workloads_info.go:387] No image sha256:4dcd0b18dea61968f20d49c4372eec43689eb52e823e1a6293e9b89c6e21e587 (102ms) I0319 18:22:26.148946 1 gather_workloads_info.go:387] No image sha256:3d68a9523f4c5c22a48dfa151bbfde78b2f360697afe3b9d8bfb53e8c359bf95 (99ms) I0319 18:22:26.247622 1 gather_workloads_info.go:387] No image sha256:9db50052057ad4ec94099bc3c9ab476f5ac7b00954c99cb0b26fe729abba3387 (99ms) I0319 18:22:26.349484 1 gather_workloads_info.go:387] No image sha256:3e889ea85c9c4748c548b84ae135a8adc88f42026ce8073538ca2fe7d89b8004 (102ms) I0319 18:22:26.449439 1 gather_workloads_info.go:387] No image sha256:317131706dcce9c87eb4e1efb8446bdbf5e0726e120a8f6c655e2acd89d60e2d (100ms) I0319 18:22:26.548109 1 gather_workloads_info.go:387] No image sha256:2af652f68b17ed49c2565928e4c101880e949c4484c6ee958793a8e668e47bea (99ms) I0319 18:22:26.647973 1 gather_workloads_info.go:387] No image sha256:2f6f91dfd7b71cad1c6db82835d709d20f4153cf500d60ee32145712af922214 (100ms) I0319 18:22:26.750922 1 gather_workloads_info.go:387] No image sha256:2ba73aa991da589b1ee1fa6903abcdd56387007f3d650d345fbc8458bc2baeac (103ms) I0319 18:22:26.849489 1 gather_workloads_info.go:387] No image sha256:25d9148ea34149935323231c64952679727b6f7f235febc3be1df63e0bf18c5e (99ms) I0319 18:22:26.950392 1 gather_workloads_info.go:387] No image sha256:1bf773bc4ec02bf6d68c82f32175366b01fb24f6da8c724b7621748692aec7dc (101ms) I0319 18:22:27.050130 1 gather_workloads_info.go:387] No image sha256:d876a5e41b8467cdde921032f2cd53e77bef99ebcd8b61d72a3ad411469ad352 (100ms) I0319 18:22:27.147942 1 gather_workloads_info.go:387] No image sha256:1f4a37529de4f750099af18e113556bd995fe6922030c614b7d6d23eaa0bd318 (98ms) I0319 18:22:27.249184 1 gather_workloads_info.go:387] No image sha256:95801a943ba7d0ec4af53c3cd1177483e649bcd528b5a838a32ad2d860e6503a (101ms) I0319 18:22:27.348632 1 gather_workloads_info.go:387] No image sha256:5efa227ddac82e56eddc03761e220af1f2d6a4b22ee00b2baf426523f26c0319 (99ms) I0319 18:22:27.447215 1 gather_workloads_info.go:387] No image sha256:1aabd00f813e1afccdab16a845e6346288f5cf7e320fa5bb97c683702191c575 (99ms) I0319 18:22:27.550987 1 gather_workloads_info.go:387] No image sha256:6d83c7e0e068178360998c815a3e0c729ca80a349954172aadafad5da5180d0c (104ms) I0319 18:22:27.648042 1 gather_workloads_info.go:387] No image sha256:72025568de4c6786f56594714efbe3c5db3c2791fb1a7a47e6361e5b53a35107 (97ms) I0319 18:22:27.748353 1 gather_workloads_info.go:387] No image sha256:27f3be442cf1d9f640657fb6d7cea6b133652226f9a80c3442df226b3cb3c2dc (100ms) I0319 18:22:27.848001 1 gather_workloads_info.go:387] No image sha256:21f434602c18e048720c594186f117e230abdeae298a0f363dea36e02b7fb77b (100ms) I0319 18:22:27.949408 1 gather_workloads_info.go:387] No image sha256:30a054cfe1b8330bd57d762921ca5cd281ad2fdd7994fd3093ffadb5d767a564 (101ms) I0319 18:22:28.048420 1 gather_workloads_info.go:387] No image sha256:b336f6e44314dd23a21236118a604411522baf5e63732463f1d08daccb417f39 (99ms) I0319 18:22:28.148683 1 gather_workloads_info.go:387] No image sha256:60648d27017709a420e43353f9a83d7c466017b3b771200672a4dbc3c3f7a3ae (100ms) I0319 18:22:28.247829 1 gather_workloads_info.go:387] No image sha256:11de6fe1741f5dd485e59ee2e703868badda81de0a179513be70b6ac9fc4655f (99ms) I0319 18:22:28.351350 1 gather_workloads_info.go:387] No image sha256:03a7d45ceb838c746bdcdac1a5cc02d26273b319d9da4498bb3ee31c3e130b43 (103ms) I0319 18:22:28.455313 1 gather_workloads_info.go:387] No image sha256:9dee9b6866db30456c277ff1f7a114dd4ce25bff8aced70218623af9423ec3cf (104ms) I0319 18:22:28.551422 1 gather_workloads_info.go:387] No image sha256:2a4f6a54cd5135c8121832236063419071fa5d00c10a051c197d3d01370e4d9c (96ms) I0319 18:22:28.661675 1 gather_workloads_info.go:387] No image sha256:7db22f56cf9d223b1437e8cba3aa9e04e8ae8b973e23e85133bf1d3d8d66f15e (110ms) I0319 18:22:28.752760 1 gather_workloads_info.go:387] No image sha256:7c6cba78fb26addd9f056ec3f8b9376666db353451da37a4681a51d16f2ff76c (91ms) I0319 18:22:28.847995 1 gather_workloads_info.go:398] Found image sha256:670004bffddb41c787b9f9a5c7d1615453d4acfa6452dedfa61ad2f305eea814 (95ms) I0319 18:22:28.948466 1 gather_workloads_info.go:387] No image sha256:04156cd38c436fb6692930d18af59619b0669b2d80cd3590d0973c0e038c5d6d (100ms) I0319 18:22:29.047552 1 gather_workloads_info.go:387] No image sha256:188e4e2d697e2ca3c7c16772c0ab12253f00bd8433d8691ed7058fe7bafbbbf4 (99ms) I0319 18:22:29.147468 1 gather_workloads_info.go:387] No image sha256:3666bcb850e1518aec86d322af52e95f725602ad0ab770d2b1e5b39b6dbde0b5 (100ms) I0319 18:22:29.247659 1 gather_workloads_info.go:387] No image sha256:39c72cf5b519408d1659bb630ac4ea5f9d067b049019fe4d1b5cc7680afa9060 (100ms) I0319 18:22:29.347329 1 gather_workloads_info.go:387] No image sha256:04c7a2673571e9069c934d86b4308c0f57b2715b45db500837b7708ae1f20ae7 (100ms) I0319 18:22:29.447572 1 gather_workloads_info.go:387] No image sha256:315260877c58d13e0511bbff7bdcc389bcd86deca48785023e28194b08c4c924 (100ms) I0319 18:22:29.547468 1 gather_workloads_info.go:387] No image sha256:83a8710df2032471c379f4cfbb3861ec9c4c7794f8b487483dbfb8cf57207750 (100ms) I0319 18:22:29.647294 1 gather_workloads_info.go:387] No image sha256:05376e7e2304d22dd866d8e25dd1e5ff8988b2232059fadfd12cb440af785635 (100ms) I0319 18:22:29.747355 1 gather_workloads_info.go:387] No image sha256:2962c6c31d7078b34cf7ba0c9b0ea685dc539cea67b06bd38b6ac566298bb790 (100ms) I0319 18:22:29.849126 1 gather_workloads_info.go:387] No image sha256:79fca57a4fd5ef00472a287218b84061b941905c61fcc55e89073dd93fa77648 (102ms) I0319 18:22:29.946737 1 gather_workloads_info.go:387] No image sha256:75f95da9eb6a94452c1eb45339c3687fcf60c0d847c961f1c4302d14c1ecde3a (98ms) I0319 18:22:30.047450 1 gather_workloads_info.go:387] No image sha256:11a864539ae77a94afcf8d2e1abf9168081c5081a1bc06088f4be5206cdc02df (101ms) I0319 18:22:30.148113 1 gather_workloads_info.go:387] No image sha256:120485f6d677cfeaae0509169f46df14f3ebfafd87b3d876dfa59b738f846c77 (101ms) I0319 18:22:30.247416 1 gather_workloads_info.go:387] No image sha256:47a0b3f12211320d1828524a324ab3ec9deac97c17b9d3f056c87d3384d9eb79 (99ms) I0319 18:22:30.347554 1 gather_workloads_info.go:387] No image sha256:0c90ca6acbe2f4fdbd253ae22f5b3286247c6940321f7d2b9801aea21ef5269f (100ms) I0319 18:22:30.449489 1 gather_workloads_info.go:398] Found image sha256:db40a3d4e50016c645fdb4a1f389dfcdbc9febc10c174816d681abe5c14153cd (102ms) I0319 18:22:30.547762 1 gather_workloads_info.go:387] No image sha256:7f426776d3ef70b585018b8d813f5eedd70373536681c6240df224ebf9ff6329 (98ms) I0319 18:22:30.647066 1 gather_workloads_info.go:387] No image sha256:3e262d5207d5e2077e4a14f4fbddc70d4d2651dbaaf955147013feb58a18cddd (99ms) I0319 18:22:30.748954 1 gather_workloads_info.go:387] No image sha256:bbb5af28bbed69e8189825fdc6665f4e8dbcd0c0c83fa479943ea102fe9452d6 (102ms) I0319 18:22:30.847572 1 gather_workloads_info.go:387] No image sha256:7261ee18d6fd8d42614e94ae3bdb77c5acad54f2b9898365bf8668c60a32589a (99ms) I0319 18:22:30.946582 1 gather_workloads_info.go:387] No image sha256:1ff360b712000d39cbb58158186ff0a18764278e6fa98ef9f21d113b978d9a79 (99ms) I0319 18:22:31.048608 1 gather_workloads_info.go:387] No image sha256:384e639f2ce3e75712a5ee1cc5520161342e9dd1e94444d134d7ad4a39dde279 (102ms) I0319 18:22:31.147809 1 gather_workloads_info.go:387] No image sha256:a9dc28a6962d195dcce0c76a544aad5f784dd90b98282c169d7b94c870a85f8b (99ms) I0319 18:22:31.248168 1 gather_workloads_info.go:387] No image sha256:e4bb09dfe4513cdbe349507495c7f4044623a3c2aff866b7946d941e82a7a639 (100ms) I0319 18:22:31.349149 1 gather_workloads_info.go:387] No image sha256:43efa5a676666dd4ae039ad5518eaf636f5baf8d688d9a4b475822c32ad1ddfb (101ms) I0319 18:22:31.448401 1 gather_workloads_info.go:387] No image sha256:876fb57ffb255ed75a452405bcd1d116788a34b19264886e6e087ff544c17083 (99ms) I0319 18:22:31.548182 1 gather_workloads_info.go:387] No image sha256:1e9a9f1b5ecdab7d074722c860daab977519123fe4385affdfd86d8c7f3c473c (100ms) I0319 18:22:31.647430 1 gather_workloads_info.go:387] No image sha256:d88f384ee9cf4475b16d9589e88b28c9f2407247fbedfa3d9bfcaed95753514e (99ms) I0319 18:22:31.747782 1 gather_workloads_info.go:387] No image sha256:1c325fa866a794c56a7b30ca9d4fe225606566651bb12a2ecce8f60cbddb2809 (100ms) I0319 18:22:31.848253 1 gather_workloads_info.go:387] No image sha256:6a0d1f472283c686e438dbfbeec826db9d7d95f45f92842c4073f70704860d2f (100ms) I0319 18:22:31.947431 1 gather_workloads_info.go:387] No image sha256:50b7c5185c4a6e63145753415c92723c59136d14b951ea0f4f06739b00893329 (99ms) I0319 18:22:32.048337 1 gather_workloads_info.go:387] No image sha256:17e68da37c74bdc95a4eb43b51ce671027807ee610a4ee5237324a8c0dc9d044 (101ms) I0319 18:22:32.147683 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (99ms) I0319 18:22:32.247112 1 gather_workloads_info.go:387] No image sha256:240a9553315990a06a9d52eaf6e96e3aa1c743f1fbff33b95b489d41cef18f5a (99ms) I0319 18:22:32.347716 1 gather_workloads_info.go:387] No image sha256:3e772c41ef85aaccf63c409ee72b43d632b36d4cc66a3790835dffdcf9b7433e (101ms) I0319 18:22:32.447540 1 gather_workloads_info.go:387] No image sha256:050fe6fe1298c45a27982e3f4d0afe637ce01d9b394e85b7eb202309ecdd0731 (100ms) I0319 18:22:32.547613 1 gather_workloads_info.go:387] No image sha256:b8d380990acb0184ab685b005e22fb8369139dd8cd6550b4521bd4bc3ca23b40 (100ms) I0319 18:22:32.647985 1 gather_workloads_info.go:387] No image sha256:1ca6fa787c621462fa2c67b9a42b1de67688883af90316475b0df35be3ed805d (100ms) I0319 18:22:32.648034 1 tasks_processing.go:74] worker 1 stopped. I0319 18:22:32.648545 1 recorder.go:75] Recording config/workload_info with fingerprint=e8b3cfc8d7f61c6ec8edbee97b5a4d7603324189e2f1bc65f16f0f3e2e08f544 I0319 18:22:32.648567 1 gather.go:177] gatherer "workloads" function "workload_info" took 17.471499584s to process 1 records I0319 18:22:32.648584 1 periodic.go:263] Periodic gather workloads completed in 17.472s I0319 18:22:32.648594 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0319 18:22:32.648599 1 periodic.go:216] Running conditional gatherer I0319 18:22:32.655124 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.5/gathering_rules I0319 18:22:32.795868 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0319 18:22:32.797938 1 conditional_gatherer.go:374] alert "Watchdog" has state "firing" I0319 18:22:32.797954 1 conditional_gatherer.go:374] alert "AlertmanagerReceiversNotConfigured" has state "firing" I0319 18:22:32.798031 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0319 18:22:32.803798 1 conditional_gatherer.go:392] cluster version is '4.21.5' I0319 18:22:32.803830 1 tasks_processing.go:45] number of workers: 3 I0319 18:22:32.803844 1 tasks_processing.go:69] worker 2 listening for tasks. I0319 18:22:32.803850 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0319 18:22:32.803858 1 tasks_processing.go:69] worker 0 listening for tasks. I0319 18:22:32.803871 1 tasks_processing.go:69] worker 1 listening for tasks. I0319 18:22:32.803880 1 tasks_processing.go:74] worker 1 stopped. I0319 18:22:32.803873 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0319 18:22:32.803889 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0319 18:22:32.803966 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0319 18:22:32.803984 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.488µs to process 1 records I0319 18:22:32.804028 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0319 18:22:32.804040 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.488µs to process 1 records I0319 18:22:32.804048 1 tasks_processing.go:74] worker 0 stopped. I0319 18:22:32.804178 1 tasks_processing.go:74] worker 2 stopped. I0319 18:22:32.804192 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 277.854µs to process 0 records I0319 18:22:32.804207 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0319 18:22:32.804226 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0319 18:22:32.829155 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=20d566ab734a10aa456db23664756f8b08f5131d5392fe51a81375d231ea9ea4 I0319 18:22:32.829292 1 diskrecorder.go:70] Writing 136 records to /var/lib/insights-operator/insights-2026-03-19-182232.tar.gz I0319 18:22:32.839365 1 diskrecorder.go:51] Wrote 136 records to disk in 10ms I0319 18:22:32.839395 1 periodic.go:285] Gathering cluster info every 2h0m0s I0319 18:22:32.839408 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0319 18:23:31.368368 1 diskrecorder.go:223] Found files to send: insights-2026-03-19-182232.tar.gz I0319 18:23:31.368397 1 insightsuploader.go:150] Checking archives to upload periodically every 16m9.998743404s I0319 18:23:31.368405 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0319 18:23:31.377919 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0319 18:23:31.573958 1 requests.go:87] Successfully reported id=2026-03-19T18:23:31Z x-rh-insights-request-id=1a95eab23879465d8c9577f65f1685ab, wrote=75472 I0319 18:23:31.574041 1 insightsuploader.go:187] Uploaded report successfully in 205.615929ms I0319 18:23:31.574065 1 controller.go:129] Initializing last reported time to 2026-03-19T18:23:31Z I0319 18:23:31.574131 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0319 18:23:31.574139 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0319 18:23:31.574147 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0319 18:23:31.582042 1 controller.go:554] The operator is healthy I0319 18:24:02.582736 1 controller.go:554] The operator is healthy I0319 18:24:32.182944 1 insightsreport.go:137] Pulling report from smart-proxy I0319 18:24:32.182971 1 insightsreport.go:149] Retrieving report I0319 18:24:32.189315 1 requests.go:110] Retrieving report for cluster: 92f28552-5f52-4204-b7d7-fa8e0acf2bdc I0319 18:24:32.189328 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/92f28552-5f52-4204-b7d7-fa8e0acf2bdc/reports I0319 18:24:32.192804 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/92f28552-5f52-4204-b7d7-fa8e0acf2bdc/reports I0319 18:24:32.368372 1 insightsreport.go:184] Report retrieved I0319 18:24:32.413191 1 insightsreport.go:239] Report retrieved correctly I0319 18:24:38.443092 1 configmapobserver.go:84] configmaps "insights-config" not found I0319 18:24:38.476170 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0319 18:24:38.480669 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0319 18:24:38.480713 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0319 18:24:38.485039 1 secretconfigobserver.go:119] support secret does not exist I0319 18:26:02.585701 1 controller.go:554] The operator is healthy I0319 18:28:02.586355 1 controller.go:554] The operator is healthy