W0423 19:36:40.814384 1 cmd.go:257] Using insecure, self-signed certificates I0423 19:36:41.445123 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:36:41.445504 1 observer_polling.go:159] Starting file observer I0423 19:36:42.760678 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0423 19:36:42.760867 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0423 19:36:42.761428 1 secure_serving.go:57] Forcing use of http/1.1 only I0423 19:36:42.761441 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0423 19:36:42.761447 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0423 19:36:42.761473 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0423 19:36:42.761499 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0423 19:36:42.761504 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0423 19:36:42.761508 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0423 19:36:42.761512 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0423 19:36:42.766220 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0423 19:36:42.766406 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"299ddc53-1bf3-41b8-84c8-cb33e59d56cb", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0423 19:36:42.766816 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0423 19:36:42.766838 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0423 19:36:42.766841 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0423 19:36:42.766839 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0423 19:36:42.766859 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 19:36:42.766865 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 19:36:42.767103 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-2625966115/tls.crt::/tmp/serving-cert-2625966115/tls.key" I0423 19:36:42.767323 1 secure_serving.go:213] Serving securely on [::]:8443 I0423 19:36:42.767352 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0423 19:36:42.770740 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0423 19:36:42.770760 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 19:36:42.770918 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0423 19:36:42.775698 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 19:36:42.775719 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 19:36:42.779322 1 secretconfigobserver.go:119] support secret does not exist I0423 19:36:42.783279 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 19:36:42.787560 1 secretconfigobserver.go:119] support secret does not exist I0423 19:36:42.788879 1 recorder.go:161] Pruning old reports every 8h1m35s, max age is 288h0m0s I0423 19:36:42.793086 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0423 19:36:42.793101 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0423 19:36:42.793114 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0423 19:36:42.793137 1 insightsreport.go:296] Starting report retriever I0423 19:36:42.793144 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0423 19:36:42.793124 1 periodic.go:209] Running clusterconfig gatherer I0423 19:36:42.793182 1 tasks_processing.go:45] number of workers: 64 I0423 19:36:42.793211 1 tasks_processing.go:69] worker 14 listening for tasks. I0423 19:36:42.793223 1 tasks_processing.go:69] worker 7 listening for tasks. I0423 19:36:42.793226 1 tasks_processing.go:71] worker 14 working on openshift_machine_api_events task. I0423 19:36:42.793230 1 tasks_processing.go:69] worker 38 listening for tasks. I0423 19:36:42.793230 1 tasks_processing.go:69] worker 6 listening for tasks. I0423 19:36:42.793235 1 tasks_processing.go:71] worker 38 working on ceph_cluster task. I0423 19:36:42.793233 1 tasks_processing.go:69] worker 11 listening for tasks. I0423 19:36:42.793240 1 tasks_processing.go:69] worker 12 listening for tasks. I0423 19:36:42.793241 1 tasks_processing.go:69] worker 10 listening for tasks. I0423 19:36:42.793245 1 tasks_processing.go:69] worker 13 listening for tasks. I0423 19:36:42.793246 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 19:36:42.793252 1 tasks_processing.go:69] worker 8 listening for tasks. I0423 19:36:42.793255 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 19:36:42.793258 1 tasks_processing.go:69] worker 9 listening for tasks. I0423 19:36:42.793260 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 19:36:42.793256 1 tasks_processing.go:69] worker 3 listening for tasks. I0423 19:36:42.793267 1 tasks_processing.go:69] worker 4 listening for tasks. I0423 19:36:42.793272 1 tasks_processing.go:69] worker 32 listening for tasks. I0423 19:36:42.793272 1 tasks_processing.go:69] worker 5 listening for tasks. I0423 19:36:42.793277 1 tasks_processing.go:69] worker 17 listening for tasks. I0423 19:36:42.793277 1 tasks_processing.go:69] worker 27 listening for tasks. I0423 19:36:42.793277 1 tasks_processing.go:69] worker 20 listening for tasks. I0423 19:36:42.793282 1 tasks_processing.go:69] worker 23 listening for tasks. I0423 19:36:42.793290 1 tasks_processing.go:69] worker 28 listening for tasks. I0423 19:36:42.793291 1 tasks_processing.go:69] worker 24 listening for tasks. I0423 19:36:42.793297 1 tasks_processing.go:69] worker 18 listening for tasks. I0423 19:36:42.793275 1 tasks_processing.go:69] worker 26 listening for tasks. I0423 19:36:42.793299 1 tasks_processing.go:69] worker 16 listening for tasks. I0423 19:36:42.793305 1 tasks_processing.go:69] worker 19 listening for tasks. I0423 19:36:42.793293 1 tasks_processing.go:69] worker 15 listening for tasks. I0423 19:36:42.793307 1 tasks_processing.go:69] worker 31 listening for tasks. I0423 19:36:42.793309 1 tasks_processing.go:69] worker 33 listening for tasks. I0423 19:36:42.793312 1 tasks_processing.go:69] worker 34 listening for tasks. I0423 19:36:42.793296 1 tasks_processing.go:69] worker 29 listening for tasks. I0423 19:36:42.793302 1 tasks_processing.go:69] worker 30 listening for tasks. I0423 19:36:42.793311 1 tasks_processing.go:69] worker 25 listening for tasks. I0423 19:36:42.793321 1 tasks_processing.go:69] worker 48 listening for tasks. I0423 19:36:42.793322 1 tasks_processing.go:69] worker 52 listening for tasks. I0423 19:36:42.793323 1 tasks_processing.go:69] worker 54 listening for tasks. I0423 19:36:42.793329 1 tasks_processing.go:69] worker 41 listening for tasks. I0423 19:36:42.793331 1 tasks_processing.go:69] worker 53 listening for tasks. I0423 19:36:42.793332 1 tasks_processing.go:69] worker 47 listening for tasks. I0423 19:36:42.793331 1 tasks_processing.go:69] worker 49 listening for tasks. I0423 19:36:42.793337 1 tasks_processing.go:69] worker 56 listening for tasks. I0423 19:36:42.793331 1 tasks_processing.go:69] worker 57 listening for tasks. I0423 19:36:42.793343 1 tasks_processing.go:69] worker 62 listening for tasks. I0423 19:36:42.793344 1 tasks_processing.go:69] worker 61 listening for tasks. I0423 19:36:42.793344 1 tasks_processing.go:69] worker 59 listening for tasks. I0423 19:36:42.793311 1 tasks_processing.go:69] worker 51 listening for tasks. I0423 19:36:42.793343 1 tasks_processing.go:69] worker 55 listening for tasks. I0423 19:36:42.793353 1 tasks_processing.go:69] worker 37 listening for tasks. I0423 19:36:42.793313 1 tasks_processing.go:69] worker 44 listening for tasks. I0423 19:36:42.793310 1 tasks_processing.go:69] worker 22 listening for tasks. I0423 19:36:42.793297 1 tasks_processing.go:69] worker 35 listening for tasks. I0423 19:36:42.793319 1 tasks_processing.go:69] worker 39 listening for tasks. I0423 19:36:42.793320 1 tasks_processing.go:69] worker 45 listening for tasks. I0423 19:36:42.793323 1 tasks_processing.go:69] worker 36 listening for tasks. I0423 19:36:42.793302 1 tasks_processing.go:69] worker 21 listening for tasks. I0423 19:36:42.793324 1 tasks_processing.go:69] worker 40 listening for tasks. I0423 19:36:42.793329 1 tasks_processing.go:69] worker 50 listening for tasks. I0423 19:36:42.793331 1 tasks_processing.go:69] worker 46 listening for tasks. I0423 19:36:42.793337 1 tasks_processing.go:69] worker 42 listening for tasks. I0423 19:36:42.793337 1 tasks_processing.go:69] worker 60 listening for tasks. I0423 19:36:42.793342 1 tasks_processing.go:69] worker 63 listening for tasks. I0423 19:36:42.793342 1 tasks_processing.go:69] worker 43 listening for tasks. I0423 19:36:42.793342 1 tasks_processing.go:69] worker 58 listening for tasks. I0423 19:36:42.793227 1 tasks_processing.go:71] worker 7 working on service_accounts task. I0423 19:36:42.793414 1 tasks_processing.go:71] worker 13 working on authentication task. I0423 19:36:42.793414 1 tasks_processing.go:71] worker 11 working on machine_configs task. I0423 19:36:42.793421 1 tasks_processing.go:71] worker 58 working on machine_sets task. I0423 19:36:42.793425 1 tasks_processing.go:71] worker 57 working on nodenetworkconfigurationpolicies task. I0423 19:36:42.793430 1 tasks_processing.go:71] worker 24 working on active_alerts task. I0423 19:36:42.793432 1 tasks_processing.go:71] worker 50 working on machine_autoscalers task. I0423 19:36:42.793441 1 tasks_processing.go:71] worker 29 working on jaegers task. I0423 19:36:42.793454 1 tasks_processing.go:71] worker 10 working on openstack_version task. I0423 19:36:42.793460 1 tasks_processing.go:71] worker 44 working on operators_pods_and_events task. I0423 19:36:42.793466 1 tasks_processing.go:71] worker 30 working on sap_pods task. I0423 19:36:42.793470 1 tasks_processing.go:71] worker 61 working on schedulers task. I0423 19:36:42.793476 1 tasks_processing.go:71] worker 23 working on aggregated_monitoring_cr_names task. I0423 19:36:42.793511 1 tasks_processing.go:71] worker 54 working on openstack_controlplanes task. I0423 19:36:42.793527 1 tasks_processing.go:71] worker 59 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0423 19:36:42.793534 1 tasks_processing.go:71] worker 51 working on machine_config_pools task. I0423 19:36:42.793541 1 tasks_processing.go:71] worker 26 working on lokistack task. W0423 19:36:42.793457 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:36:42.793554 1 tasks_processing.go:71] worker 24 working on config_maps task. I0423 19:36:42.793414 1 tasks_processing.go:71] worker 6 working on image_pruners task. I0423 19:36:42.793665 1 tasks_processing.go:71] worker 36 working on machine_healthchecks task. I0423 19:36:42.793674 1 tasks_processing.go:71] worker 21 working on install_plans task. I0423 19:36:42.793690 1 tasks_processing.go:71] worker 35 working on container_images task. I0423 19:36:42.793726 1 tasks_processing.go:71] worker 45 working on pod_network_connectivity_checks task. I0423 19:36:42.793731 1 tasks_processing.go:71] worker 16 working on image task. I0423 19:36:42.793733 1 tasks_processing.go:71] worker 48 working on silenced_alerts task. I0423 19:36:42.793736 1 tasks_processing.go:71] worker 52 working on certificate_signing_requests task. I0423 19:36:42.793422 1 tasks_processing.go:71] worker 39 working on image_registries task. I0423 19:36:42.793744 1 tasks_processing.go:71] worker 2 working on dvo_metrics task. I0423 19:36:42.793425 1 tasks_processing.go:71] worker 28 working on olm_operators task. W0423 19:36:42.793759 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:36:42.793731 1 tasks_processing.go:71] worker 8 working on nodenetworkstates task. I0423 19:36:42.793782 1 tasks_processing.go:71] worker 3 working on ingress task. I0423 19:36:42.793843 1 tasks_processing.go:71] worker 31 working on pdbs task. I0423 19:36:42.793537 1 tasks_processing.go:71] worker 18 working on openstack_dataplanenodesets task. I0423 19:36:42.793881 1 tasks_processing.go:71] worker 33 working on validating_webhook_configurations task. I0423 19:36:42.793903 1 tasks_processing.go:71] worker 37 working on nodes task. I0423 19:36:42.793732 1 tasks_processing.go:71] worker 25 working on sap_datahubs task. I0423 19:36:42.794002 1 tasks_processing.go:71] worker 19 working on operators task. I0423 19:36:42.794025 1 tasks_processing.go:71] worker 40 working on monitoring_persistent_volumes task. I0423 19:36:42.793736 1 tasks_processing.go:71] worker 1 working on storage_classes task. I0423 19:36:42.794040 1 tasks_processing.go:71] worker 46 working on qemu_kubevirt_launcher_logs task. I0423 19:36:42.794048 1 tasks_processing.go:71] worker 41 working on infrastructures task. I0423 19:36:42.794048 1 tasks_processing.go:71] worker 22 working on openstack_dataplanedeployments task. I0423 19:36:42.794092 1 tasks_processing.go:71] worker 4 working on mutating_webhook_configurations task. I0423 19:36:42.793740 1 tasks_processing.go:71] worker 9 working on clusterroles task. I0423 19:36:42.794176 1 tasks_processing.go:71] worker 17 working on oauths task. I0423 19:36:42.794229 1 tasks_processing.go:71] worker 27 working on node_logs task. I0423 19:36:42.793462 1 tasks_processing.go:71] worker 55 working on feature_gates task. I0423 19:36:42.794290 1 tasks_processing.go:71] worker 47 working on proxies task. I0423 19:36:42.794313 1 tasks_processing.go:71] worker 32 working on version task. I0423 19:36:42.794325 1 tasks_processing.go:71] worker 49 working on storage_cluster task. I0423 19:36:42.794339 1 tasks_processing.go:71] worker 53 working on support_secret task. I0423 19:36:42.794346 1 tasks_processing.go:71] worker 5 working on ingress_certificates task. I0423 19:36:42.793466 1 tasks_processing.go:71] worker 62 working on tsdb_status task. I0423 19:36:42.794374 1 tasks_processing.go:71] worker 20 working on machines task. I0423 19:36:42.794096 1 tasks_processing.go:71] worker 42 working on container_runtime_configs task. I0423 19:36:42.793725 1 tasks_processing.go:71] worker 0 working on metrics task. I0423 19:36:42.794104 1 tasks_processing.go:71] worker 63 working on openshift_logging task. W0423 19:36:42.795032 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:36:42.793421 1 tasks_processing.go:71] worker 56 working on overlapping_namespace_uids task. I0423 19:36:42.794107 1 tasks_processing.go:71] worker 43 working on cluster_apiserver task. I0423 19:36:42.794136 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 117.222µs to process 0 records I0423 19:36:42.793415 1 tasks_processing.go:71] worker 12 working on cost_management_metrics_configs task. I0423 19:36:42.794152 1 tasks_processing.go:71] worker 15 working on networks task. I0423 19:36:42.794100 1 tasks_processing.go:71] worker 60 working on crds task. W0423 19:36:42.794403 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:36:42.795184 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 27.309µs to process 0 records I0423 19:36:42.795361 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 159.719µs to process 0 records I0423 19:36:42.795192 1 tasks_processing.go:74] worker 48 stopped. I0423 19:36:42.794145 1 tasks_processing.go:71] worker 34 working on sap_config task. I0423 19:36:42.795388 1 tasks_processing.go:74] worker 62 stopped. I0423 19:36:42.795394 1 tasks_processing.go:74] worker 0 stopped. I0423 19:36:42.795372 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 966.918µs to process 0 records I0423 19:36:42.797786 1 tasks_processing.go:74] worker 38 stopped. I0423 19:36:42.797801 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 4.539207ms to process 0 records I0423 19:36:42.799194 1 tasks_processing.go:74] worker 14 stopped. I0423 19:36:42.799210 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 5.960169ms to process 0 records I0423 19:36:42.799229 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 5.659442ms to process 0 records I0423 19:36:42.799235 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 5.705489ms to process 0 records I0423 19:36:42.799237 1 tasks_processing.go:74] worker 26 stopped. I0423 19:36:42.799243 1 tasks_processing.go:74] worker 54 stopped. I0423 19:36:42.799246 1 tasks_processing.go:74] worker 10 stopped. I0423 19:36:42.799259 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 5.779207ms to process 0 records I0423 19:36:42.799272 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 5.826473ms to process 0 records I0423 19:36:42.799278 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 5.839706ms to process 0 records I0423 19:36:42.799284 1 tasks_processing.go:74] worker 57 stopped. I0423 19:36:42.799287 1 tasks_processing.go:74] worker 50 stopped. I0423 19:36:42.799388 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0423 19:36:42.799403 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0423 19:36:42.799408 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 19:36:42.799411 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0423 19:36:42.799424 1 controller.go:489] The operator is still being initialized I0423 19:36:42.799432 1 controller.go:512] The operator is healthy I0423 19:36:42.799505 1 tasks_processing.go:74] worker 13 stopped. I0423 19:36:42.799698 1 recorder.go:75] Recording config/authentication with fingerprint=2ccce5678b531772dfa4ca0b59227b22cf28f57342c364b43977e6e8633c2eff I0423 19:36:42.799710 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 6.079343ms to process 1 records I0423 19:36:42.805882 1 tasks_processing.go:74] worker 8 stopped. I0423 19:36:42.805898 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 12.104972ms to process 0 records I0423 19:36:42.805907 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 11.923887ms to process 0 records I0423 19:36:42.805914 1 tasks_processing.go:74] worker 25 stopped. I0423 19:36:42.805934 1 tasks_processing.go:74] worker 45 stopped. E0423 19:36:42.805949 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0423 19:36:42.805957 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 12.195769ms to process 0 records I0423 19:36:42.806183 1 tasks_processing.go:74] worker 29 stopped. I0423 19:36:42.806195 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 12.72998ms to process 0 records I0423 19:36:42.806221 1 tasks_processing.go:74] worker 31 stopped. I0423 19:36:42.806329 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=3b922669c61d3bcd5d58662bfdcd30a0e14bb142765a026c0bd0a8cf9d0a6645 I0423 19:36:42.806349 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=c61921287a77d386acd3b38ec35a20d5203394430a311354deb79bf610fca874 I0423 19:36:42.806366 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=39d9444a33a7594001dc7cf1b376602834783d92ea9b2ad6574df341f0a8e80b I0423 19:36:42.806374 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 12.364985ms to process 3 records I0423 19:36:42.806455 1 tasks_processing.go:74] worker 6 stopped. I0423 19:36:42.806562 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=bc44e68fc56080f923566c671663821aca8485fa9dfd73e48c06d95056a40d34 I0423 19:36:42.806571 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 12.708968ms to process 1 records I0423 19:36:42.815907 1 tasks_processing.go:74] worker 36 stopped. E0423 19:36:42.815917 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0423 19:36:42.815930 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 22.22223ms to process 0 records I0423 19:36:42.815945 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 22.15276ms to process 0 records I0423 19:36:42.815950 1 tasks_processing.go:74] worker 28 stopped. I0423 19:36:42.816855 1 tasks_processing.go:74] worker 3 stopped. I0423 19:36:42.816992 1 recorder.go:75] Recording config/ingress with fingerprint=e8f109d93fc0668503861c03ed86221d7c0daeacb0869496256c1d2630fa4e20 I0423 19:36:42.817007 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 23.056351ms to process 1 records I0423 19:36:42.817710 1 tasks_processing.go:74] worker 18 stopped. I0423 19:36:42.817726 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 23.841379ms to process 0 records I0423 19:36:42.817831 1 tasks_processing.go:74] worker 30 stopped. I0423 19:36:42.817843 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 24.355821ms to process 0 records I0423 19:36:42.817925 1 recorder.go:75] Recording config/image with fingerprint=8d905cea08a981874f963f88f72d9e92a5b9351788178dda0352a82559b737dd I0423 19:36:42.817934 1 gather.go:177] gatherer "clusterconfig" function "image" took 24.10823ms to process 1 records I0423 19:36:42.817940 1 tasks_processing.go:74] worker 16 stopped. I0423 19:36:42.817971 1 tasks_processing.go:74] worker 17 stopped. I0423 19:36:42.818176 1 recorder.go:75] Recording config/oauth with fingerprint=6423b1132e1d4455d8bd5429cfe8e49d754f9f064eed7faff1251990876fe89a I0423 19:36:42.818191 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 23.778401ms to process 1 records I0423 19:36:42.818310 1 tasks_processing.go:74] worker 41 stopped. I0423 19:36:42.819731 1 recorder.go:75] Recording config/infrastructure with fingerprint=716e57e73ecacd2755d1fcb8eada867c6411b86cfa184a5c091ae0ca71d3ef67 I0423 19:36:42.819769 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 23.967173ms to process 1 records I0423 19:36:42.819782 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 24.094106ms to process 0 records I0423 19:36:42.819800 1 tasks_processing.go:74] worker 22 stopped. I0423 19:36:42.819826 1 tasks_processing.go:74] worker 58 stopped. I0423 19:36:42.819808 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 24.973777ms to process 0 records I0423 19:36:42.819950 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 25.319915ms to process 0 records I0423 19:36:42.819972 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 25.453386ms to process 0 records I0423 19:36:42.819971 1 tasks_processing.go:74] worker 42 stopped. I0423 19:36:42.819983 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 24.851393ms to process 0 records I0423 19:36:42.819987 1 tasks_processing.go:74] worker 49 stopped. I0423 19:36:42.819995 1 tasks_processing.go:74] worker 63 stopped. I0423 19:36:42.820751 1 tasks_processing.go:74] worker 33 stopped. I0423 19:36:42.820927 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=80fb6d7797dafc42da505fcb0b73ceee481724d3d5f8d811c133eb8a6de2dc5b I0423 19:36:42.821014 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=cec5e8056e4f1e15dad2ee7778515456a4ddfe46414f1e38a7adfa87bd30e3d6 I0423 19:36:42.821034 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=87065d13e1d80e3c7b1bf0233f1ec94e16024a15e33bbadcee9cdfe1b2612cd0 I0423 19:36:42.821065 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=94c2ff49648fc9ae29dab86678de1441d821e1e0ca8032d0ee18f12932b8eb84 I0423 19:36:42.821090 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=4ee551001f670bf8739caeb0f5d782af80cb027b9c81a0aaf59c0e1ded5ad2b0 I0423 19:36:42.821113 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=4305f1630c052d34b0a65070c3cc180176e977da8f8dfffc87749191d1df805d I0423 19:36:42.821144 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=eb2019eaaa410889d42805cc4bbd87f4e6ce7420034f4ccadbc5211a25b9e46a I0423 19:36:42.821180 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=2d4963cde78d14fcd443746118e260a5d8995761b6e758483464161202470041 I0423 19:36:42.821203 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=ad399c0a5ef07ac09c08b287b256ca9925383d89de0f830a5970ea85becb68b8 I0423 19:36:42.821232 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=81594995ea2f2412f6a468fea8bda0eb3c13d21ed2852cabce17b599f6310401 I0423 19:36:42.821255 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=b1e4613cc496e8b6f7435c39d4cc40ecab9e2e2ab28e207bb77ac2f8688a9fe6 I0423 19:36:42.821263 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 26.856324ms to process 11 records E0423 19:36:42.821275 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0423 19:36:42.821283 1 gather.go:177] gatherer "clusterconfig" function "machines" took 26.371936ms to process 0 records I0423 19:36:42.821333 1 recorder.go:75] Recording config/proxy with fingerprint=c976b7dd0d09f8279aeb27d004ce96e89e95992879c4b3a72e7e87bbd919e107 I0423 19:36:42.821342 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 26.48323ms to process 1 records I0423 19:36:42.821347 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 26.786868ms to process 0 records I0423 19:36:42.821354 1 tasks_processing.go:74] worker 20 stopped. I0423 19:36:42.821359 1 tasks_processing.go:74] worker 47 stopped. I0423 19:36:42.821372 1 tasks_processing.go:74] worker 40 stopped. I0423 19:36:42.821433 1 tasks_processing.go:74] worker 55 stopped. I0423 19:36:42.821461 1 recorder.go:75] Recording config/featuregate with fingerprint=9113c665e654029ec84c467fb57ba6c7690397cf3097a1bace53b02cda8aab7d I0423 19:36:42.821470 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 26.691141ms to process 1 records I0423 19:36:42.821563 1 tasks_processing.go:74] worker 43 stopped. I0423 19:36:42.821593 1 recorder.go:75] Recording config/apiserver with fingerprint=de85e26aec2c4690b56d37909cec2fc79d23721cdb16700bbd4235267469d3dd I0423 19:36:42.821602 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 26.028986ms to process 1 records I0423 19:36:42.821758 1 tasks_processing.go:74] worker 52 stopped. I0423 19:36:42.821773 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 28.006289ms to process 0 records I0423 19:36:42.827372 1 tasks_processing.go:74] worker 34 stopped. I0423 19:36:42.827386 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 31.982871ms to process 0 records I0423 19:36:42.827448 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=05aa32501ce976f70f0af5bd836e847e8849c909ec1da8175a1a4636eae36ec6 I0423 19:36:42.827457 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 33.903481ms to process 1 records I0423 19:36:42.827470 1 tasks_processing.go:74] worker 61 stopped. I0423 19:36:42.827531 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=0a2813590aa6ba15e65d7a307212d54a8d94c754d5a77503aa8d3bebadd0ba5d I0423 19:36:42.827546 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=2fb8aea293abf6dd032152991058004d65f6ce9a08d4bcac649d181c0ddbded6 I0423 19:36:42.827551 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 33.371533ms to process 2 records I0423 19:36:42.827555 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 33.263351ms to process 0 records I0423 19:36:42.827560 1 tasks_processing.go:74] worker 27 stopped. I0423 19:36:42.827563 1 tasks_processing.go:74] worker 1 stopped. I0423 19:36:42.827957 1 tasks_processing.go:74] worker 53 stopped. E0423 19:36:42.827969 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0423 19:36:42.827980 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 33.610484ms to process 0 records I0423 19:36:42.828058 1 tasks_processing.go:74] worker 39 stopped. I0423 19:36:42.828389 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=cd1271615ec1f083fea329759e65530fda7cf39d5992285bca1d72097a2c667b I0423 19:36:42.828403 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 34.306806ms to process 1 records I0423 19:36:42.828410 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 32.938825ms to process 0 records I0423 19:36:42.828415 1 tasks_processing.go:74] worker 12 stopped. I0423 19:36:42.838582 1 tasks_processing.go:74] worker 23 stopped. I0423 19:36:42.838596 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 45.072238ms to process 0 records I0423 19:36:42.838733 1 tasks_processing.go:74] worker 37 stopped. I0423 19:36:42.838740 1 gather_logs.go:145] no pods in namespace were found I0423 19:36:42.838896 1 recorder.go:75] Recording config/node/ip-10-0-0-88.ec2.internal with fingerprint=d06c9fcdc3bb812657d8a886c8c99a639674745d43a594e61e432821c0eefdc2 I0423 19:36:42.838996 1 recorder.go:75] Recording config/node/ip-10-0-1-49.ec2.internal with fingerprint=6db47a8f2e00bd7a19f894735dc872463b7503c4ee66627f4a0812ee3913980e I0423 19:36:42.839090 1 recorder.go:75] Recording config/node/ip-10-0-2-29.ec2.internal with fingerprint=dad87c8cee4a20ba1d73df465d735dd208200461b23bbaadf0270926beb5ba71 I0423 19:36:42.839102 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 44.675407ms to process 3 records I0423 19:36:42.839116 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 44.702372ms to process 0 records I0423 19:36:42.839125 1 tasks_processing.go:74] worker 46 stopped. I0423 19:36:42.840054 1 tasks_processing.go:74] worker 56 stopped. I0423 19:36:42.840133 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0423 19:36:42.840169 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 44.996136ms to process 1 records I0423 19:36:42.841314 1 tasks_processing.go:74] worker 4 stopped. I0423 19:36:42.841521 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=83606eabdd50356ee6e4d51a0891bdcc639a415dc4a42b39ddbfaab3bb86971f I0423 19:36:42.841584 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=820b052b356fae922cc57a2d4632fcc63315571b9c539353acac533a3b234873 I0423 19:36:42.841609 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=555cadbde549f719ad83c04e46286d5ede9f017b758abd001863eb1b87ce0be2 I0423 19:36:42.841617 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 47.180401ms to process 3 records I0423 19:36:42.841700 1 tasks_processing.go:74] worker 15 stopped. I0423 19:36:42.841747 1 recorder.go:75] Recording config/network with fingerprint=444a6468121a028526d0f8742ae09deb392e093c5952beff7f3f18c2efdfeac1 I0423 19:36:42.841756 1 gather.go:177] gatherer "clusterconfig" function "networks" took 46.387443ms to process 1 records W0423 19:36:42.844080 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:36:42.847034 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0423 19:36:42.847038 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0423 19:36:42.847167 1 operator.go:288] started I0423 19:36:42.847207 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0423 19:36:42.848861 1 tasks_processing.go:74] worker 9 stopped. I0423 19:36:42.849020 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=487c21830e33ad3f57ef05c89ece918396ecd65cee40d31adedb467e0fdf95ed I0423 19:36:42.849106 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=b428d9691f8ca7da7d4f389f9175f27e9a665ea6bd0fe36c113c81aed75081e7 I0423 19:36:42.849115 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 54.682749ms to process 2 records I0423 19:36:42.853015 1 tasks_processing.go:74] worker 60 stopped. I0423 19:36:42.853556 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=76ddc27dfd3e0358498909b6944244e6faee948ec183b13bfc701a7a8d6e1efa I0423 19:36:42.853761 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=182f2f967977a72f1676ba5fc02f97d8d5cf709b6dfef5575361957cc6ed5716 I0423 19:36:42.853771 1 gather.go:177] gatherer "clusterconfig" function "crds" took 57.732651ms to process 2 records I0423 19:36:42.855677 1 tasks_processing.go:74] worker 35 stopped. I0423 19:36:42.855944 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 19:36:42.855954 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0423 19:36:42.855958 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0423 19:36:42.855962 1 controller.go:212] Source scaController *sca.Controller is not ready I0423 19:36:42.855965 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0423 19:36:42.855980 1 controller.go:489] The operator is still being initialized I0423 19:36:42.855987 1 controller.go:512] The operator is healthy I0423 19:36:42.856928 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-4f9qs with fingerprint=6a2b8e8f9d3a9c7bb6a3f61f480b7011afda8595402479f7e0ea12c2f3585591 I0423 19:36:42.856979 1 recorder.go:75] Recording config/running_containers with fingerprint=f4d8771454022d6ae970fae830393fdfab680992a0ef409d0b8f9c284adacc29 I0423 19:36:42.856990 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 61.962078ms to process 2 records I0423 19:36:42.857595 1 prometheus_rules.go:88] Prometheus rules successfully created I0423 19:36:42.857629 1 tasks_processing.go:74] worker 59 stopped. I0423 19:36:42.857648 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 64.078934ms to process 0 records E0423 19:36:42.862340 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27adacedca-d189-4895-9b18-195b9dc2a96b%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:54272->172.30.0.10:53: read: connection refused I0423 19:36:42.862353 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27adacedca-d189-4895-9b18-195b9dc2a96b%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:54272->172.30.0.10:53: read: connection refused I0423 19:36:42.865072 1 tasks_processing.go:74] worker 24 stopped. E0423 19:36:42.865084 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0423 19:36:42.865090 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0423 19:36:42.865093 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0423 19:36:42.865102 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0423 19:36:42.865125 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0423 19:36:42.865132 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0423 19:36:42.865137 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0423 19:36:42.865141 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0423 19:36:42.865178 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0423 19:36:42.865185 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0423 19:36:42.865191 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 71.503844ms to process 7 records I0423 19:36:42.867696 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0423 19:36:42.867705 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 19:36:42.867754 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 19:36:42.871636 1 base_controller.go:82] Caches are synced for ConfigController I0423 19:36:42.871646 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0423 19:36:42.875419 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:36:42.913241 1 tasks_processing.go:74] worker 32 stopped. I0423 19:36:42.913492 1 recorder.go:75] Recording config/version with fingerprint=58ea515f507c2f1c799dbb494ba58ec6f52ecd05816e61dc7cf15844689078d2 I0423 19:36:42.913504 1 recorder.go:75] Recording config/id with fingerprint=3b73d763ffeac02a6864cc3e55aac17a8c20db86beba5e7b040f77ee12f44750 I0423 19:36:42.913511 1 gather.go:177] gatherer "clusterconfig" function "version" took 118.911292ms to process 2 records I0423 19:36:42.926925 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0423 19:36:42.929936 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:37211->172.30.0.10:53: read: connection refused I0423 19:36:42.929947 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:37211->172.30.0.10:53: read: connection refused I0423 19:36:42.947491 1 base_controller.go:82] Caches are synced for LoggingSyncer I0423 19:36:42.947506 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0423 19:36:42.955399 1 tasks_processing.go:74] worker 5 stopped. E0423 19:36:42.955416 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0423 19:36:42.955427 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ps23knnlvuqufr99fsfuv62s7ftvc3g-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ps23knnlvuqufr99fsfuv62s7ftvc3g-primary-cert-bundle-secret" not found I0423 19:36:42.955503 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=a17b62c7df41a912b4ae9504cff9c9df1421c7a7454bd73f71c9a646b0b958b4 I0423 19:36:42.955519 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 161.03726ms to process 1 records W0423 19:36:43.842262 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:36:43.853170 1 tasks_processing.go:74] worker 51 stopped. I0423 19:36:43.853188 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 1.059615528s to process 0 records I0423 19:36:43.861394 1 tasks_processing.go:74] worker 11 stopped. I0423 19:36:43.861426 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0423 19:36:43.861439 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 1.067962561s to process 1 records I0423 19:36:44.030001 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0423 19:36:44.030012 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0423 19:36:44.030695 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-8c5dh pod in namespace openshift-dns (previous: false). I0423 19:36:44.256686 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-8c5dh pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-8c5dh\" is waiting to start: ContainerCreating" I0423 19:36:44.256705 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-8c5dh\" is waiting to start: ContainerCreating" I0423 19:36:44.256715 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-8c5dh pod in namespace openshift-dns (previous: false). I0423 19:36:44.260655 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0423 19:36:44.438995 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-8c5dh pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-8c5dh\" is waiting to start: ContainerCreating" I0423 19:36:44.439012 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-8c5dh\" is waiting to start: ContainerCreating" I0423 19:36:44.439023 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-cmzqm pod in namespace openshift-dns (previous: false). I0423 19:36:44.661130 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-cmzqm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-cmzqm\" is waiting to start: ContainerCreating" I0423 19:36:44.661147 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-cmzqm\" is waiting to start: ContainerCreating" I0423 19:36:44.661156 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-cmzqm pod in namespace openshift-dns (previous: false). I0423 19:36:44.838220 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-cmzqm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-cmzqm\" is waiting to start: ContainerCreating" I0423 19:36:44.838238 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-cmzqm\" is waiting to start: ContainerCreating" I0423 19:36:44.838254 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-s77tc pod in namespace openshift-dns (previous: false). W0423 19:36:44.842843 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:36:44.871718 1 tasks_processing.go:74] worker 19 stopped. I0423 19:36:44.871767 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=dc9cf69140286266d4ebacc4db7e734ce14735440ddbe985ef709defbbfaca1e I0423 19:36:44.871799 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=4b085e5f58a494d49b2fa9d6a814f95f917a779127fd72c78fb51174f0dfa14b I0423 19:36:44.871830 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0423 19:36:44.871857 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=bcfaf06157fca6471a6620d39b66b5ec5c94556bbedfbbe2fe17ed7798d10405 I0423 19:36:44.871875 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0423 19:36:44.871914 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=8fc2b123bfb16dba609b16f2ab91c1d063bcfe4eda52415e94c298e776d44ec9 I0423 19:36:44.871944 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=3115219b812eec7b8854726e280ee262bea2e1f4363ac8b52a5c47af7ed063f2 I0423 19:36:44.871968 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=0e925de26c3742e7b26f2eb64eb278d5b22f2adf01e3c50d480da00c6d3e088a I0423 19:36:44.871981 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=737c5dd1fdd594bd3644e41579cb59df853e3186b4f7855427760556d626bbb0 I0423 19:36:44.871998 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=cf2838993419906409644ee6115be0115f53192af27d2a14016971e0c2b1bdb4 I0423 19:36:44.872008 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0423 19:36:44.872024 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=dae51b10425d6548d9f68ae6d2b8ccaf2696a0443ee74c664b5dfa9d1cd55181 I0423 19:36:44.872032 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0423 19:36:44.872049 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=42cd42769be07cea849b26954130200412f07548ebb2160e83cf24aa02f014d3 I0423 19:36:44.872058 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0423 19:36:44.872077 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=983333186dc3c10064bc1d896510c6366e5096245324ad6b38730d46868ce624 I0423 19:36:44.872086 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0423 19:36:44.872101 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=9a4d5054db6720dc6bb6a6b9626732302bcc01037f75c83b55fbefd048ed379c I0423 19:36:44.872232 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=f48f3396dc20c8a91ea9486ba0d38dded5e6c47b663d255eab0f886a93c37674 I0423 19:36:44.872243 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0423 19:36:44.872250 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0423 19:36:44.872271 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0423 19:36:44.872292 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=87572a7360cf7191dab0961983889d542487107b180aae612ddedc939a377983 I0423 19:36:44.872315 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=21d56e1a78d0b1627514099f3abedbf82f88b91cd4e0fdd498fdde96e7b10d91 I0423 19:36:44.872327 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0423 19:36:44.872350 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=06ff63f4e0832138c5759650d8957d40e3ca99b5834fe48209110e27cd22ca0c I0423 19:36:44.872363 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0423 19:36:44.872383 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=4ffac7732fd2da0e8dadc93233ea0ff5bc1c1e3be2306d1fc1e9e3be7e1114d6 I0423 19:36:44.872405 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=6eb87c0827758ef82fc8c5fb3fbe2a9f15d6ea56c1bafbc2b2f81931bc1083b0 I0423 19:36:44.872426 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=06f628f209fcaa85e4d7fe350fc4024f8aba40d50737bdba19f79c70128e63b0 I0423 19:36:44.872441 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=545758d16cb2e21142f6ede5cbd39428f8fa248ee40d36df88498c2098df05bf I0423 19:36:44.872462 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=8885cb2d2b93f2568944be8dea33d290e42caabcbf68393aa30200fe73eb7923 I0423 19:36:44.872471 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0423 19:36:44.872522 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=1147b8682d1f349a26807f4d1d12bcd6f3250e962647942c8afc5d0b2b922865 I0423 19:36:44.872541 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0423 19:36:44.872550 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0423 19:36:44.872559 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.077697373s to process 36 records I0423 19:36:45.080069 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-s77tc pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-s77tc\" is waiting to start: ContainerCreating" I0423 19:36:45.080084 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-s77tc\" is waiting to start: ContainerCreating" I0423 19:36:45.080092 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-s77tc pod in namespace openshift-dns (previous: false). I0423 19:36:45.235790 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-s77tc pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-s77tc\" is waiting to start: ContainerCreating" I0423 19:36:45.235805 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-s77tc\" is waiting to start: ContainerCreating" I0423 19:36:45.235817 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-nnm62 pod in namespace openshift-dns (previous: false). I0423 19:36:45.435468 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:36:45.435498 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-xtkhq pod in namespace openshift-dns (previous: false). I0423 19:36:45.632285 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:36:45.632301 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-z69zs pod in namespace openshift-dns (previous: false). I0423 19:36:45.834363 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:36:45.834415 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6bc7985b49-t9ffv pod in namespace openshift-image-registry (previous: false). W0423 19:36:45.842217 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:36:46.042292 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6bc7985b49-t9ffv pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6bc7985b49-t9ffv\" is waiting to start: ContainerCreating" I0423 19:36:46.042305 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6bc7985b49-t9ffv\" is waiting to start: ContainerCreating" I0423 19:36:46.042340 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-76f7f74c86-8gnbw pod in namespace openshift-image-registry (previous: false). I0423 19:36:46.238781 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-76f7f74c86-8gnbw pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-76f7f74c86-8gnbw\" is waiting to start: ContainerCreating" I0423 19:36:46.238796 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-76f7f74c86-8gnbw\" is waiting to start: ContainerCreating" I0423 19:36:46.238844 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-76f7f74c86-k485x pod in namespace openshift-image-registry (previous: false). I0423 19:36:46.432779 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-76f7f74c86-k485x pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-76f7f74c86-k485x\" is waiting to start: ContainerCreating" I0423 19:36:46.432792 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-76f7f74c86-k485x\" is waiting to start: ContainerCreating" I0423 19:36:46.432802 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-2k7kz pod in namespace openshift-image-registry (previous: false). I0423 19:36:46.639066 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:36:46.639081 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-bbqpx pod in namespace openshift-image-registry (previous: false). I0423 19:36:46.831997 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:36:46.832010 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-mt4cr pod in namespace openshift-image-registry (previous: false). W0423 19:36:46.842641 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:36:47.031862 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:36:47.031879 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5bcbc96c66-4qhr2 pod in namespace openshift-ingress (previous: false). I0423 19:36:47.231756 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5bcbc96c66-4qhr2 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5bcbc96c66-4qhr2\" is waiting to start: ContainerCreating" I0423 19:36:47.231773 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5bcbc96c66-4qhr2\" is waiting to start: ContainerCreating" I0423 19:36:47.231783 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5bcbc96c66-nwtvf pod in namespace openshift-ingress (previous: false). I0423 19:36:47.431870 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5bcbc96c66-nwtvf pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5bcbc96c66-nwtvf\" is waiting to start: ContainerCreating" I0423 19:36:47.431884 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5bcbc96c66-nwtvf\" is waiting to start: ContainerCreating" I0423 19:36:47.431893 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-8d864fbbf-5c4f6 pod in namespace openshift-ingress (previous: false). I0423 19:36:47.632219 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-8d864fbbf-5c4f6 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-8d864fbbf-5c4f6\" is waiting to start: ContainerCreating" I0423 19:36:47.632234 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-8d864fbbf-5c4f6\" is waiting to start: ContainerCreating" I0423 19:36:47.632272 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dpxkh pod in namespace openshift-ingress-canary (previous: false). I0423 19:36:47.832428 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dpxkh pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dpxkh\" is waiting to start: ContainerCreating" I0423 19:36:47.832441 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dpxkh\" is waiting to start: ContainerCreating" I0423 19:36:47.832467 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-fwlmw pod in namespace openshift-ingress-canary (previous: false). W0423 19:36:47.841846 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0423 19:36:47.841865 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0423 19:36:47.841878 1 tasks_processing.go:74] worker 2 stopped. E0423 19:36:47.841888 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0423 19:36:47.841898 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0423 19:36:47.841911 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0423 19:36:47.841922 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.048123054s to process 1 records I0423 19:36:48.032447 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-fwlmw pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fwlmw\" is waiting to start: ContainerCreating" I0423 19:36:48.032460 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fwlmw\" is waiting to start: ContainerCreating" I0423 19:36:48.032497 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-zhfrl pod in namespace openshift-ingress-canary (previous: false). I0423 19:36:48.231818 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-zhfrl pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-zhfrl\" is waiting to start: ContainerCreating" I0423 19:36:48.231831 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-zhfrl\" is waiting to start: ContainerCreating" I0423 19:36:48.231841 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-ph4g7 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0423 19:36:48.432142 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-ph4g7 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0423 19:36:48.631407 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-dgpxp pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0423 19:36:48.834110 1 tasks_processing.go:74] worker 44 stopped. I0423 19:36:48.834208 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=71775485a1a660af6ca24bde8dfffe9ceffbe2026f15cc833cb77ce7a8ff82fd I0423 19:36:48.834275 1 recorder.go:75] Recording events/openshift-dns with fingerprint=d56b3d341418be726508dfbd3b17d33d084a581342d3c3e2dca28fae6f856eb6 I0423 19:36:48.834385 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=999df1a8536d224c076f4162b4b249592c024638aa889f9d337eb4f13a6a6005 I0423 19:36:48.834424 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=7315820198cdbfc45aab3e3b71b515b29b9d10c528ed511d2f81753ec1b444dc I0423 19:36:48.834500 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=12e727cbb3f5d81b568b7fbdb188b531d7c74a6f451b029d53b073f4a019902e I0423 19:36:48.834530 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=1c4274c49c95bf4872c146fa111cfe1f7dcf6f935d5b1887c74c681a55d284fc I0423 19:36:48.834557 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=26e278ecee646b685a4225612a2c8a9f109e99744932d12a540bc9ec3ddaecb5 I0423 19:36:48.834620 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=e0f66c7f0528e86a5f0eaf43ad25e59485d20f8eadc61f6fe5dbff22a0c5905f I0423 19:36:48.834762 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-8c5dh with fingerprint=5cde7f3cde9780018612192e94f23c47e0acfa5a99f7842924e255874f6e265d I0423 19:36:48.834904 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-6bc7985b49-t9ffv with fingerprint=e20902ea80b8505b1fa3a17adb15dabde6530db1e02482b08d249f8f2d02bd9d I0423 19:36:48.835009 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-76f7f74c86-8gnbw with fingerprint=40c8421fefa2527faf5c20b812a4af66891a00b0b62eaa3cf6eac0c363c197a8 I0423 19:36:48.835108 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-76f7f74c86-k485x with fingerprint=6a69b68757c5dc3b09ed37079b30b540b984150dc44e9a143b146e1ab251b0d0 I0423 19:36:48.835175 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-dpxkh with fingerprint=eb0ec8efe156d92f84a1f178b43c619be62fef3b8d50dd06a463190273494420 I0423 19:36:48.835239 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-fwlmw with fingerprint=d386ebb28800e2c9c811ac9f9374b0224c2df572673c4d462fe092debb4a626c I0423 19:36:48.835305 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-zhfrl with fingerprint=3a16a6f7dc56e63925c268cce565c3c7b0f6cc5b9356e26692d6b44bd135cda1 I0423 19:36:48.835330 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-ph4g7/migrator_current.log with fingerprint=22921e1f3d1706adf14a5245d256ae85d4ce01697dbbfd4574a09030d6e68c92 I0423 19:36:48.835340 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-ph4g7/graceful-termination_current.log with fingerprint=5f2eb378b670787b828b72db3f1461e2a96b751d46288141d18191f2b988b915 I0423 19:36:48.835416 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-dgpxp/kube-storage-version-migrator-operator_current.log with fingerprint=213ba337586fa8b3b83173948a72caef4cf954e94fb8a48e8d3039ea297705d5 I0423 19:36:48.835429 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.040631865s to process 18 records I0423 19:36:53.828218 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:36:54.031368 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:36:54.189058 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:36:55.627550 1 tasks_processing.go:74] worker 21 stopped. I0423 19:36:55.627591 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0423 19:36:55.627603 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.833857069s to process 1 records I0423 19:36:56.399627 1 tasks_processing.go:74] worker 7 stopped. I0423 19:36:56.399888 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=6f7ec23b175ef6978e72e155e10475689a15847bf4d5aede6cec1cd3ba8d2466 I0423 19:36:56.399905 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.606199705s to process 1 records E0423 19:36:56.399982 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.606s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0423 19:36:56.401085 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0423 19:36:56.401100 1 periodic.go:209] Running workloads gatherer I0423 19:36:56.401114 1 tasks_processing.go:45] number of workers: 2 I0423 19:36:56.401121 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 19:36:56.401125 1 tasks_processing.go:71] worker 1 working on workload_info task. I0423 19:36:56.401131 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 19:36:56.401211 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0423 19:36:56.422691 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0423 19:36:56.427664 1 tasks_processing.go:74] worker 0 stopped. I0423 19:36:56.427679 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 26.433063ms to process 0 records I0423 19:36:56.431053 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (9ms) I0423 19:36:56.439374 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0423 19:36:56.447348 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0423 19:36:56.455409 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (8ms) I0423 19:36:56.463058 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (8ms) I0423 19:36:56.470896 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (8ms) I0423 19:36:56.478932 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (8ms) I0423 19:36:56.487129 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (8ms) I0423 19:36:56.497454 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (10ms) I0423 19:36:56.506838 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (9ms) I0423 19:36:56.530530 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (24ms) I0423 19:36:56.632974 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (102ms) I0423 19:36:56.732682 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0423 19:36:56.832177 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (99ms) I0423 19:36:56.932095 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0423 19:36:57.031632 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0423 19:36:57.132360 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (101ms) I0423 19:36:57.231735 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (99ms) I0423 19:36:57.332178 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (100ms) I0423 19:36:57.431534 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (99ms) I0423 19:36:57.531543 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0423 19:36:57.631430 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0423 19:36:57.734013 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (103ms) I0423 19:36:57.831964 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (98ms) I0423 19:36:57.931716 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0423 19:36:58.031144 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0423 19:36:58.131530 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0423 19:36:58.231349 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0423 19:36:58.332220 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (101ms) I0423 19:36:58.431531 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0423 19:36:58.531633 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0423 19:36:58.648419 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (117ms) I0423 19:36:58.734093 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (86ms) I0423 19:36:58.831262 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (97ms) I0423 19:36:58.831293 1 tasks_processing.go:74] worker 1 stopped. E0423 19:36:58.831304 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0423 19:36:58.831684 1 recorder.go:75] Recording config/workload_info with fingerprint=61db895fb605bc04b9f0125f8e6cee95d9ca60f191bb018f47341778f8507a5f I0423 19:36:58.831703 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.430160151s to process 1 records E0423 19:36:58.831731 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.43s with: function \"workload_info\" failed with an error" I0423 19:36:58.832844 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0423 19:36:58.832857 1 periodic.go:209] Running conditional gatherer I0423 19:36:58.838062 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0423 19:36:58.845071 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.14:53271->172.30.0.10:53: read: connection refused E0423 19:36:58.845321 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:36:58.845381 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0423 19:36:58.851898 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0423 19:36:58.851913 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851919 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851922 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851926 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851928 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851932 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851934 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851937 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:36:58.851940 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0423 19:36:58.851956 1 tasks_processing.go:45] number of workers: 3 I0423 19:36:58.851968 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 19:36:58.851972 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0423 19:36:58.851982 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 19:36:58.851995 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 19:36:58.852000 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0423 19:36:58.852003 1 tasks_processing.go:74] worker 1 stopped. I0423 19:36:58.852024 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0423 19:36:58.852053 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0423 19:36:58.852062 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 777ns to process 1 records I0423 19:36:58.852093 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0423 19:36:58.852103 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 2.078µs to process 1 records I0423 19:36:58.852109 1 tasks_processing.go:74] worker 0 stopped. I0423 19:36:58.852313 1 tasks_processing.go:74] worker 2 stopped. I0423 19:36:58.852328 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 253.032µs to process 0 records I0423 19:36:58.852350 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.14:53271->172.30.0.10:53: read: connection refused I0423 19:36:58.852370 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0423 19:36:58.872671 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=4de0ffd046eb2320d9f8eb7dafd073eb8f2b8001c973e2798221416e7b957a36 I0423 19:36:58.872812 1 diskrecorder.go:70] Writing 113 records to /var/lib/insights-operator/insights-2026-04-23-193658.tar.gz I0423 19:36:58.880316 1 diskrecorder.go:51] Wrote 113 records to disk in 7ms I0423 19:36:58.880345 1 periodic.go:278] Gathering cluster info every 2h0m0s I0423 19:36:58.880361 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0423 19:37:02.356221 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:37:46.446792 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="d12b142d085ec533672f4f4082af50d1d85f6573876e3aa25352c785c90d3eea") W0423 19:37:46.446823 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.key was created I0423 19:37:46.446887 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0423 19:37:46.446903 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="865d85b89a46579e69d3359a9bc525ef82131fd025b03d62c13f1bdbd6f1b5dd") I0423 19:37:46.446909 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0423 19:37:46.446948 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0423 19:37:46.446956 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="f3e02335c2ec16116d703483314d3676354e8387ba2b838d14d1641f81fd6ab1") I0423 19:37:46.446948 1 base_controller.go:181] Shutting down ConfigController ... I0423 19:37:46.446975 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0423 19:37:46.446980 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0423 19:37:46.446997 1 requestheader_controller.go:194] Shutting down RequestHeaderAuthRequestController I0423 19:37:46.447011 1 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0423 19:37:46.447025 1 secure_serving.go:258] Stopped listening on [::]:8443 I0423 19:37:46.447038 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" I0423 19:37:46.447047 1 periodic.go:170] Shutting down