W0424 15:07:39.620730 1 cmd.go:257] Using insecure, self-signed certificates I0424 15:07:40.621069 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 15:07:40.621375 1 observer_polling.go:159] Starting file observer I0424 15:07:41.188294 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0424 15:07:41.188483 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0424 15:07:41.189045 1 secure_serving.go:57] Forcing use of http/1.1 only W0424 15:07:41.189069 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. I0424 15:07:41.189065 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0424 15:07:41.189074 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0424 15:07:41.189092 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0424 15:07:41.189095 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0424 15:07:41.189099 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0424 15:07:41.189102 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0424 15:07:41.193014 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0424 15:07:41.193027 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"b27c77ce-c88f-4b6f-b20f-57cb000ff827", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0424 15:07:41.194670 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0424 15:07:41.194678 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0424 15:07:41.194686 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0424 15:07:41.194691 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0424 15:07:41.194700 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0424 15:07:41.194714 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0424 15:07:41.195022 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3207270005/tls.crt::/tmp/serving-cert-3207270005/tls.key" I0424 15:07:41.195692 1 secure_serving.go:213] Serving securely on [::]:8443 I0424 15:07:41.196028 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0424 15:07:41.197921 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0424 15:07:41.197948 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0424 15:07:41.198044 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0424 15:07:41.202556 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:07:41.202572 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0424 15:07:41.206952 1 secretconfigobserver.go:119] support secret does not exist I0424 15:07:41.211317 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:07:41.215413 1 secretconfigobserver.go:119] support secret does not exist I0424 15:07:41.218409 1 recorder.go:161] Pruning old reports every 5h35m5s, max age is 288h0m0s I0424 15:07:41.223089 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0424 15:07:41.223105 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0424 15:07:41.223113 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0424 15:07:41.223122 1 insightsreport.go:296] Starting report retriever I0424 15:07:41.223126 1 periodic.go:209] Running clusterconfig gatherer I0424 15:07:41.223128 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0424 15:07:41.223163 1 tasks_processing.go:45] number of workers: 64 I0424 15:07:41.223186 1 tasks_processing.go:69] worker 4 listening for tasks. I0424 15:07:41.223196 1 tasks_processing.go:69] worker 16 listening for tasks. I0424 15:07:41.223198 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 15:07:41.223203 1 tasks_processing.go:69] worker 5 listening for tasks. I0424 15:07:41.223204 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 15:07:41.223208 1 tasks_processing.go:69] worker 2 listening for tasks. I0424 15:07:41.223210 1 tasks_processing.go:71] worker 5 working on sap_config task. I0424 15:07:41.223209 1 tasks_processing.go:69] worker 30 listening for tasks. I0424 15:07:41.223217 1 tasks_processing.go:69] worker 23 listening for tasks. I0424 15:07:41.223221 1 tasks_processing.go:71] worker 30 working on openstack_dataplanedeployments task. I0424 15:07:41.223226 1 tasks_processing.go:69] worker 6 listening for tasks. I0424 15:07:41.223227 1 tasks_processing.go:69] worker 11 listening for tasks. I0424 15:07:41.223227 1 tasks_processing.go:69] worker 17 listening for tasks. I0424 15:07:41.223231 1 tasks_processing.go:69] worker 18 listening for tasks. I0424 15:07:41.223220 1 tasks_processing.go:69] worker 10 listening for tasks. I0424 15:07:41.223240 1 tasks_processing.go:69] worker 8 listening for tasks. I0424 15:07:41.223233 1 tasks_processing.go:69] worker 7 listening for tasks. I0424 15:07:41.223245 1 tasks_processing.go:69] worker 21 listening for tasks. I0424 15:07:41.223243 1 tasks_processing.go:69] worker 12 listening for tasks. I0424 15:07:41.223252 1 tasks_processing.go:69] worker 13 listening for tasks. I0424 15:07:41.223254 1 tasks_processing.go:69] worker 22 listening for tasks. I0424 15:07:41.223260 1 tasks_processing.go:69] worker 20 listening for tasks. I0424 15:07:41.223261 1 tasks_processing.go:69] worker 14 listening for tasks. I0424 15:07:41.223215 1 tasks_processing.go:71] worker 2 working on operators task. I0424 15:07:41.223285 1 tasks_processing.go:69] worker 31 listening for tasks. I0424 15:07:41.223294 1 tasks_processing.go:69] worker 32 listening for tasks. I0424 15:07:41.223299 1 tasks_processing.go:69] worker 33 listening for tasks. I0424 15:07:41.223301 1 tasks_processing.go:69] worker 48 listening for tasks. I0424 15:07:41.223309 1 tasks_processing.go:69] worker 35 listening for tasks. I0424 15:07:41.223306 1 tasks_processing.go:69] worker 34 listening for tasks. I0424 15:07:41.223320 1 tasks_processing.go:69] worker 44 listening for tasks. I0424 15:07:41.223326 1 tasks_processing.go:69] worker 45 listening for tasks. I0424 15:07:41.223334 1 tasks_processing.go:69] worker 46 listening for tasks. I0424 15:07:41.223373 1 tasks_processing.go:69] worker 29 listening for tasks. I0424 15:07:41.223378 1 tasks_processing.go:69] worker 9 listening for tasks. I0424 15:07:41.223387 1 tasks_processing.go:69] worker 54 listening for tasks. I0424 15:07:41.223388 1 tasks_processing.go:69] worker 55 listening for tasks. I0424 15:07:41.223382 1 tasks_processing.go:69] worker 52 listening for tasks. I0424 15:07:41.223393 1 tasks_processing.go:69] worker 53 listening for tasks. I0424 15:07:41.223386 1 tasks_processing.go:69] worker 28 listening for tasks. I0424 15:07:41.223396 1 tasks_processing.go:71] worker 16 working on mutating_webhook_configurations task. I0424 15:07:41.223398 1 tasks_processing.go:69] worker 38 listening for tasks. I0424 15:07:41.223403 1 tasks_processing.go:69] worker 47 listening for tasks. I0424 15:07:41.223407 1 tasks_processing.go:71] worker 4 working on pod_network_connectivity_checks task. I0424 15:07:41.223401 1 tasks_processing.go:69] worker 63 listening for tasks. I0424 15:07:41.223412 1 tasks_processing.go:69] worker 59 listening for tasks. I0424 15:07:41.223412 1 tasks_processing.go:69] worker 24 listening for tasks. I0424 15:07:41.223399 1 tasks_processing.go:69] worker 49 listening for tasks. I0424 15:07:41.223419 1 tasks_processing.go:69] worker 56 listening for tasks. I0424 15:07:41.223419 1 tasks_processing.go:69] worker 58 listening for tasks. I0424 15:07:41.223420 1 tasks_processing.go:69] worker 51 listening for tasks. I0424 15:07:41.223427 1 tasks_processing.go:69] worker 61 listening for tasks. I0424 15:07:41.223427 1 tasks_processing.go:69] worker 60 listening for tasks. I0424 15:07:41.223406 1 tasks_processing.go:69] worker 25 listening for tasks. I0424 15:07:41.223468 1 tasks_processing.go:71] worker 35 working on openshift_machine_api_events task. I0424 15:07:41.223468 1 tasks_processing.go:71] worker 14 working on networks task. I0424 15:07:41.223477 1 tasks_processing.go:71] worker 8 working on openshift_logging task. I0424 15:07:41.223486 1 tasks_processing.go:71] worker 55 working on jaegers task. I0424 15:07:41.223480 1 tasks_processing.go:71] worker 22 working on ingress task. I0424 15:07:41.223483 1 tasks_processing.go:69] worker 43 listening for tasks. I0424 15:07:41.223460 1 tasks_processing.go:71] worker 10 working on ingress_certificates task. I0424 15:07:41.223506 1 tasks_processing.go:71] worker 43 working on cost_management_metrics_configs task. I0424 15:07:41.223512 1 tasks_processing.go:71] worker 53 working on image_registries task. I0424 15:07:41.223554 1 tasks_processing.go:71] worker 28 working on certificate_signing_requests task. I0424 15:07:41.223430 1 tasks_processing.go:69] worker 41 listening for tasks. I0424 15:07:41.223434 1 tasks_processing.go:69] worker 26 listening for tasks. I0424 15:07:41.223579 1 tasks_processing.go:71] worker 49 working on machine_autoscalers task. I0424 15:07:41.223427 1 tasks_processing.go:69] worker 62 listening for tasks. I0424 15:07:41.223585 1 tasks_processing.go:71] worker 58 working on image_pruners task. I0424 15:07:41.223435 1 tasks_processing.go:69] worker 42 listening for tasks. I0424 15:07:41.223437 1 tasks_processing.go:69] worker 40 listening for tasks. I0424 15:07:41.223437 1 tasks_processing.go:69] worker 37 listening for tasks. I0424 15:07:41.223443 1 tasks_processing.go:71] worker 6 working on machine_healthchecks task. I0424 15:07:41.223643 1 tasks_processing.go:71] worker 56 working on infrastructures task. I0424 15:07:41.223461 1 tasks_processing.go:71] worker 54 working on openstack_version task. I0424 15:07:41.223667 1 tasks_processing.go:71] worker 47 working on olm_operators task. I0424 15:07:41.223769 1 tasks_processing.go:71] worker 37 working on dvo_metrics task. I0424 15:07:41.223775 1 tasks_processing.go:71] worker 61 working on overlapping_namespace_uids task. I0424 15:07:41.223927 1 tasks_processing.go:71] worker 34 working on install_plans task. I0424 15:07:41.223936 1 tasks_processing.go:69] worker 15 listening for tasks. I0424 15:07:41.223951 1 tasks_processing.go:71] worker 15 working on nodes task. I0424 15:07:41.223449 1 tasks_processing.go:71] worker 31 working on cluster_apiserver task. I0424 15:07:41.223211 1 tasks_processing.go:71] worker 1 working on ceph_cluster task. I0424 15:07:41.223399 1 tasks_processing.go:71] worker 0 working on config_maps task. I0424 15:07:41.224485 1 tasks_processing.go:71] worker 44 working on aggregated_monitoring_cr_names task. I0424 15:07:41.224693 1 tasks_processing.go:71] worker 41 working on nodenetworkconfigurationpolicies task. I0424 15:07:41.223442 1 tasks_processing.go:71] worker 23 working on openstack_controlplanes task. I0424 15:07:41.224801 1 tasks_processing.go:71] worker 29 working on machine_config_pools task. I0424 15:07:41.223771 1 tasks_processing.go:71] worker 51 working on version task. I0424 15:07:41.223455 1 tasks_processing.go:71] worker 32 working on image task. I0424 15:07:41.223452 1 tasks_processing.go:71] worker 60 working on proxies task. I0424 15:07:41.225175 1 tasks_processing.go:71] worker 38 working on openstack_dataplanenodesets task. I0424 15:07:41.225217 1 tasks_processing.go:71] worker 42 working on nodenetworkstates task. I0424 15:07:41.225250 1 tasks_processing.go:71] worker 62 working on schedulers task. I0424 15:07:41.223455 1 tasks_processing.go:71] worker 21 working on monitoring_persistent_volumes task. I0424 15:07:41.223214 1 tasks_processing.go:69] worker 3 listening for tasks. I0424 15:07:41.225453 1 tasks_processing.go:71] worker 3 working on container_images task. I0424 15:07:41.223458 1 tasks_processing.go:71] worker 33 working on container_runtime_configs task. I0424 15:07:41.225567 1 tasks_processing.go:71] worker 40 working on clusterroles task. I0424 15:07:41.223506 1 tasks_processing.go:71] worker 52 working on storage_classes task. I0424 15:07:41.225642 1 tasks_processing.go:71] worker 63 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0424 15:07:41.223458 1 tasks_processing.go:71] worker 11 working on authentication task. I0424 15:07:41.223455 1 tasks_processing.go:71] worker 7 working on machine_sets task. I0424 15:07:41.223460 1 tasks_processing.go:71] worker 48 working on node_logs task. I0424 15:07:41.223463 1 tasks_processing.go:71] worker 17 working on storage_cluster task. I0424 15:07:41.223458 1 tasks_processing.go:69] worker 36 listening for tasks. I0424 15:07:41.223471 1 tasks_processing.go:71] worker 25 working on support_secret task. I0424 15:07:41.223459 1 tasks_processing.go:71] worker 12 working on pdbs task. I0424 15:07:41.223464 1 tasks_processing.go:71] worker 13 working on operators_pods_and_events task. I0424 15:07:41.223278 1 tasks_processing.go:69] worker 39 listening for tasks. I0424 15:07:41.223499 1 tasks_processing.go:71] worker 59 working on sap_datahubs task. I0424 15:07:41.223575 1 tasks_processing.go:71] worker 24 working on active_alerts task. I0424 15:07:41.224744 1 tasks_processing.go:71] worker 45 working on machine_configs task. I0424 15:07:41.224752 1 tasks_processing.go:71] worker 46 working on metrics task. I0424 15:07:41.223451 1 tasks_processing.go:71] worker 9 working on qemu_kubevirt_launcher_logs task. I0424 15:07:41.223455 1 tasks_processing.go:71] worker 18 working on validating_webhook_configurations task. I0424 15:07:41.225104 1 tasks_processing.go:69] worker 57 listening for tasks. I0424 15:07:41.225110 1 tasks_processing.go:71] worker 26 working on crds task. W0424 15:07:41.226414 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 15:07:41.226433 1 tasks_processing.go:71] worker 46 working on oauths task. I0424 15:07:41.225125 1 tasks_processing.go:69] worker 19 listening for tasks. I0424 15:07:41.223411 1 tasks_processing.go:69] worker 50 listening for tasks. I0424 15:07:41.226450 1 tasks_processing.go:71] worker 19 working on sap_pods task. I0424 15:07:41.223470 1 tasks_processing.go:71] worker 20 working on feature_gates task. I0424 15:07:41.226261 1 tasks_processing.go:71] worker 36 working on silenced_alerts task. W0424 15:07:41.226815 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 15:07:41.227168 1 tasks_processing.go:71] worker 39 working on service_accounts task. W0424 15:07:41.227187 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 15:07:41.227194 1 tasks_processing.go:69] worker 27 listening for tasks. I0424 15:07:41.227203 1 tasks_processing.go:71] worker 57 working on tsdb_status task. I0424 15:07:41.227221 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 38.825µs to process 0 records I0424 15:07:41.227233 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 362.924µs to process 0 records I0424 15:07:41.227241 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 3.911271ms to process 0 records W0424 15:07:41.227223 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 15:07:41.227250 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 878.213µs to process 0 records I0424 15:07:41.227205 1 tasks_processing.go:71] worker 27 working on machines task. I0424 15:07:41.227255 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 44.364µs to process 0 records I0424 15:07:41.227264 1 tasks_processing.go:74] worker 57 stopped. I0424 15:07:41.227270 1 tasks_processing.go:74] worker 24 stopped. I0424 15:07:41.227272 1 tasks_processing.go:74] worker 5 stopped. I0424 15:07:41.227277 1 tasks_processing.go:74] worker 36 stopped. I0424 15:07:41.227283 1 tasks_processing.go:71] worker 50 working on lokistack task. I0424 15:07:41.227440 1 tasks_processing.go:74] worker 49 stopped. I0424 15:07:41.227464 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 3.851575ms to process 0 records I0424 15:07:41.227477 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 3.92878ms to process 0 records E0424 15:07:41.227485 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0424 15:07:41.227494 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 4.035668ms to process 0 records I0424 15:07:41.227497 1 tasks_processing.go:74] worker 43 stopped. I0424 15:07:41.227502 1 tasks_processing.go:74] worker 4 stopped. I0424 15:07:41.228172 1 tasks_processing.go:74] worker 30 stopped. I0424 15:07:41.228187 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 4.940981ms to process 0 records I0424 15:07:41.244980 1 tasks_processing.go:74] worker 35 stopped. I0424 15:07:41.244995 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 21.500341ms to process 0 records I0424 15:07:41.245152 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0424 15:07:41.245174 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0424 15:07:41.245183 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0424 15:07:41.245188 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0424 15:07:41.245203 1 tasks_processing.go:74] worker 55 stopped. I0424 15:07:41.245205 1 controller.go:489] The operator is still being initialized I0424 15:07:41.245218 1 controller.go:512] The operator is healthy I0424 15:07:41.245217 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 21.704846ms to process 0 records I0424 15:07:41.245228 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 21.714069ms to process 0 records I0424 15:07:41.245236 1 tasks_processing.go:74] worker 8 stopped. I0424 15:07:41.245256 1 tasks_processing.go:74] worker 1 stopped. I0424 15:07:41.245274 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 20.826592ms to process 0 records I0424 15:07:41.245286 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 20.47375ms to process 0 records I0424 15:07:41.245297 1 tasks_processing.go:74] worker 23 stopped. I0424 15:07:41.245440 1 tasks_processing.go:74] worker 6 stopped. E0424 15:07:41.245457 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0424 15:07:41.245468 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 21.826544ms to process 0 records I0424 15:07:41.245483 1 tasks_processing.go:74] worker 56 stopped. I0424 15:07:41.245984 1 recorder.go:75] Recording config/infrastructure with fingerprint=ade3411cc61a871093799dcc181d3525aa3fde373299bb062d96683405e6c721 I0424 15:07:41.245998 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 21.826026ms to process 1 records I0424 15:07:41.246009 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 21.853919ms to process 0 records I0424 15:07:41.246022 1 tasks_processing.go:74] worker 54 stopped. I0424 15:07:41.246093 1 tasks_processing.go:74] worker 22 stopped. I0424 15:07:41.246159 1 recorder.go:75] Recording config/ingress with fingerprint=d4ab4c85ed216be841b5471b76a86d4b8691606263818d9637160b4175d71cd6 I0424 15:07:41.246168 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 22.057405ms to process 1 records I0424 15:07:41.246275 1 tasks_processing.go:74] worker 14 stopped. I0424 15:07:41.246332 1 recorder.go:75] Recording config/network with fingerprint=9ce49973ec4fb0b5821d1b382445bed276ca9ddc7cc927def41f3bdb9ff23e33 I0424 15:07:41.246345 1 gather.go:177] gatherer "clusterconfig" function "networks" took 22.073336ms to process 1 records I0424 15:07:41.246441 1 tasks_processing.go:74] worker 58 stopped. I0424 15:07:41.246625 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=acf6fec6be5633a259655fb2d6c86eec965c3e033c3f54128e698221b6990989 I0424 15:07:41.246634 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 22.072179ms to process 1 records I0424 15:07:41.246642 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 20.933942ms to process 0 records I0424 15:07:41.246646 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 20.898391ms to process 0 records I0424 15:07:41.246703 1 tasks_processing.go:74] worker 42 stopped. I0424 15:07:41.246704 1 tasks_processing.go:74] worker 38 stopped. I0424 15:07:41.246762 1 recorder.go:75] Recording config/image with fingerprint=df5655480484878dd0e260e4e0bc03d3e2390093f35d6b94368e050ac2567a6e I0424 15:07:41.246771 1 gather.go:177] gatherer "clusterconfig" function "image" took 21.55454ms to process 1 records I0424 15:07:41.246779 1 tasks_processing.go:74] worker 32 stopped. I0424 15:07:41.246788 1 tasks_processing.go:74] worker 33 stopped. I0424 15:07:41.246798 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 21.247324ms to process 0 records I0424 15:07:41.246856 1 tasks_processing.go:74] worker 41 stopped. I0424 15:07:41.246870 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 22.148656ms to process 0 records I0424 15:07:41.248187 1 tasks_processing.go:74] worker 60 stopped. I0424 15:07:41.248781 1 recorder.go:75] Recording config/proxy with fingerprint=4061ab89bda7bdc7217da3075e8bf3ef634859cc5d5fdda56ef813cac1d583c6 I0424 15:07:41.248842 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 23.044966ms to process 1 records I0424 15:07:41.248897 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 24.541646ms to process 0 records I0424 15:07:41.248949 1 tasks_processing.go:74] worker 47 stopped. I0424 15:07:41.250551 1 tasks_processing.go:74] worker 11 stopped. I0424 15:07:41.251122 1 recorder.go:75] Recording config/authentication with fingerprint=722e4cfbaa8f49a5eac47ea7c7e7f12a652f30662dc1e16ff953c24d39867d87 I0424 15:07:41.251138 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 24.686754ms to process 1 records I0424 15:07:41.252574 1 tasks_processing.go:74] worker 28 stopped. I0424 15:07:41.252588 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 28.995722ms to process 0 records W0424 15:07:41.253074 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:07:41.253351 1 tasks_processing.go:74] worker 17 stopped. I0424 15:07:41.253366 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 27.283039ms to process 0 records I0424 15:07:41.253417 1 tasks_processing.go:74] worker 7 stopped. I0424 15:07:41.253432 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 27.353965ms to process 0 records I0424 15:07:41.253482 1 tasks_processing.go:74] worker 62 stopped. I0424 15:07:41.253583 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=a54c40e3c6fcd7569bb2757eb5d11519dd20cada9fbf5160a8a489eb0be62a2b I0424 15:07:41.253595 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 28.151054ms to process 1 records I0424 15:07:41.253682 1 tasks_processing.go:74] worker 31 stopped. I0424 15:07:41.253710 1 recorder.go:75] Recording config/apiserver with fingerprint=1e724814a2b344df932f7cf7816ffc38499af56c0fbca9b66f7d73b91785b08b I0424 15:07:41.253722 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 29.180304ms to process 1 records I0424 15:07:41.254130 1 tasks_processing.go:74] worker 61 stopped. I0424 15:07:41.254157 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0424 15:07:41.254165 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 30.336125ms to process 1 records I0424 15:07:41.262762 1 tasks_processing.go:74] worker 50 stopped. I0424 15:07:41.262779 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 35.468445ms to process 0 records I0424 15:07:41.263677 1 tasks_processing.go:74] worker 19 stopped. I0424 15:07:41.263690 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 37.217611ms to process 0 records I0424 15:07:41.263696 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 37.378342ms to process 0 records I0424 15:07:41.263701 1 tasks_processing.go:74] worker 59 stopped. I0424 15:07:41.263701 1 gather_logs.go:145] no pods in namespace were found I0424 15:07:41.263709 1 tasks_processing.go:74] worker 9 stopped. I0424 15:07:41.263715 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 37.307822ms to process 0 records E0424 15:07:41.263722 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0424 15:07:41.263727 1 gather.go:177] gatherer "clusterconfig" function "machines" took 36.454055ms to process 0 records I0424 15:07:41.263733 1 tasks_processing.go:74] worker 27 stopped. I0424 15:07:41.264082 1 tasks_processing.go:74] worker 53 stopped. I0424 15:07:41.264398 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=894f9491d7ebe1686474ba89f4f97d10c998f2cbb8444968a41fb1cd0d5cb5a3 I0424 15:07:41.264413 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 40.56218ms to process 1 records I0424 15:07:41.264676 1 tasks_processing.go:74] worker 52 stopped. I0424 15:07:41.264755 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=928dc3337570ecdf3d1102af3e5ecaa3fa900cf2e68e87d4462cca7f1d4f90ad I0424 15:07:41.264772 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=0703e62040808d414b6c341a5a800a10c7b16695bbc0998c32ec80ad026f64d4 I0424 15:07:41.264778 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 39.0615ms to process 2 records I0424 15:07:41.271309 1 tasks_processing.go:74] worker 25 stopped. E0424 15:07:41.271320 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0424 15:07:41.271328 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 45.146964ms to process 0 records I0424 15:07:41.271333 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 45.932131ms to process 0 records I0424 15:07:41.271342 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 45.255018ms to process 0 records I0424 15:07:41.271347 1 tasks_processing.go:74] worker 48 stopped. I0424 15:07:41.271351 1 tasks_processing.go:74] worker 21 stopped. I0424 15:07:41.271360 1 tasks_processing.go:74] worker 12 stopped. I0424 15:07:41.271443 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=dbb2cd400b9013ca34bb2264c113d0b3f98eb1d5b02b4e441bfdcc5280faafe0 I0424 15:07:41.271467 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=f29898ade18a80f6ed7d9401cf16c864a4a181162d5959a7ba8ff6435063988d I0424 15:07:41.271489 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=eaa8679ebd270210d98094e39537013a1f75a1d67951553352157e7a4eb26b5d I0424 15:07:41.271497 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 45.183534ms to process 3 records I0424 15:07:41.271576 1 tasks_processing.go:74] worker 46 stopped. I0424 15:07:41.271672 1 recorder.go:75] Recording config/oauth with fingerprint=f3e4ef956eabff18e0b0de6553fbd3c68b7acd074c4a2948e408ed90cacbe0c2 I0424 15:07:41.271679 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 45.056092ms to process 1 records I0424 15:07:41.271718 1 tasks_processing.go:74] worker 44 stopped. I0424 15:07:41.271735 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 47.215459ms to process 0 records I0424 15:07:41.272417 1 tasks_processing.go:74] worker 20 stopped. I0424 15:07:41.272531 1 recorder.go:75] Recording config/featuregate with fingerprint=beeccd5bbcc8e519ef52891c3b5a7b53cfc02e043a07b18b6d29a7743385d7a1 I0424 15:07:41.272543 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 45.916618ms to process 1 records I0424 15:07:41.273124 1 tasks_processing.go:74] worker 16 stopped. I0424 15:07:41.273606 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=8982832be414f4b464b0c1f0675d7dc2d4825023c80b1c92dd6e71013c04d7e9 I0424 15:07:41.273709 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=03fd45d109ea104eda03236557bfa9b699f60a39e703f5eeb9264eba0cbb7ee6 I0424 15:07:41.273794 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=d10c0bca3732a4afddd9a289505727a7066664491ede3949f592fd7b6036a0eb I0424 15:07:41.273825 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 49.714375ms to process 3 records W0424 15:07:41.274198 1 operator.go:288] started I0424 15:07:41.274260 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0424 15:07:41.274363 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0424 15:07:41.274486 1 tasks_processing.go:74] worker 15 stopped. I0424 15:07:41.274378 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0424 15:07:41.274519 1 recorder.go:75] Recording config/node/ip-10-0-0-164.ec2.internal with fingerprint=854850ca04fbb8736f571c4aa2bc1c081bf98769f603af77aad990fa5417cd26 I0424 15:07:41.275392 1 recorder.go:75] Recording config/node/ip-10-0-1-206.ec2.internal with fingerprint=2f44e62e9b66b93e3f53b0d463a3e5b5a217242b771bd3d4166e058dd6a93766 I0424 15:07:41.275475 1 recorder.go:75] Recording config/node/ip-10-0-2-176.ec2.internal with fingerprint=3f4db64872f488132acdb358d12bcd407bc6448631807af3c5a7f7a54d5cd520 I0424 15:07:41.275488 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 49.354416ms to process 3 records I0424 15:07:41.275658 1 tasks_processing.go:74] worker 40 stopped. I0424 15:07:41.275679 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=3dbe491a75194af1563c43b7a1356e2f4f83bd5bb05927884404c9e73d24431b I0424 15:07:41.275792 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=237fab5c7d950aa2808f56f033a93da9f06a6aebd6c133da37956fd70eb0047d I0424 15:07:41.275800 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 47.896455ms to process 2 records I0424 15:07:41.275898 1 tasks_processing.go:74] worker 18 stopped. I0424 15:07:41.275953 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=8afa140cd48ca16a871339df9ca0b69b5bda8f875e35b72276c3678da9fd6bf4 I0424 15:07:41.276050 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=0bb853c74def223e887cbc63a9d40ea5a5fa6daefe9900bf0a507b2424951648 I0424 15:07:41.276074 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=cc313c23baef049ebadca7d88ffee2b1797b6973384ed23d74c23a43673d002b I0424 15:07:41.276108 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=cd485ff7f74c5d5aea7602b27d27ff99308a421e7c498bc4049cb29cda65da32 I0424 15:07:41.276135 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=739867d97d8cd0bf30cbb0806878f137474e23e70c3720a38ef61ba474262061 I0424 15:07:41.276159 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=c35d08d67008543a6783e7c299fc2993256f041e1a31872e97c1f23cb0630e57 I0424 15:07:41.276202 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=4225e1f3657bc9f024af1f2b1dae55fec74e700b0305c5ee6572cd7160d8792e I0424 15:07:41.276246 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=63fa83e58424a622d54c2d60f5f8cd4484532395576bc9cefdb1a84db571c3f8 I0424 15:07:41.276279 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=d1d9f0d2afd16007b7aedd93bab2fc57328636e01328ad96670f93d895da6028 I0424 15:07:41.276306 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=ad76ecebaa871c512c8500fd06ed66423da6bb60b71139fd56460a82e34d2943 I0424 15:07:41.276348 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=3232a00297e34faf566a1c2c7ce467503f1c823fe5a0e8c9470f1cf1cdee08d3 I0424 15:07:41.276376 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 47.616139ms to process 11 records I0424 15:07:41.281903 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0424 15:07:41.281913 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0424 15:07:41.281916 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0424 15:07:41.281919 1 controller.go:212] Source scaController *sca.Controller is not ready I0424 15:07:41.281922 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0424 15:07:41.281937 1 controller.go:489] The operator is still being initialized I0424 15:07:41.281942 1 controller.go:512] The operator is healthy I0424 15:07:41.282509 1 tasks_processing.go:74] worker 26 stopped. I0424 15:07:41.283499 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=2860284160f0009bc7ba733ec25a4eb590eb7193c25a8efa5d384b60417955f4 I0424 15:07:41.283809 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=a888a90699a6e0eb166c9cbd76d276ec19c41a5e2051210b123b79aa0e4a45a1 I0424 15:07:41.283858 1 gather.go:177] gatherer "clusterconfig" function "crds" took 56.083946ms to process 2 records I0424 15:07:41.287672 1 tasks_processing.go:74] worker 3 stopped. I0424 15:07:41.289048 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-vv8vk with fingerprint=94539a1939bcc87372d5f91411c25b6d1d3ed1cb8c9f57363f54510ec72902b3 I0424 15:07:41.289115 1 recorder.go:75] Recording config/running_containers with fingerprint=fdfb0a16757e62f88e952970a12f4c8b606a64c934c563674213775f380743b1 I0424 15:07:41.289129 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 62.193733ms to process 2 records I0424 15:07:41.289146 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 63.010811ms to process 0 records I0424 15:07:41.289156 1 tasks_processing.go:74] worker 63 stopped. I0424 15:07:41.289714 1 prometheus_rules.go:88] Prometheus rules successfully created I0424 15:07:41.294698 1 tasks_processing.go:74] worker 0 stopped. E0424 15:07:41.294714 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0424 15:07:41.294722 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found I0424 15:07:41.294725 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController E0424 15:07:41.294728 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0424 15:07:41.294735 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0424 15:07:41.294741 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0424 15:07:41.294766 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0424 15:07:41.294781 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0424 15:07:41.294788 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0424 15:07:41.294794 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0424 15:07:41.294799 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0424 15:07:41.294838 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0424 15:07:41.294847 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0424 15:07:41.294852 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 70.259214ms to process 7 records E0424 15:07:41.297784 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%273018e52d-d166-4dfd-8335-f1ae74752955%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.13:42010->172.30.0.10:53: read: connection refused I0424 15:07:41.297798 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%273018e52d-d166-4dfd-8335-f1ae74752955%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.13:42010->172.30.0.10:53: read: connection refused I0424 15:07:41.298820 1 base_controller.go:82] Caches are synced for ConfigController I0424 15:07:41.298832 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0424 15:07:41.303240 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:07:41.355047 1 tasks_processing.go:74] worker 51 stopped. I0424 15:07:41.355322 1 recorder.go:75] Recording config/version with fingerprint=7260f17baca703938afc60b80085841be9e0707ff3a2314126be350b25dbf8ca I0424 15:07:41.355338 1 recorder.go:75] Recording config/id with fingerprint=e11a5fcff703eeef121ab9f04f31da139f1d3e2b6854104fef75b6f154ab6fa5 I0424 15:07:41.355345 1 gather.go:177] gatherer "clusterconfig" function "version" took 130.086398ms to process 2 records I0424 15:07:41.372400 1 tasks_processing.go:74] worker 45 stopped. I0424 15:07:41.372426 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0424 15:07:41.372435 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 146.066399ms to process 1 records I0424 15:07:41.374514 1 base_controller.go:82] Caches are synced for LoggingSyncer I0424 15:07:41.374526 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0424 15:07:41.381305 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0424 15:07:41.381790 1 tasks_processing.go:74] worker 10 stopped. E0424 15:07:41.381803 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0424 15:07:41.381809 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2psja0jhmukit6oknklbqe0g7hb54t4t-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2psja0jhmukit6oknklbqe0g7hb54t4t-primary-cert-bundle-secret" not found I0424 15:07:41.381903 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=5e064f7d5423714c099b18b2e220736aaae8607781e67b460b9fa81b11aff75f I0424 15:07:41.381921 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 158.274986ms to process 1 records W0424 15:07:41.384761 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.13:57612->172.30.0.10:53: read: connection refused I0424 15:07:41.384773 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.13:57612->172.30.0.10:53: read: connection refused W0424 15:07:42.252297 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:07:42.276930 1 tasks_processing.go:74] worker 29 stopped. I0424 15:07:42.276946 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 1.052105976s to process 0 records I0424 15:07:42.482842 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0424 15:07:42.482855 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0424 15:07:42.483541 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-mz6q6 pod in namespace openshift-dns (previous: false). I0424 15:07:42.685497 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0424 15:07:42.711520 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mz6q6 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-mz6q6\" is waiting to start: ContainerCreating" I0424 15:07:42.711536 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-mz6q6\" is waiting to start: ContainerCreating" I0424 15:07:42.711545 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-mz6q6 pod in namespace openshift-dns (previous: false). I0424 15:07:42.885612 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mz6q6 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-mz6q6\" is waiting to start: ContainerCreating" I0424 15:07:42.885625 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-mz6q6\" is waiting to start: ContainerCreating" I0424 15:07:42.885637 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-sdr5h pod in namespace openshift-dns (previous: false). I0424 15:07:43.109976 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-sdr5h pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-sdr5h\" is waiting to start: ContainerCreating" I0424 15:07:43.109991 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-sdr5h\" is waiting to start: ContainerCreating" I0424 15:07:43.109998 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-sdr5h pod in namespace openshift-dns (previous: false). W0424 15:07:43.251462 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:07:43.285371 1 tasks_processing.go:74] worker 2 stopped. I0424 15:07:43.285428 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=c4eae45f3c6af7756d595734fdd62c11918486b401ba16b71ca8d4dbfae0f20d I0424 15:07:43.285461 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=dd1e2d3bca876b56bb16bb76251fe8bc667c7662cb64fb514263c086f439201d I0424 15:07:43.285490 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0424 15:07:43.285517 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=e514bb2d2316ebdad267b93d81964aca423547c00624a84961be21caefaa4431 I0424 15:07:43.285533 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0424 15:07:43.285570 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=65e1d240135577534464e5f8b68a4aa2d86199741eeade5cb9dabb382755c28d I0424 15:07:43.285600 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=13bdbc152af603f89aecfceaa97f59b25c6257e2feba87bfff6c4fa347f63800 I0424 15:07:43.285623 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=d3a952a73d0b63971fa68ace93bda41068ecf35ac6e2044c1946402cab2dd21b I0424 15:07:43.285638 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=fae2a0f192ff1f1479ec2711bb6db4bd3dd0364f816d5b3608d08c7f0843fbcb I0424 15:07:43.285656 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=69de1d13fe68dd79cd5688ba242135d5ae4defbee0c5fc9f8a383cc19c5faa14 I0424 15:07:43.285666 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0424 15:07:43.285683 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=6e903b8201f73f8659f315fca219a9e74bf829061edbb63802db0a40a7d7e33d I0424 15:07:43.285692 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0424 15:07:43.285708 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=d7da4d321c0ca494c819536ab9667b67071523a75a3bf5bb8ba8f9add9abe347 I0424 15:07:43.285715 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0424 15:07:43.285734 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=53243a17b414ef0047f7ea0e95d3b9c6689bdb919f856946814da2d40bc8774b I0424 15:07:43.285744 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0424 15:07:43.285759 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=3a8817b2872fbed5b6a9f57c404b1c56e996a1035abb8cfa24ab5276501706f6 I0424 15:07:43.285925 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=535d5dbd798fc4a199c07f4848dd8872c46eca50ad80106c2831fe849e7f695a I0424 15:07:43.285936 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0424 15:07:43.285943 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0424 15:07:43.285964 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0424 15:07:43.285986 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=e0641a1f8e475f2d8a999aed2c42ebbfa58a99cf03eaf6205c84cb230d5dbd9c I0424 15:07:43.286010 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=9e82d1ff9790945b654a30994584bde19e75989cfe811197f93ce1a6368f5c82 I0424 15:07:43.286019 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0424 15:07:43.286036 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=99414e2071b8a4bacba4da65048e3f9510510dadca101bce1d19fd4f90c9c694 I0424 15:07:43.286046 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0424 15:07:43.286065 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=b30d04bed7893d8133c55b751e6d500305085d1ba50bb49496786af49aa6668a I0424 15:07:43.286080 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=bb2ff02da3a55e6099ffdab89bc43b094860954b9e5b6016ca93ed04febacd2c I0424 15:07:43.286094 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=0926b6ce0528e479774158cbcd98aae34db69321bae6ad81b04f724a8431f2e9 I0424 15:07:43.286031 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-sdr5h pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-sdr5h\" is waiting to start: ContainerCreating" I0424 15:07:43.286109 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=ae9dd0561c35ab4504080e8ae50f80b222c551c3fe13cde068f081dd0d7eca81 I0424 15:07:43.286116 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-sdr5h\" is waiting to start: ContainerCreating" I0424 15:07:43.286128 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-z9wsl pod in namespace openshift-dns (previous: false). I0424 15:07:43.286130 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=0a0cc0220b02346de4b69b0daf9fe406c765fe171e103eea205b56f072493492 I0424 15:07:43.286137 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0424 15:07:43.286160 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=1b15279a2478b847fdc4f173f7a4b1ad0d408b454d432522135347d36736b131 I0424 15:07:43.286177 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0424 15:07:43.286186 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0424 15:07:43.286192 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.062078076s to process 36 records I0424 15:07:43.505574 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-z9wsl pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-z9wsl\" is waiting to start: ContainerCreating" I0424 15:07:43.505591 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-z9wsl\" is waiting to start: ContainerCreating" I0424 15:07:43.505599 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-z9wsl pod in namespace openshift-dns (previous: false). I0424 15:07:43.686034 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-z9wsl pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-z9wsl\" is waiting to start: ContainerCreating" I0424 15:07:43.686048 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-z9wsl\" is waiting to start: ContainerCreating" I0424 15:07:43.686057 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-h68ds pod in namespace openshift-dns (previous: false). I0424 15:07:43.887453 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:07:43.887467 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-qtblg pod in namespace openshift-dns (previous: false). I0424 15:07:44.085632 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:07:44.085645 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-vp57m pod in namespace openshift-dns (previous: false). W0424 15:07:44.251302 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:07:44.286709 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:07:44.286723 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-55bf5df46d-lz6tk pod in namespace openshift-image-registry (previous: false). I0424 15:07:44.487831 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-55bf5df46d-lz6tk pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-55bf5df46d-lz6tk\" is waiting to start: ContainerCreating" I0424 15:07:44.487846 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-55bf5df46d-lz6tk\" is waiting to start: ContainerCreating" I0424 15:07:44.487856 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-8645d987d7-4878c pod in namespace openshift-image-registry (previous: false). I0424 15:07:44.701786 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-8645d987d7-4878c pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-8645d987d7-4878c\" is waiting to start: ContainerCreating" I0424 15:07:44.701800 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-8645d987d7-4878c\" is waiting to start: ContainerCreating" I0424 15:07:44.701810 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-8645d987d7-9cbvd pod in namespace openshift-image-registry (previous: false). I0424 15:07:44.902691 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-8645d987d7-9cbvd pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-8645d987d7-9cbvd\" is waiting to start: ContainerCreating" I0424 15:07:44.902709 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-8645d987d7-9cbvd\" is waiting to start: ContainerCreating" I0424 15:07:44.902723 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-49lq8 pod in namespace openshift-image-registry (previous: false). I0424 15:07:45.086228 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:07:45.086243 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-cgsqk pod in namespace openshift-image-registry (previous: false). W0424 15:07:45.251722 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:07:45.289420 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:07:45.289432 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-grnmt pod in namespace openshift-image-registry (previous: false). I0424 15:07:45.485935 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:07:45.485952 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5947f5f6f5-tgvk7 pod in namespace openshift-ingress (previous: false). I0424 15:07:45.686734 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5947f5f6f5-tgvk7 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5947f5f6f5-tgvk7\" is waiting to start: ContainerCreating" I0424 15:07:45.686753 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5947f5f6f5-tgvk7\" is waiting to start: ContainerCreating" I0424 15:07:45.686765 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5947f5f6f5-tlxpb pod in namespace openshift-ingress (previous: false). I0424 15:07:45.885289 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5947f5f6f5-tlxpb pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5947f5f6f5-tlxpb\" is waiting to start: ContainerCreating" I0424 15:07:45.885302 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5947f5f6f5-tlxpb\" is waiting to start: ContainerCreating" I0424 15:07:45.885311 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-59789b96b7-q99ks pod in namespace openshift-ingress (previous: false). I0424 15:07:46.087971 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-59789b96b7-q99ks pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-59789b96b7-q99ks\" is waiting to start: ContainerCreating" I0424 15:07:46.087983 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-59789b96b7-q99ks\" is waiting to start: ContainerCreating" I0424 15:07:46.087994 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-cndjz pod in namespace openshift-ingress-canary (previous: false). W0424 15:07:46.252170 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0424 15:07:46.252195 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0424 15:07:46.252213 1 tasks_processing.go:74] worker 37 stopped. E0424 15:07:46.252227 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0424 15:07:46.252242 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0424 15:07:46.252262 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0424 15:07:46.252278 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.028427051s to process 1 records I0424 15:07:46.287534 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-cndjz pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-cndjz\" is waiting to start: ContainerCreating" I0424 15:07:46.287547 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-cndjz\" is waiting to start: ContainerCreating" I0424 15:07:46.287558 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-qgszr pod in namespace openshift-ingress-canary (previous: false). I0424 15:07:46.486215 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-qgszr pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-qgszr\" is waiting to start: ContainerCreating" I0424 15:07:46.486230 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-qgszr\" is waiting to start: ContainerCreating" I0424 15:07:46.486244 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-rvxqq pod in namespace openshift-ingress-canary (previous: false). I0424 15:07:46.687296 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-rvxqq pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rvxqq\" is waiting to start: ContainerCreating" I0424 15:07:46.687313 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rvxqq\" is waiting to start: ContainerCreating" I0424 15:07:46.687324 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-kjvhs pod in namespace openshift-kube-storage-version-migrator (previous: false). I0424 15:07:46.890324 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-kjvhs pod in namespace openshift-kube-storage-version-migrator (previous: false). I0424 15:07:47.087357 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-wt25h pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0424 15:07:47.289726 1 tasks_processing.go:74] worker 13 stopped. I0424 15:07:47.289860 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=d8e183135fb6351d08e72cb67da4e1216d5b4673d073d364f36485fa04138bd9 I0424 15:07:47.289951 1 recorder.go:75] Recording events/openshift-dns with fingerprint=45e8cd2cb040b0dd7a63dda1e820d5dcd8cfed2f9ce29fb43186a5ef45e86c36 I0424 15:07:47.290055 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=506ed4571b186a0ca442bbfaa0351d7ce1d2bae405c97f868bb700a13857db1a I0424 15:07:47.290085 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=c9212f6c6d14c09a1e29e5e6f92f3830b9ae27108b22223cc3b379e972c57b50 I0424 15:07:47.290130 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=b8186ad40d986405aa8dae4ee0510a2a9d82189e9856afd0296faf49df057507 I0424 15:07:47.290148 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=fee24fc0d14f1c1daef67b8176ffb646fbd40ed9758170e67dfde038309bbb0a I0424 15:07:47.290162 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=cff84e1d5b218e1277b3581f91efc09e20ed13eae67072c1358dc632d04bf97c I0424 15:07:47.290212 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=6600f4b1baee8d772cc9336e7dc0af0d61d04a8b3c35bb049635d971e3cfaaeb I0424 15:07:47.290227 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-kjvhs/migrator_current.log with fingerprint=a247dc183cb8c538b4c416c9ef04f361be620113d53582d7a855b77b1a940e58 I0424 15:07:47.290233 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-kjvhs/graceful-termination_current.log with fingerprint=8950bfc0612325110b1bec6899af7eb2889fc9e7c8489c15f6d7727d98fe471a I0424 15:07:47.290297 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-wt25h/kube-storage-version-migrator-operator_current.log with fingerprint=138cdea20b2d2a68b02b297af016da3bcc90d2a976d429d71eeeaa705e082579 I0424 15:07:47.290307 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.063413486s to process 11 records I0424 15:07:53.407829 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:07:53.449257 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:07:53.607249 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:07:53.859674 1 tasks_processing.go:74] worker 34 stopped. I0424 15:07:53.859713 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0424 15:07:53.859727 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.635725632s to process 1 records I0424 15:07:54.633545 1 tasks_processing.go:74] worker 39 stopped. I0424 15:07:54.633961 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=8134d5e41c8b88cd9e4cb52a44a4aaa7dc9c85df1ecff9db5f654f5b009336ee I0424 15:07:54.633982 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.406349713s to process 1 records E0424 15:07:54.634083 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.41s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0424 15:07:54.635193 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0424 15:07:54.635209 1 periodic.go:209] Running workloads gatherer I0424 15:07:54.635224 1 tasks_processing.go:45] number of workers: 2 I0424 15:07:54.635239 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 15:07:54.635243 1 tasks_processing.go:71] worker 1 working on workload_info task. I0424 15:07:54.635254 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 15:07:54.635270 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0424 15:07:54.658066 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0424 15:07:54.660338 1 tasks_processing.go:74] worker 0 stopped. I0424 15:07:54.660354 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 25.055654ms to process 0 records I0424 15:07:54.670155 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (13ms) I0424 15:07:54.680184 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0424 15:07:54.688627 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0424 15:07:54.697163 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (9ms) I0424 15:07:54.705627 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (8ms) I0424 15:07:54.714050 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (8ms) I0424 15:07:54.722308 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (8ms) I0424 15:07:54.730948 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (9ms) I0424 15:07:54.739451 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (8ms) I0424 15:07:54.748076 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (9ms) I0424 15:07:54.767191 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (19ms) I0424 15:07:54.867092 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0424 15:07:54.968228 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (101ms) I0424 15:07:55.068824 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (101ms) I0424 15:07:55.168059 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0424 15:07:55.268628 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (101ms) I0424 15:07:55.367978 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (99ms) I0424 15:07:55.468006 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0424 15:07:55.572342 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (104ms) I0424 15:07:55.673518 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0424 15:07:55.768264 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (95ms) I0424 15:07:55.867603 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (99ms) I0424 15:07:55.967977 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0424 15:07:56.068172 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0424 15:07:56.168017 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0424 15:07:56.266948 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0424 15:07:56.368422 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0424 15:07:56.469086 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0424 15:07:56.575437 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (106ms) I0424 15:07:56.668058 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (93ms) I0424 15:07:56.767894 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0424 15:07:56.867911 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (100ms) I0424 15:07:56.968330 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0424 15:07:56.968357 1 tasks_processing.go:74] worker 1 stopped. E0424 15:07:56.968368 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0424 15:07:56.968654 1 recorder.go:75] Recording config/workload_info with fingerprint=4ee43681e24e89abf2e2c2c933908f2e048be8bf8bfd8be5c7b4f143818212a3 I0424 15:07:56.968668 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.333106407s to process 1 records E0424 15:07:56.968702 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.333s with: function \"workload_info\" failed with an error" I0424 15:07:56.969800 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0424 15:07:56.969811 1 periodic.go:209] Running conditional gatherer I0424 15:07:56.975499 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0424 15:07:56.981638 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.13:47748->172.30.0.10:53: read: connection refused E0424 15:07:56.981861 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 15:07:56.981930 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0424 15:07:56.987606 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0424 15:07:56.987620 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987625 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987628 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987633 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987635 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987638 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987641 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987644 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:07:56.987647 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0424 15:07:56.987660 1 tasks_processing.go:45] number of workers: 3 I0424 15:07:56.987673 1 tasks_processing.go:69] worker 2 listening for tasks. I0424 15:07:56.987676 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0424 15:07:56.987680 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 15:07:56.987690 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0424 15:07:56.987690 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 15:07:56.987696 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0424 15:07:56.987698 1 tasks_processing.go:74] worker 1 stopped. I0424 15:07:56.987744 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0424 15:07:56.987755 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 735ns to process 1 records I0424 15:07:56.987785 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0424 15:07:56.987792 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.014µs to process 1 records I0424 15:07:56.987798 1 tasks_processing.go:74] worker 0 stopped. I0424 15:07:56.987940 1 tasks_processing.go:74] worker 2 stopped. I0424 15:07:56.987953 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 235.598µs to process 0 records I0424 15:07:56.987980 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.13:47748->172.30.0.10:53: read: connection refused I0424 15:07:56.987998 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0424 15:07:57.009411 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=3311b7c7d166b84e10b83eff284500bbcb80867d3b94dd6263abaa4c06ccace8 I0424 15:07:57.009524 1 diskrecorder.go:70] Writing 106 records to /var/lib/insights-operator/insights-2026-04-24-150757.tar.gz I0424 15:07:57.016137 1 diskrecorder.go:51] Wrote 106 records to disk in 6ms I0424 15:07:57.016163 1 periodic.go:278] Gathering cluster info every 2h0m0s I0424 15:07:57.016175 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0424 15:08:09.030455 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:09:00.622080 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="0fc33dd2a67a5be74c21d29bd03b84fbd5efc3a360a4582988d8ab1c4adca743") W0424 15:09:00.622120 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0424 15:09:00.622164 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="60a626faa7f558314ab6106beb4c56948d8a1f243aa6fc102bf1dd4d33ff0e4a") I0424 15:09:00.622252 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="d850fb3e2a29d28ec66dfba61467c8d8ae422d5cb6b894f547ecb981a40bfe5b") I0424 15:09:00.622257 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0424 15:09:00.622275 1 base_controller.go:181] Shutting down ConfigController ... I0424 15:09:00.622293 1 base_controller.go:181] Shutting down LoggingSyncer ... I0424 15:09:00.622293 1 periodic.go:170] Shutting down I0424 15:09:00.622312 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0424 15:09:00.622319 1 base_controller.go:113] All ConfigController workers have been terminated I0424 15:09:00.622327 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0424 15:09:00.622256 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0424 15:09:00.622345 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0424 15:09:00.622354 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" I0424 15:09:00.622362 1 base_controller.go:113] All LoggingSyncer workers have been terminated