W0512 14:58:31.026198 1 cmd.go:257] Using insecure, self-signed certificates I0512 14:58:31.337134 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 14:58:31.337465 1 observer_polling.go:159] Starting file observer I0512 14:58:31.620334 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0512 14:58:31.620532 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0512 14:58:31.621108 1 secure_serving.go:57] Forcing use of http/1.1 only W0512 14:58:31.621133 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0512 14:58:31.621143 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. I0512 14:58:31.621139 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0512 14:58:31.621150 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0512 14:58:31.621154 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0512 14:58:31.621158 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0512 14:58:31.621163 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0512 14:58:31.625755 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0512 14:58:31.625775 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"ad49380a-fbe1-4792-a744-bc6d65cd081a", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0512 14:58:31.627741 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0512 14:58:31.627756 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0512 14:58:31.627763 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0512 14:58:31.627780 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 14:58:31.627782 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 14:58:31.627767 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0512 14:58:31.628037 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-642830154/tls.crt::/tmp/serving-cert-642830154/tls.key" I0512 14:58:31.628258 1 secure_serving.go:213] Serving securely on [::]:8443 I0512 14:58:31.628289 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0512 14:58:31.640801 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0512 14:58:31.640840 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 14:58:31.640960 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0512 14:58:31.650242 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 14:58:31.650267 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 14:58:31.657345 1 secretconfigobserver.go:119] support secret does not exist I0512 14:58:31.663669 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 14:58:31.670077 1 secretconfigobserver.go:119] support secret does not exist I0512 14:58:31.675120 1 recorder.go:161] Pruning old reports every 6h0m50s, max age is 288h0m0s I0512 14:58:31.682021 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0512 14:58:31.682026 1 periodic.go:209] Running clusterconfig gatherer I0512 14:58:31.682042 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0512 14:58:31.682075 1 tasks_processing.go:45] number of workers: 64 I0512 14:58:31.682109 1 tasks_processing.go:69] worker 14 listening for tasks. I0512 14:58:31.682120 1 tasks_processing.go:69] worker 63 listening for tasks. I0512 14:58:31.682124 1 tasks_processing.go:71] worker 14 working on machine_sets task. I0512 14:58:31.682124 1 tasks_processing.go:69] worker 35 listening for tasks. I0512 14:58:31.682126 1 tasks_processing.go:69] worker 36 listening for tasks. I0512 14:58:31.682130 1 tasks_processing.go:69] worker 37 listening for tasks. I0512 14:58:31.682133 1 tasks_processing.go:69] worker 6 listening for tasks. I0512 14:58:31.682136 1 tasks_processing.go:69] worker 15 listening for tasks. I0512 14:58:31.682028 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0512 14:58:31.682142 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 14:58:31.682143 1 tasks_processing.go:69] worker 39 listening for tasks. I0512 14:58:31.682144 1 insightsreport.go:296] Starting report retriever I0512 14:58:31.682150 1 tasks_processing.go:69] worker 16 listening for tasks. I0512 14:58:31.682150 1 tasks_processing.go:69] worker 10 listening for tasks. I0512 14:58:31.682153 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0512 14:58:31.682152 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 14:58:31.682157 1 tasks_processing.go:69] worker 17 listening for tasks. I0512 14:58:31.682163 1 tasks_processing.go:69] worker 4 listening for tasks. I0512 14:58:31.682148 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 14:58:31.682167 1 tasks_processing.go:69] worker 53 listening for tasks. I0512 14:58:31.682170 1 tasks_processing.go:69] worker 7 listening for tasks. I0512 14:58:31.682161 1 tasks_processing.go:69] worker 18 listening for tasks. I0512 14:58:31.682172 1 tasks_processing.go:69] worker 5 listening for tasks. I0512 14:58:31.682163 1 tasks_processing.go:69] worker 52 listening for tasks. I0512 14:58:31.682158 1 tasks_processing.go:69] worker 3 listening for tasks. I0512 14:58:31.682167 1 tasks_processing.go:69] worker 19 listening for tasks. I0512 14:58:31.682172 1 tasks_processing.go:69] worker 20 listening for tasks. I0512 14:58:31.682175 1 tasks_processing.go:69] worker 54 listening for tasks. I0512 14:58:31.682186 1 tasks_processing.go:69] worker 30 listening for tasks. I0512 14:58:31.682176 1 tasks_processing.go:69] worker 21 listening for tasks. I0512 14:58:31.682181 1 tasks_processing.go:69] worker 29 listening for tasks. I0512 14:58:31.682181 1 tasks_processing.go:69] worker 55 listening for tasks. I0512 14:58:31.682192 1 tasks_processing.go:69] worker 40 listening for tasks. I0512 14:58:31.682181 1 tasks_processing.go:69] worker 51 listening for tasks. I0512 14:58:31.682188 1 tasks_processing.go:69] worker 56 listening for tasks. I0512 14:58:31.682195 1 tasks_processing.go:69] worker 57 listening for tasks. I0512 14:58:31.682193 1 tasks_processing.go:69] worker 28 listening for tasks. I0512 14:58:31.682205 1 tasks_processing.go:69] worker 46 listening for tasks. I0512 14:58:31.682229 1 tasks_processing.go:69] worker 9 listening for tasks. I0512 14:58:31.682229 1 tasks_processing.go:69] worker 47 listening for tasks. I0512 14:58:31.682236 1 tasks_processing.go:69] worker 49 listening for tasks. I0512 14:58:31.682627 1 tasks_processing.go:69] worker 8 listening for tasks. I0512 14:58:31.682643 1 tasks_processing.go:69] worker 43 listening for tasks. I0512 14:58:31.682652 1 tasks_processing.go:69] worker 61 listening for tasks. I0512 14:58:31.682650 1 tasks_processing.go:69] worker 41 listening for tasks. I0512 14:58:31.682659 1 tasks_processing.go:69] worker 59 listening for tasks. I0512 14:58:31.682667 1 tasks_processing.go:69] worker 60 listening for tasks. I0512 14:58:31.682668 1 tasks_processing.go:69] worker 44 listening for tasks. I0512 14:58:31.682675 1 tasks_processing.go:69] worker 34 listening for tasks. I0512 14:58:31.682678 1 tasks_processing.go:69] worker 32 listening for tasks. I0512 14:58:31.682676 1 tasks_processing.go:69] worker 26 listening for tasks. I0512 14:58:31.682686 1 tasks_processing.go:69] worker 13 listening for tasks. I0512 14:58:31.682683 1 tasks_processing.go:69] worker 62 listening for tasks. I0512 14:58:31.682139 1 tasks_processing.go:69] worker 38 listening for tasks. I0512 14:58:31.682694 1 tasks_processing.go:69] worker 23 listening for tasks. I0512 14:58:31.682841 1 tasks_processing.go:71] worker 3 working on operators_pods_and_events task. I0512 14:58:31.683048 1 tasks_processing.go:69] worker 33 listening for tasks. I0512 14:58:31.683061 1 tasks_processing.go:71] worker 10 working on image task. I0512 14:58:31.683060 1 tasks_processing.go:69] worker 27 listening for tasks. I0512 14:58:31.683072 1 tasks_processing.go:71] worker 56 working on silenced_alerts task. I0512 14:58:31.683083 1 tasks_processing.go:69] worker 24 listening for tasks. I0512 14:58:31.683094 1 tasks_processing.go:71] worker 47 working on networks task. W0512 14:58:31.683102 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 14:58:31.683109 1 tasks_processing.go:69] worker 11 listening for tasks. I0512 14:58:31.683120 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 38.463µs to process 0 records I0512 14:58:31.683124 1 tasks_processing.go:71] worker 6 working on storage_cluster task. I0512 14:58:31.683129 1 tasks_processing.go:71] worker 20 working on sap_config task. I0512 14:58:31.683132 1 tasks_processing.go:69] worker 42 listening for tasks. I0512 14:58:31.683144 1 tasks_processing.go:69] worker 12 listening for tasks. I0512 14:58:31.683151 1 tasks_processing.go:71] worker 0 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0512 14:58:31.683156 1 tasks_processing.go:71] worker 35 working on metrics task. W0512 14:58:31.683184 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 14:58:31.683126 1 tasks_processing.go:71] worker 19 working on machine_configs task. I0512 14:58:31.683197 1 tasks_processing.go:71] worker 63 working on sap_pods task. I0512 14:58:31.683197 1 tasks_processing.go:71] worker 39 working on schedulers task. I0512 14:58:31.683851 1 tasks_processing.go:69] worker 31 listening for tasks. I0512 14:58:31.683978 1 tasks_processing.go:71] worker 31 working on machine_healthchecks task. I0512 14:58:31.683985 1 tasks_processing.go:69] worker 45 listening for tasks. I0512 14:58:31.684321 1 tasks_processing.go:71] worker 45 working on version task. I0512 14:58:31.684376 1 tasks_processing.go:71] worker 5 working on oauths task. I0512 14:58:31.684413 1 tasks_processing.go:71] worker 52 working on ingress_certificates task. I0512 14:58:31.684440 1 tasks_processing.go:71] worker 41 working on image_registries task. I0512 14:58:31.684455 1 tasks_processing.go:71] worker 24 working on openshift_logging task. I0512 14:58:31.684475 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 33.42µs to process 0 records I0512 14:58:31.684477 1 tasks_processing.go:71] worker 12 working on support_secret task. I0512 14:58:31.684330 1 tasks_processing.go:71] worker 27 working on qemu_kubevirt_launcher_logs task. I0512 14:58:31.684392 1 tasks_processing.go:71] worker 56 working on openshift_machine_api_events task. I0512 14:58:31.683864 1 tasks_processing.go:71] worker 60 working on config_maps task. I0512 14:58:31.683873 1 tasks_processing.go:69] worker 25 listening for tasks. I0512 14:58:31.684896 1 tasks_processing.go:71] worker 25 working on install_plans task. I0512 14:58:31.683914 1 tasks_processing.go:71] worker 44 working on pod_network_connectivity_checks task. I0512 14:58:31.685091 1 tasks_processing.go:71] worker 59 working on tsdb_status task. I0512 14:58:31.683788 1 tasks_processing.go:69] worker 48 listening for tasks. I0512 14:58:31.685182 1 tasks_processing.go:71] worker 48 working on mutating_webhook_configurations task. I0512 14:58:31.683928 1 tasks_processing.go:71] worker 29 working on openstack_dataplanedeployments task. W0512 14:58:31.685185 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 14:58:31.683801 1 tasks_processing.go:69] worker 50 listening for tasks. I0512 14:58:31.683939 1 tasks_processing.go:71] worker 37 working on ingress task. I0512 14:58:31.685204 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 93.679µs to process 0 records I0512 14:58:31.685237 1 tasks_processing.go:71] worker 59 working on authentication task. I0512 14:58:31.683952 1 tasks_processing.go:71] worker 18 working on cluster_apiserver task. I0512 14:58:31.683952 1 tasks_processing.go:71] worker 53 working on container_runtime_configs task. I0512 14:58:31.683959 1 tasks_processing.go:71] worker 2 working on machine_config_pools task. I0512 14:58:31.683952 1 tasks_processing.go:71] worker 36 working on olm_operators task. I0512 14:58:31.683957 1 tasks_processing.go:71] worker 7 working on service_accounts task. I0512 14:58:31.684024 1 tasks_processing.go:71] worker 54 working on cost_management_metrics_configs task. I0512 14:58:31.684027 1 tasks_processing.go:71] worker 46 working on operators task. I0512 14:58:31.684032 1 tasks_processing.go:71] worker 34 working on overlapping_namespace_uids task. I0512 14:58:31.684030 1 tasks_processing.go:71] worker 21 working on node_logs task. I0512 14:58:31.684050 1 tasks_processing.go:71] worker 30 working on feature_gates task. I0512 14:58:31.686395 1 tasks_processing.go:74] worker 14 stopped. I0512 14:58:31.686410 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 4.262761ms to process 0 records I0512 14:58:31.684070 1 tasks_processing.go:71] worker 57 working on image_pruners task. I0512 14:58:31.683201 1 tasks_processing.go:71] worker 15 working on lokistack task. I0512 14:58:31.684071 1 tasks_processing.go:71] worker 28 working on pdbs task. I0512 14:58:31.684083 1 tasks_processing.go:71] worker 62 working on nodes task. I0512 14:58:31.684085 1 tasks_processing.go:71] worker 9 working on machine_autoscalers task. I0512 14:58:31.684126 1 tasks_processing.go:71] worker 33 working on sap_datahubs task. I0512 14:58:31.684130 1 tasks_processing.go:71] worker 17 working on openstack_controlplanes task. I0512 14:58:31.684131 1 tasks_processing.go:71] worker 40 working on nodenetworkstates task. I0512 14:58:31.684160 1 tasks_processing.go:71] worker 4 working on ceph_cluster task. I0512 14:58:31.684181 1 tasks_processing.go:71] worker 32 working on validating_webhook_configurations task. I0512 14:58:31.684188 1 tasks_processing.go:71] worker 38 working on clusterroles task. I0512 14:58:31.684200 1 tasks_processing.go:71] worker 26 working on infrastructures task. I0512 14:58:31.684251 1 tasks_processing.go:71] worker 43 working on machines task. I0512 14:58:31.684261 1 tasks_processing.go:71] worker 55 working on aggregated_monitoring_cr_names task. I0512 14:58:31.684264 1 tasks_processing.go:71] worker 51 working on jaegers task. I0512 14:58:31.684270 1 tasks_processing.go:71] worker 49 working on active_alerts task. I0512 14:58:31.684275 1 tasks_processing.go:71] worker 23 working on container_images task. W0512 14:58:31.687183 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 14:58:31.687191 1 tasks_processing.go:74] worker 49 stopped. I0512 14:58:31.684276 1 tasks_processing.go:71] worker 1 working on dvo_metrics task. I0512 14:58:31.687199 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 22.993µs to process 0 records I0512 14:58:31.684301 1 tasks_processing.go:71] worker 13 working on proxies task. I0512 14:58:31.684341 1 tasks_processing.go:71] worker 11 working on certificate_signing_requests task. I0512 14:58:31.684344 1 tasks_processing.go:71] worker 8 working on openstack_version task. I0512 14:58:31.684369 1 tasks_processing.go:71] worker 42 working on crds task. I0512 14:58:31.683201 1 tasks_processing.go:71] worker 16 working on openstack_dataplanenodesets task. I0512 14:58:31.684401 1 tasks_processing.go:71] worker 61 working on nodenetworkconfigurationpolicies task. I0512 14:58:31.683838 1 tasks_processing.go:71] worker 35 working on monitoring_persistent_volumes task. I0512 14:58:31.683837 1 tasks_processing.go:69] worker 22 listening for tasks. I0512 14:58:31.687655 1 tasks_processing.go:74] worker 22 stopped. I0512 14:58:31.683811 1 tasks_processing.go:69] worker 58 listening for tasks. I0512 14:58:31.687662 1 tasks_processing.go:74] worker 58 stopped. I0512 14:58:31.685235 1 tasks_processing.go:71] worker 50 working on storage_classes task. I0512 14:58:31.689096 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0512 14:58:31.689112 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 14:58:31.689117 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0512 14:58:31.689121 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0512 14:58:31.689165 1 controller.go:489] The operator is still being initialized I0512 14:58:31.689173 1 controller.go:512] The operator is healthy I0512 14:58:31.690507 1 tasks_processing.go:74] worker 6 stopped. I0512 14:58:31.690520 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 7.368812ms to process 0 records I0512 14:58:31.690527 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 6.020948ms to process 0 records I0512 14:58:31.690532 1 tasks_processing.go:74] worker 24 stopped. I0512 14:58:31.702191 1 tasks_processing.go:74] worker 63 stopped. I0512 14:58:31.702230 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 18.406005ms to process 0 records I0512 14:58:31.702651 1 tasks_processing.go:74] worker 54 stopped. I0512 14:58:31.702662 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 17.079918ms to process 0 records I0512 14:58:31.703231 1 tasks_processing.go:74] worker 44 stopped. E0512 14:58:31.703243 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0512 14:58:31.703251 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 18.161524ms to process 0 records I0512 14:58:31.703870 1 tasks_processing.go:74] worker 40 stopped. I0512 14:58:31.703882 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 17.062014ms to process 0 records I0512 14:58:31.705754 1 tasks_processing.go:74] worker 5 stopped. I0512 14:58:31.705980 1 recorder.go:75] Recording config/oauth with fingerprint=8054ea23ef46035ea8a905860ed7dcdbc9a691e9f590b9a85aadf0df41a45c3b I0512 14:58:31.705994 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 21.318383ms to process 1 records I0512 14:58:31.707273 1 tasks_processing.go:74] worker 15 stopped. I0512 14:58:31.707287 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 20.77567ms to process 0 records I0512 14:58:31.707297 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 22.074522ms to process 0 records I0512 14:58:31.707303 1 tasks_processing.go:74] worker 29 stopped. I0512 14:58:31.707400 1 tasks_processing.go:74] worker 17 stopped. I0512 14:58:31.707408 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 20.632695ms to process 0 records I0512 14:58:31.707414 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 20.766801ms to process 0 records I0512 14:58:31.707419 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 20.688926ms to process 0 records I0512 14:58:31.707423 1 tasks_processing.go:74] worker 33 stopped. I0512 14:58:31.707426 1 tasks_processing.go:74] worker 4 stopped. I0512 14:58:31.707427 1 tasks_processing.go:74] worker 9 stopped. I0512 14:58:31.707433 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 20.567557ms to process 0 records I0512 14:58:31.707534 1 tasks_processing.go:74] worker 51 stopped. I0512 14:58:31.707552 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 20.425479ms to process 0 records I0512 14:58:31.707638 1 tasks_processing.go:74] worker 43 stopped. E0512 14:58:31.707650 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0512 14:58:31.707658 1 gather.go:177] gatherer "clusterconfig" function "machines" took 20.587261ms to process 0 records I0512 14:58:31.707761 1 tasks_processing.go:74] worker 18 stopped. I0512 14:58:31.707878 1 recorder.go:75] Recording config/apiserver with fingerprint=bd36f5f4e902972c0dd5ad14ea305ccee948ff8a9f4345e7df97ffeb93306fb1 I0512 14:58:31.707890 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 22.469002ms to process 1 records I0512 14:58:31.708018 1 tasks_processing.go:74] worker 61 stopped. I0512 14:58:31.708032 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 20.456326ms to process 0 records I0512 14:58:31.708039 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 20.492252ms to process 0 records I0512 14:58:31.708043 1 tasks_processing.go:74] worker 16 stopped. I0512 14:58:31.712515 1 tasks_processing.go:74] worker 8 stopped. I0512 14:58:31.712536 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 25.132599ms to process 0 records E0512 14:58:31.712557 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0512 14:58:31.712570 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 28.526667ms to process 0 records I0512 14:58:31.712581 1 tasks_processing.go:74] worker 31 stopped. I0512 14:58:31.712616 1 tasks_processing.go:74] worker 10 stopped. I0512 14:58:31.712697 1 recorder.go:75] Recording config/image with fingerprint=4688410503321b1f719c931080e68fc218f9e57a0c2ee755201039212f688924 I0512 14:58:31.712711 1 gather.go:177] gatherer "clusterconfig" function "image" took 29.549524ms to process 1 records I0512 14:58:31.714565 1 tasks_processing.go:74] worker 47 stopped. I0512 14:58:31.714723 1 recorder.go:75] Recording config/network with fingerprint=11f08ba29b8ae5e25301e276ab38255387b6aa2103917464e7663fa13edb69cc I0512 14:58:31.714737 1 gather.go:177] gatherer "clusterconfig" function "networks" took 31.445459ms to process 1 records I0512 14:58:31.715318 1 tasks_processing.go:74] worker 39 stopped. I0512 14:58:31.715399 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=d01691774a309aabb1e68e71e7ed64984c08839589afde2f12c17b1157f58b3d I0512 14:58:31.715414 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 31.38555ms to process 1 records I0512 14:58:31.725894 1 tasks_processing.go:74] worker 20 stopped. I0512 14:58:31.725913 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 42.748913ms to process 0 records I0512 14:58:31.726170 1 tasks_processing.go:74] worker 53 stopped. I0512 14:58:31.726184 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 40.858384ms to process 0 records I0512 14:58:31.728344 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0512 14:58:31.728370 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 14:58:31.728350 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 14:58:31.728690 1 tasks_processing.go:74] worker 59 stopped. I0512 14:58:31.728847 1 recorder.go:75] Recording config/authentication with fingerprint=245e29ee23c8473af892284bcfd53fab9477e2e5790dab50339614f9024e5ae8 I0512 14:58:31.728860 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 43.432915ms to process 1 records I0512 14:58:31.730814 1 tasks_processing.go:74] worker 35 stopped. I0512 14:58:31.730836 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 43.172698ms to process 0 records I0512 14:58:31.730922 1 tasks_processing.go:74] worker 50 stopped. I0512 14:58:31.730949 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=f3237526606088a6d81e4673057b116f15c26dee95dfc4a31059be1a225cf637 I0512 14:58:31.730965 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=38aa28254cd27d63d5a4b81ce1983a3e1fa48e43644ddb16e0ba71d17f72abd9 I0512 14:58:31.730971 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 43.16484ms to process 2 records I0512 14:58:31.731272 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0512 14:58:31.731366 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0512 14:58:31.731394 1 operator.go:288] started I0512 14:58:31.731414 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0512 14:58:31.731683 1 tasks_processing.go:74] worker 48 stopped. I0512 14:58:31.731889 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=0ff9c482d50a11f6bd45e5cf202bbefa9ab91549a34d2a8fe235c99de363ef6a I0512 14:58:31.731928 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=aa150e30067be2b6720502c3ce55c39ea2d5428a151149bccfe7b7a345e48b6b I0512 14:58:31.731953 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=04334dc023846cd6a609c3ecaa4ece5ff39bf75eda725eaf7faa7a7676fc9334 I0512 14:58:31.731962 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 46.481352ms to process 3 records I0512 14:58:31.732370 1 tasks_processing.go:74] worker 12 stopped. E0512 14:58:31.732381 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0512 14:58:31.732389 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 47.878028ms to process 0 records I0512 14:58:31.732702 1 tasks_processing.go:74] worker 11 stopped. I0512 14:58:31.732717 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 45.414029ms to process 0 records W0512 14:58:31.735278 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 14:58:31.742038 1 base_controller.go:82] Caches are synced for ConfigController I0512 14:58:31.742057 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0512 14:58:31.746360 1 tasks_processing.go:74] worker 26 stopped. I0512 14:58:31.746927 1 recorder.go:75] Recording config/infrastructure with fingerprint=6ed7bf302bc4a59fd612ab373c5c9a922cef6bcc16829e15d02ff429c51d171c I0512 14:58:31.746944 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 59.340029ms to process 1 records I0512 14:58:31.747041 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=e10bb01ebbce36be136826be1a353072c7f86e51fc280de2da0b55a5deb7ca75 I0512 14:58:31.747044 1 tasks_processing.go:74] worker 32 stopped. I0512 14:58:31.747117 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=671f66eb19a871257091bc9867912ec3ada3ee1a90acf8fe5acb9a6203d48098 I0512 14:58:31.747136 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=d32a1c0efcce8a299ab8aba4f57acac893fcb9cb7ac29cf9a867d04ff3945ec7 I0512 14:58:31.747173 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=aa7bab8f81215fe3214322465c2b3ca94b814e2dfc61a521b83a753adae4d67c I0512 14:58:31.747198 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=16b12ee4144865a8feb1e2dd762fa1c804d2c20c3896493266894ed7dcdf9cbc I0512 14:58:31.747245 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=f2281a38275d52808db49c94d1be48df43f756e2847627b0ba32153c02597cf4 I0512 14:58:31.747270 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=c61452067ef6be2cdb1fe2a3d30e4522a3c6b7d25ecac771205b62a892c18426 I0512 14:58:31.747303 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=abedcd8ea47e10a724866ef7dff363ac93deb08921417e083b13aa2d87fa6bc2 I0512 14:58:31.747327 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=0429fc2e8b35c0bace847175705f951be24a0979b1345d34226992d2c3fd6457 I0512 14:58:31.747349 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=16dee3777ab1a72c4fab5e4c18c38d645963f3ef13082cd61250a52cc6cce886 I0512 14:58:31.747373 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=3e98eea0f36bd3afb0a9200289a910f2ca13db1a5d7e6d62f15b75432f9b94b3 I0512 14:58:31.747381 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 59.529412ms to process 11 records I0512 14:58:31.747459 1 tasks_processing.go:74] worker 37 stopped. I0512 14:58:31.747488 1 recorder.go:75] Recording config/ingress with fingerprint=c0c7de27ee19f2c61f46faaea234d0f0c96c55526d36f20b001f544a43c08abb I0512 14:58:31.747497 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 61.66018ms to process 1 records I0512 14:58:31.747502 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 62.323117ms to process 0 records I0512 14:58:31.747508 1 tasks_processing.go:74] worker 56 stopped. I0512 14:58:31.749529 1 tasks_processing.go:74] worker 28 stopped. I0512 14:58:31.749724 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=cb9d0ebc98eff9ca72615ef73113e8bc9d3ad083e25a77870560920570aa9a0b I0512 14:58:31.749810 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=8b8333db30923f8f866e878d9e798716e5b02560918fb59273a8af04578cd126 I0512 14:58:31.749859 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=4d9e8e9d56cef9406ac9ae67b7aa8daa4bf5e25e70ac9832fc2f40342a9de06c I0512 14:58:31.749897 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 62.99083ms to process 3 records I0512 14:58:31.751020 1 tasks_processing.go:74] worker 45 stopped. I0512 14:58:31.751393 1 recorder.go:75] Recording config/version with fingerprint=32f2aca978651cf317a27208f8eb8ed2c48419b06f8f3d8d281a8b0bc0b4a1cb I0512 14:58:31.751416 1 recorder.go:75] Recording config/id with fingerprint=8e57a34b29acb3742aa6aa633ac66f5d0394638313cf027b6a316a5f99611672 I0512 14:58:31.751431 1 gather.go:177] gatherer "clusterconfig" function "version" took 66.632589ms to process 2 records I0512 14:58:31.751482 1 gather_logs.go:145] no pods in namespace were found I0512 14:58:31.751515 1 tasks_processing.go:74] worker 57 stopped. I0512 14:58:31.751616 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=bdaca3b0e09fa94bac8ab1ed7ebecb2e064a190b9eddb4058b8069fa9d803c52 I0512 14:58:31.751627 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 64.916751ms to process 1 records I0512 14:58:31.751678 1 recorder.go:75] Recording config/proxy with fingerprint=a04f28bb5f7cf284c91c33ab54c48e249150ffb3ec647f25fee3f1b0c0f164cf I0512 14:58:31.751686 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 64.146754ms to process 1 records I0512 14:58:31.751721 1 tasks_processing.go:74] worker 13 stopped. I0512 14:58:31.751784 1 tasks_processing.go:74] worker 62 stopped. I0512 14:58:31.751938 1 recorder.go:75] Recording config/node/ip-10-0-0-65.ec2.internal with fingerprint=4908e66c90ed798342613b63de8ebe0aabfed0ca9fef95a347c1e00b300b9c89 I0512 14:58:31.752001 1 recorder.go:75] Recording config/node/ip-10-0-1-25.ec2.internal with fingerprint=79c339aecb135fafdc2c66aa1fb8b10c088ec3633a60b7de0cefe21c4b8dbf17 I0512 14:58:31.752087 1 recorder.go:75] Recording config/node/ip-10-0-2-223.ec2.internal with fingerprint=d4228a49a023379831b0d8da144d93adf8cc68238c37d593e4113678b7084b00 I0512 14:58:31.752096 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 64.82061ms to process 3 records I0512 14:58:31.752108 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 66.987197ms to process 0 records I0512 14:58:31.752121 1 tasks_processing.go:74] worker 27 stopped. I0512 14:58:31.756496 1 tasks_processing.go:74] worker 41 stopped. I0512 14:58:31.756811 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=1c0a6872c1e1241f8b898e750c8633367a8bc8aa1926da3c754244757b804c14 I0512 14:58:31.756825 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 72.03572ms to process 1 records I0512 14:58:31.757392 1 tasks_processing.go:74] worker 21 stopped. I0512 14:58:31.757408 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 71.378532ms to process 0 records I0512 14:58:31.757748 1 tasks_processing.go:74] worker 30 stopped. I0512 14:58:31.757926 1 recorder.go:75] Recording config/featuregate with fingerprint=4dcc39bfc86e513175780ea946b9b7b05bac43dd20dbabc4ea19b4687c004339 I0512 14:58:31.757943 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 71.367335ms to process 1 records I0512 14:58:31.757966 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0512 14:58:31.757975 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 72.266939ms to process 1 records I0512 14:58:31.757984 1 tasks_processing.go:74] worker 34 stopped. I0512 14:58:31.759420 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 14:58:31.759433 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0512 14:58:31.759437 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0512 14:58:31.759441 1 controller.go:212] Source scaController *sca.Controller is not ready I0512 14:58:31.759445 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0512 14:58:31.759466 1 controller.go:489] The operator is still being initialized I0512 14:58:31.759472 1 controller.go:512] The operator is healthy I0512 14:58:31.759752 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 14:58:31.760238 1 prometheus_rules.go:88] Prometheus rules successfully created I0512 14:58:31.762305 1 tasks_processing.go:74] worker 2 stopped. I0512 14:58:31.762323 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 76.89007ms to process 0 records E0512 14:58:31.769112 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%272be932fd-2239-4db1-a217-e917451dc5f1%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.13:45929->172.30.0.10:53: read: connection refused I0512 14:58:31.769131 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%272be932fd-2239-4db1-a217-e917451dc5f1%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.13:45929->172.30.0.10:53: read: connection refused I0512 14:58:31.772147 1 tasks_processing.go:74] worker 42 stopped. I0512 14:58:31.772699 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=c0da9752a7376833112bcd0db5d95a2528b3eeca35182f359016f9e38cb30362 I0512 14:58:31.772912 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=ca7eefc45631ad11238036050326a7ea599dfbe2b61aca35e4f5dd092c08d15d I0512 14:58:31.772923 1 gather.go:177] gatherer "clusterconfig" function "crds" took 84.687001ms to process 2 records I0512 14:58:31.776201 1 tasks_processing.go:74] worker 36 stopped. I0512 14:58:31.776278 1 recorder.go:75] Recording config/olm_operators with fingerprint=7451b78b97a69c075ed52d43908a16ce00bd5f97c3913bb9b70e0782bded8074 I0512 14:58:31.776292 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 90.731021ms to process 1 records I0512 14:58:31.778934 1 tasks_processing.go:74] worker 38 stopped. I0512 14:58:31.779128 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=b6987e782a79de4d286e09a53af9c7cf92e68851aeb9ea529e57a63e1e1118d5 I0512 14:58:31.779234 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=207b01d78c1f98b1c1650fe70a09f4522984479abea592f9d0e609688702aedc I0512 14:58:31.779245 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 91.987674ms to process 2 records I0512 14:58:31.780901 1 tasks_processing.go:74] worker 55 stopped. I0512 14:58:31.780924 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 93.831689ms to process 0 records I0512 14:58:31.782191 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0512 14:58:31.785581 1 tasks_processing.go:74] worker 0 stopped. I0512 14:58:31.785606 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 102.408025ms to process 0 records I0512 14:58:31.786023 1 tasks_processing.go:74] worker 23 stopped. W0512 14:58:31.786029 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.13:52995->172.30.0.10:53: read: connection refused I0512 14:58:31.786047 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.13:52995->172.30.0.10:53: read: connection refused I0512 14:58:31.786372 1 recorder.go:75] Recording config/running_containers with fingerprint=af6a6a82a6a56a757d2d3b2457f94d95f73e8452b1d0c9c34314016d009ac2f3 I0512 14:58:31.786405 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 98.8308ms to process 1 records I0512 14:58:31.798962 1 tasks_processing.go:74] worker 52 stopped. E0512 14:58:31.798985 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0512 14:58:31.798993 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q8epm7mkn3f557mogbg81pdvmdhs3mn-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q8epm7mkn3f557mogbg81pdvmdhs3mn-primary-cert-bundle-secret" not found I0512 14:58:31.799054 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=d2c3121ea22497c3cb92230fa9b670a746cbfa288c3ffc3709ae8d8667fd768b I0512 14:58:31.799068 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 114.51477ms to process 1 records I0512 14:58:31.807249 1 tasks_processing.go:74] worker 60 stopped. E0512 14:58:31.807267 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0512 14:58:31.807273 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0512 14:58:31.807276 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0512 14:58:31.807289 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0512 14:58:31.807320 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0512 14:58:31.807327 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0512 14:58:31.807331 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0512 14:58:31.807335 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0512 14:58:31.807375 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0512 14:58:31.807382 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0512 14:58:31.807388 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 122.49789ms to process 7 records I0512 14:58:31.831730 1 base_controller.go:82] Caches are synced for LoggingSyncer I0512 14:58:31.831746 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0512 14:58:31.839944 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0512 14:58:31.850279 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0512 14:58:32.128446 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0512 14:58:32.128463 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0512 14:58:32.130405 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-68kmt pod in namespace openshift-dns (previous: false). I0512 14:58:32.368299 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-68kmt pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-68kmt\" is waiting to start: ContainerCreating" I0512 14:58:32.368320 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-68kmt\" is waiting to start: ContainerCreating" I0512 14:58:32.368328 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-68kmt pod in namespace openshift-dns (previous: false). I0512 14:58:32.535359 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-68kmt pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-68kmt\" is waiting to start: ContainerCreating" I0512 14:58:32.535382 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-68kmt\" is waiting to start: ContainerCreating" I0512 14:58:32.535415 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-w9wgh pod in namespace openshift-dns (previous: false). W0512 14:58:32.735523 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 14:58:32.753761 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-w9wgh pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-w9wgh\" is waiting to start: ContainerCreating" I0512 14:58:32.753783 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-w9wgh\" is waiting to start: ContainerCreating" I0512 14:58:32.753793 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-w9wgh pod in namespace openshift-dns (previous: false). I0512 14:58:32.779293 1 tasks_processing.go:74] worker 19 stopped. I0512 14:58:32.779335 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0512 14:58:32.779349 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 1.096077451s to process 1 records I0512 14:58:32.933850 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-w9wgh pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-w9wgh\" is waiting to start: ContainerCreating" I0512 14:58:32.933869 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-w9wgh\" is waiting to start: ContainerCreating" I0512 14:58:32.933900 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-wszdb pod in namespace openshift-dns (previous: false). I0512 14:58:33.168427 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wszdb pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-wszdb\" is waiting to start: ContainerCreating" I0512 14:58:33.168448 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-wszdb\" is waiting to start: ContainerCreating" I0512 14:58:33.168459 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-wszdb pod in namespace openshift-dns (previous: false). I0512 14:58:33.204781 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0512 14:58:33.334967 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wszdb pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-wszdb\" is waiting to start: ContainerCreating" I0512 14:58:33.334986 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-wszdb\" is waiting to start: ContainerCreating" I0512 14:58:33.334997 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-mgwf8 pod in namespace openshift-dns (previous: false). I0512 14:58:33.535149 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 14:58:33.535174 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-tps7t pod in namespace openshift-dns (previous: false). I0512 14:58:33.602407 1 tasks_processing.go:74] worker 46 stopped. I0512 14:58:33.602465 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=3b6514bc0604c10f70083557afb2727b7d3fddf7dd41184242bf8d1a75a7d28c I0512 14:58:33.602500 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=be121c18cd10268b4a34a2fda362c042042c2b4ea49c56a5dd201654c2bcfa2d I0512 14:58:33.602527 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0512 14:58:33.602555 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=f22028c2813abf754f936f8e597c66fa9477a574d94063d17e868cc41b9eb71b I0512 14:58:33.602572 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0512 14:58:33.602611 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=5f25b9aef2d3e79e65100564973bdab2b5e41847c91e31d40a20fe32dc6ab896 I0512 14:58:33.602643 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=48f7846d2f3d929d9d075e751ea83271d79ec50ab152e86d078bca20cdc156d4 I0512 14:58:33.602667 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=1ab7438c053a158165ee0dbb335863337c7ef0e4b129c0fd3ef421c26a126a5e I0512 14:58:33.602681 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=541c8ba8df2901a45259199e6784daff22caff3a61a0307bf84349f15fd47991 I0512 14:58:33.602697 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=612e17fb1cf9a843ffb1947bade3ffefee5adb635fea4b48ffd75f1d8a1263cd I0512 14:58:33.602706 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0512 14:58:33.602723 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=f10b5f125bc29696e35668e5a8ee27c631f6d5486f25bd1c391231d5103376c5 I0512 14:58:33.602735 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0512 14:58:33.602750 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=31bbb9fc93c99e5a8e985279f4c450fcab0e505819e6102cd2fb18d63928abac I0512 14:58:33.602760 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0512 14:58:33.602780 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=ef0c8fcf018209f406280d64c72120be5c051e74ad372717f5233fe1135f20ca I0512 14:58:33.602789 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0512 14:58:33.602805 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=75cd221e977f0fc3d2feb0cf352088ec71aff2850d9be055d8768188f766731f I0512 14:58:33.602929 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=1982bfd2362cb2f79e36a75a9c5e8c8d0e5b6f2dfa28315ed5cc191768847347 I0512 14:58:33.602940 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0512 14:58:33.602947 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0512 14:58:33.602968 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0512 14:58:33.602989 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=54ef7abbc1ad9ec6435291da6c1976962214c49fe9140c0741a1708b5d0810af I0512 14:58:33.603013 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=81a208279fd0ad1c71b1ad5b99d04202203cf46f394674d28b09fbe06664f364 I0512 14:58:33.603023 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0512 14:58:33.603038 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=cad5da9ee81530aba59bbcbcfee60aabc6dfee4513ef5bbe3d395401a5b8b415 I0512 14:58:33.603048 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0512 14:58:33.603061 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=c0d6b3b9a8934d54cb74664f48e3783b68b4179ee1de539b956e5d54e87b4e82 I0512 14:58:33.603075 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=98f6db67eefe828f3ec312ba3728352b8025524792596f69ee1411f96628c5a6 I0512 14:58:33.603089 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=57b1d84cce1604f2091a9e42de13ef4d78ec8ae33345c6e02c6488f7c655ace5 I0512 14:58:33.603104 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=f3c2ca09233dddd7aa02c93beb26cc51f5814d41d3582230581e26601022c074 I0512 14:58:33.603117 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=b0f4fe0484803338997275f85efafe663fd06add3ec259ad9542f8acf48a42e5 I0512 14:58:33.603144 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=3b5d423e52c0a8cf7288a9af7c75ef2bf26ea824c966437aca48437f313b2f0b I0512 14:58:33.603160 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0512 14:58:33.603168 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0512 14:58:33.603175 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.916794729s to process 35 records I0512 14:58:33.733465 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 14:58:33.733488 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-vhtlb pod in namespace openshift-dns (previous: false). W0512 14:58:33.735769 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 14:58:33.938377 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 14:58:33.938435 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-67b7644654-cx4fm pod in namespace openshift-image-registry (previous: false). I0512 14:58:34.135232 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-67b7644654-cx4fm pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-67b7644654-cx4fm\" is waiting to start: ContainerCreating" I0512 14:58:34.135252 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-67b7644654-cx4fm\" is waiting to start: ContainerCreating" I0512 14:58:34.135300 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7cf666d69c-dcxhz pod in namespace openshift-image-registry (previous: false). I0512 14:58:34.333479 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7cf666d69c-dcxhz pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7cf666d69c-dcxhz\" is waiting to start: ContainerCreating" I0512 14:58:34.333499 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7cf666d69c-dcxhz\" is waiting to start: ContainerCreating" I0512 14:58:34.333548 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7cf666d69c-gnwfw pod in namespace openshift-image-registry (previous: false). I0512 14:58:34.535427 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7cf666d69c-gnwfw pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7cf666d69c-gnwfw\" is waiting to start: ContainerCreating" I0512 14:58:34.535447 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7cf666d69c-gnwfw\" is waiting to start: ContainerCreating" I0512 14:58:34.535459 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-684g5 pod in namespace openshift-image-registry (previous: false). I0512 14:58:34.735114 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 14:58:34.735134 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-nhszw pod in namespace openshift-image-registry (previous: false). W0512 14:58:34.735354 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 14:58:34.937989 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 14:58:34.938010 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-rtbvt pod in namespace openshift-image-registry (previous: false). I0512 14:58:35.132880 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 14:58:35.132903 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6dfcf7fc45-v8rnm pod in namespace openshift-ingress (previous: false). I0512 14:58:35.345129 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6dfcf7fc45-v8rnm pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6dfcf7fc45-v8rnm\" is waiting to start: ContainerCreating" I0512 14:58:35.345151 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6dfcf7fc45-v8rnm\" is waiting to start: ContainerCreating" I0512 14:58:35.345171 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7685f464d4-kt289 pod in namespace openshift-ingress (previous: false). I0512 14:58:35.536342 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7685f464d4-kt289 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7685f464d4-kt289\" is waiting to start: ContainerCreating" I0512 14:58:35.536365 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7685f464d4-kt289\" is waiting to start: ContainerCreating" I0512 14:58:35.536380 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7685f464d4-lxmwk pod in namespace openshift-ingress (previous: false). W0512 14:58:35.735166 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 14:58:35.735861 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7685f464d4-lxmwk pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7685f464d4-lxmwk\" is waiting to start: ContainerCreating" I0512 14:58:35.735876 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7685f464d4-lxmwk\" is waiting to start: ContainerCreating" I0512 14:58:35.735903 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-gkgzh pod in namespace openshift-ingress-canary (previous: false). I0512 14:58:35.934318 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-gkgzh pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-gkgzh\" is waiting to start: ContainerCreating" I0512 14:58:35.934338 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-gkgzh\" is waiting to start: ContainerCreating" I0512 14:58:35.934367 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-hvgm6 pod in namespace openshift-ingress-canary (previous: false). I0512 14:58:36.136817 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-hvgm6 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-hvgm6\" is waiting to start: ContainerCreating" I0512 14:58:36.136837 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-hvgm6\" is waiting to start: ContainerCreating" I0512 14:58:36.136871 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-lmf7j pod in namespace openshift-ingress-canary (previous: false). I0512 14:58:36.332883 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-lmf7j pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-lmf7j\" is waiting to start: ContainerCreating" I0512 14:58:36.332905 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-lmf7j\" is waiting to start: ContainerCreating" I0512 14:58:36.332929 1 tasks_processing.go:74] worker 3 stopped. I0512 14:58:36.333031 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=bb0a9678894f1680694f3df0753431f8d099146423aa127049d093df0ef6c544 I0512 14:58:36.333097 1 recorder.go:75] Recording events/openshift-dns with fingerprint=d5d42b0b960f3a44166a7301780dca3f3bdfbee44a1d6af3fa1efa930945c317 I0512 14:58:36.333261 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=28e592c051f64108776adde11a74c31ef997ca2b4de7b59c81764bb53a8ec03d I0512 14:58:36.333307 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=80e66be0378940e5ed4456a60fb12791c7ec5ca5588cfc28c8b2fc73694622e7 I0512 14:58:36.333370 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=e4cce1524ea6ea2c935db91f8816bda1d5574781dc05b00938a96528b289f4cb I0512 14:58:36.333401 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=4307bbb40d7bb62a8531e2b5a331aac7eeedf5173b87db324e6d40aa53fbb354 I0512 14:58:36.334741 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-68kmt with fingerprint=8b1ffbdec575035f85d56357fb910d3b9f4f2cc2fb6bf089afc2fba5dc53c0a3 I0512 14:58:36.334860 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-w9wgh with fingerprint=63f809026702cde71c95f760c04be2e174f50c4c97230346c243f0e92e19efe0 I0512 14:58:36.334953 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-wszdb with fingerprint=bc970fde731c58de7533d0fa2e8a33c28fe8594ed33724735487f96369bb5318 I0512 14:58:36.335075 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-67b7644654-cx4fm with fingerprint=8ad5250c2621cec2dcb4d2d55352cd32d2797ad1b34456c6c35c88a5c92f1cdc I0512 14:58:36.335185 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-7cf666d69c-dcxhz with fingerprint=a4e11c2f02fb5d2984a4e02df7c605dec3d5ccce1a0f4a982341886b60470385 I0512 14:58:36.335309 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-7cf666d69c-gnwfw with fingerprint=49c6db8ca385b278575da2518bc39adc76c43027c10921faeb4882a63729f901 I0512 14:58:36.335385 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-gkgzh with fingerprint=cd0421451c14991a4701b00b7413135fe3adf8d94287cfa4e428ae8ebcc7e266 I0512 14:58:36.335455 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-hvgm6 with fingerprint=08beda64c03dcdc124e1e6df2ee3c629491521e04643a008b6658748dfc36367 I0512 14:58:36.335526 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-lmf7j with fingerprint=b7d54fb7a64d4116a69491120fe582dcd9d3cadb8fb9021c1dd6e6c3346dc6f7 I0512 14:58:36.335541 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.65006939s to process 15 records W0512 14:58:36.735379 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0512 14:58:36.735408 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 14:58:36.735424 1 tasks_processing.go:74] worker 1 stopped. E0512 14:58:36.735433 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 14:58:36.735444 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0512 14:58:36.735457 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0512 14:58:36.735467 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.048215178s to process 1 records I0512 14:58:44.015442 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 14:58:44.566084 1 tasks_processing.go:74] worker 25 stopped. I0512 14:58:44.566129 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0512 14:58:44.566141 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.881146311s to process 1 records I0512 14:58:45.294335 1 tasks_processing.go:74] worker 7 stopped. I0512 14:58:45.294609 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=c5e0ad81555672a1ae6992800f0e8a9ad0a7406d6b5e671cb8278224bb66f9c4 I0512 14:58:45.294625 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.608831266s to process 1 records E0512 14:58:45.294702 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.612s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0512 14:58:45.295813 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0512 14:58:45.295828 1 periodic.go:209] Running workloads gatherer I0512 14:58:45.295843 1 tasks_processing.go:45] number of workers: 2 I0512 14:58:45.295853 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 14:58:45.295858 1 tasks_processing.go:71] worker 1 working on workload_info task. I0512 14:58:45.295865 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 14:58:45.295887 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0512 14:58:45.322628 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0512 14:58:45.332134 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0512 14:58:45.333616 1 tasks_processing.go:74] worker 0 stopped. I0512 14:58:45.333631 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 37.705104ms to process 0 records I0512 14:58:45.341889 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0512 14:58:45.351541 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0512 14:58:45.361801 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (10ms) I0512 14:58:45.374473 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (13ms) I0512 14:58:45.384153 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (10ms) I0512 14:58:45.393885 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (10ms) I0512 14:58:45.407091 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (13ms) I0512 14:58:45.417056 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (10ms) I0512 14:58:45.426370 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (9ms) I0512 14:58:45.435439 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (9ms) I0512 14:58:45.532732 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (97ms) I0512 14:58:45.632120 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (99ms) I0512 14:58:45.733778 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (102ms) I0512 14:58:45.832447 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (99ms) I0512 14:58:45.933668 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (101ms) I0512 14:58:46.034616 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (101ms) I0512 14:58:46.133988 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (99ms) I0512 14:58:46.232942 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (99ms) I0512 14:58:46.332662 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0512 14:58:46.434331 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (102ms) I0512 14:58:46.533182 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0512 14:58:46.633796 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (101ms) I0512 14:58:46.733374 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0512 14:58:46.832869 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0512 14:58:46.933155 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0512 14:58:47.032871 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0512 14:58:47.132443 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0512 14:58:47.233411 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (101ms) I0512 14:58:47.333624 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0512 14:58:47.433825 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0512 14:58:47.467542 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 14:58:47.534363 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (101ms) I0512 14:58:47.534397 1 tasks_processing.go:74] worker 1 stopped. E0512 14:58:47.534408 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0512 14:58:47.534740 1 recorder.go:75] Recording config/workload_info with fingerprint=830bebc068bbf749bcf03fd4dadee82225290e5efb46b266e77248ee73f13712 I0512 14:58:47.534755 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.238532545s to process 1 records E0512 14:58:47.534780 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.238s with: function \"workload_info\" failed with an error" I0512 14:58:47.535888 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0512 14:58:47.535926 1 periodic.go:209] Running conditional gatherer I0512 14:58:47.544771 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0512 14:58:47.552963 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.13:58308->172.30.0.10:53: read: connection refused E0512 14:58:47.553248 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 14:58:47.553314 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0512 14:58:47.560840 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0512 14:58:47.560860 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560866 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560870 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560873 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560876 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560879 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560882 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560885 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 14:58:47.560888 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0512 14:58:47.560904 1 tasks_processing.go:45] number of workers: 3 I0512 14:58:47.560917 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 14:58:47.560921 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0512 14:58:47.560930 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 14:58:47.560944 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 14:58:47.560965 1 tasks_processing.go:74] worker 1 stopped. I0512 14:58:47.560980 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0512 14:58:47.560982 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0512 14:58:47.561008 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0512 14:58:47.561021 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 862ns to process 1 records I0512 14:58:47.561052 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0512 14:58:47.561059 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.599µs to process 1 records I0512 14:58:47.561064 1 tasks_processing.go:74] worker 0 stopped. I0512 14:58:47.561286 1 tasks_processing.go:74] worker 2 stopped. I0512 14:58:47.561300 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 287.706µs to process 0 records I0512 14:58:47.561327 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.13:58308->172.30.0.10:53: read: connection refused I0512 14:58:47.561347 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0512 14:58:47.587506 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=9e210bce1460120975cf8dbdf25b9229ad8c2855a8ff68b47ea5bbd2d74ef74b I0512 14:58:47.587645 1 diskrecorder.go:70] Writing 109 records to /var/lib/insights-operator/insights-2026-05-12-145847.tar.gz I0512 14:58:47.594259 1 diskrecorder.go:51] Wrote 109 records to disk in 6ms I0512 14:58:47.594301 1 periodic.go:278] Gathering cluster info every 2h0m0s I0512 14:58:47.594318 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0512 14:58:47.666613 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 14:58:57.740397 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 14:59:36.338371 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="fcaebe44228e2f559b38a0277c85e75dc5922cb5110ff81175d6fc864cf52d27") W0512 14:59:36.338403 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0512 14:59:36.338452 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="fed7b72a001d05c9e8fc4793ed3ec1fdb436c625b6db42a1b70a5374c890c00d") I0512 14:59:36.338453 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0512 14:59:36.338482 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0512 14:59:36.338528 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0512 14:59:36.338539 1 periodic.go:170] Shutting down I0512 14:59:36.338549 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="29193a5f2462d0b5cd816b45454133d9431701aaa1f4513f4701ada22b000a45") I0512 14:59:36.338559 1 base_controller.go:181] Shutting down LoggingSyncer ... I0512 14:59:36.338552 1 base_controller.go:181] Shutting down ConfigController ... I0512 14:59:36.338564 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" I0512 14:59:36.338586 1 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0512 14:59:36.338588 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0512 14:59:36.338594 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0512 14:59:36.338597 1 base_controller.go:113] All LoggingSyncer workers have been terminated I0512 14:59:36.338602 1 base_controller.go:113] All ConfigController workers have been terminated I0512 14:59:36.338600 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0512 14:59:36.338625 1 requestheader_controller.go:194] Shutting down RequestHeaderAuthRequestController I0512 14:59:36.338601 1 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/tmp/serving-cert-642830154/tls.crt::/tmp/serving-cert-642830154/tls.key" I0512 14:59:36.338630 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"