W0506 20:29:54.410464 1 cmd.go:257] Using insecure, self-signed certificates I0506 20:29:55.222534 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0506 20:29:55.222827 1 observer_polling.go:159] Starting file observer I0506 20:29:55.924379 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0506 20:29:55.924597 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0506 20:29:55.925115 1 secure_serving.go:57] Forcing use of http/1.1 only W0506 20:29:55.925137 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0506 20:29:55.925142 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0506 20:29:55.925145 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0506 20:29:55.925148 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0506 20:29:55.925150 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0506 20:29:55.925153 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0506 20:29:55.925176 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0506 20:29:55.930417 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0506 20:29:55.930437 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"f5bf20ec-e03f-40c2-9c8b-33aed28c3883", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0506 20:29:55.931908 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0506 20:29:55.931923 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0506 20:29:55.931919 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0506 20:29:55.931924 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0506 20:29:55.931934 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0506 20:29:55.931937 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0506 20:29:55.932161 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1424096744/tls.crt::/tmp/serving-cert-1424096744/tls.key" I0506 20:29:55.932431 1 secure_serving.go:213] Serving securely on [::]:8443 I0506 20:29:55.932751 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0506 20:29:55.937535 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0506 20:29:55.937562 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0506 20:29:55.937662 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0506 20:29:55.945237 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 20:29:55.945276 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0506 20:29:55.951173 1 secretconfigobserver.go:119] support secret does not exist I0506 20:29:55.956948 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 20:29:55.962631 1 secretconfigobserver.go:119] support secret does not exist I0506 20:29:55.965044 1 recorder.go:161] Pruning old reports every 4h13m13s, max age is 288h0m0s I0506 20:29:55.971253 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0506 20:29:55.971269 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0506 20:29:55.971273 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0506 20:29:55.971281 1 insightsreport.go:296] Starting report retriever I0506 20:29:55.971287 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0506 20:29:55.971295 1 periodic.go:209] Running clusterconfig gatherer I0506 20:29:55.971342 1 tasks_processing.go:45] number of workers: 64 I0506 20:29:55.971379 1 tasks_processing.go:69] worker 16 listening for tasks. I0506 20:29:55.971378 1 tasks_processing.go:69] worker 4 listening for tasks. I0506 20:29:55.971389 1 tasks_processing.go:69] worker 5 listening for tasks. I0506 20:29:55.971390 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 20:29:55.971395 1 tasks_processing.go:69] worker 6 listening for tasks. I0506 20:29:55.971400 1 tasks_processing.go:69] worker 11 listening for tasks. I0506 20:29:55.971400 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 20:29:55.971389 1 tasks_processing.go:71] worker 16 working on certificate_signing_requests task. I0506 20:29:55.971403 1 tasks_processing.go:69] worker 12 listening for tasks. I0506 20:29:55.971411 1 tasks_processing.go:69] worker 3 listening for tasks. I0506 20:29:55.971413 1 tasks_processing.go:71] worker 12 working on schedulers task. I0506 20:29:55.971417 1 tasks_processing.go:69] worker 14 listening for tasks. I0506 20:29:55.971417 1 tasks_processing.go:71] worker 5 working on tsdb_status task. I0506 20:29:55.971420 1 tasks_processing.go:69] worker 9 listening for tasks. I0506 20:29:55.971420 1 tasks_processing.go:69] worker 2 listening for tasks. I0506 20:29:55.971423 1 tasks_processing.go:69] worker 13 listening for tasks. I0506 20:29:55.971429 1 tasks_processing.go:69] worker 10 listening for tasks. I0506 20:29:55.971432 1 tasks_processing.go:71] worker 1 working on version task. I0506 20:29:55.971431 1 tasks_processing.go:71] worker 6 working on install_plans task. I0506 20:29:55.971439 1 tasks_processing.go:69] worker 52 listening for tasks. I0506 20:29:55.971435 1 tasks_processing.go:69] worker 40 listening for tasks. I0506 20:29:55.971447 1 tasks_processing.go:69] worker 41 listening for tasks. I0506 20:29:55.971453 1 tasks_processing.go:69] worker 29 listening for tasks. I0506 20:29:55.971455 1 tasks_processing.go:69] worker 42 listening for tasks. W0506 20:29:55.971452 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0506 20:29:55.971461 1 tasks_processing.go:69] worker 30 listening for tasks. I0506 20:29:55.971463 1 tasks_processing.go:69] worker 43 listening for tasks. I0506 20:29:55.971458 1 tasks_processing.go:69] worker 28 listening for tasks. I0506 20:29:55.971469 1 tasks_processing.go:69] worker 36 listening for tasks. I0506 20:29:55.971471 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 38.327µs to process 0 records I0506 20:29:55.971474 1 tasks_processing.go:69] worker 17 listening for tasks. I0506 20:29:55.971473 1 tasks_processing.go:69] worker 44 listening for tasks. I0506 20:29:55.971480 1 tasks_processing.go:69] worker 45 listening for tasks. I0506 20:29:55.971478 1 tasks_processing.go:69] worker 37 listening for tasks. I0506 20:29:55.971480 1 tasks_processing.go:69] worker 35 listening for tasks. I0506 20:29:55.971490 1 tasks_processing.go:69] worker 39 listening for tasks. I0506 20:29:55.971491 1 tasks_processing.go:69] worker 19 listening for tasks. I0506 20:29:55.971484 1 tasks_processing.go:69] worker 38 listening for tasks. I0506 20:29:55.971497 1 tasks_processing.go:69] worker 23 listening for tasks. I0506 20:29:55.971484 1 tasks_processing.go:69] worker 18 listening for tasks. I0506 20:29:55.971485 1 tasks_processing.go:71] worker 11 working on storage_classes task. I0506 20:29:55.971504 1 tasks_processing.go:69] worker 50 listening for tasks. I0506 20:29:55.971508 1 tasks_processing.go:69] worker 20 listening for tasks. I0506 20:29:55.971504 1 tasks_processing.go:69] worker 48 listening for tasks. I0506 20:29:55.971515 1 tasks_processing.go:69] worker 26 listening for tasks. I0506 20:29:55.971501 1 tasks_processing.go:69] worker 32 listening for tasks. I0506 20:29:55.971522 1 tasks_processing.go:69] worker 51 listening for tasks. I0506 20:29:55.971525 1 tasks_processing.go:69] worker 46 listening for tasks. I0506 20:29:55.971527 1 tasks_processing.go:69] worker 22 listening for tasks. I0506 20:29:55.971529 1 tasks_processing.go:69] worker 27 listening for tasks. I0506 20:29:55.971531 1 tasks_processing.go:69] worker 61 listening for tasks. I0506 20:29:55.971502 1 tasks_processing.go:69] worker 24 listening for tasks. I0506 20:29:55.971536 1 tasks_processing.go:69] worker 47 listening for tasks. I0506 20:29:55.971538 1 tasks_processing.go:69] worker 59 listening for tasks. I0506 20:29:55.971540 1 tasks_processing.go:69] worker 60 listening for tasks. I0506 20:29:55.971543 1 tasks_processing.go:69] worker 55 listening for tasks. I0506 20:29:55.971409 1 tasks_processing.go:69] worker 7 listening for tasks. I0506 20:29:55.971546 1 tasks_processing.go:69] worker 56 listening for tasks. I0506 20:29:55.971543 1 tasks_processing.go:69] worker 58 listening for tasks. I0506 20:29:55.971552 1 tasks_processing.go:69] worker 54 listening for tasks. I0506 20:29:55.971555 1 tasks_processing.go:71] worker 43 working on cost_management_metrics_configs task. I0506 20:29:55.971551 1 tasks_processing.go:69] worker 53 listening for tasks. I0506 20:29:55.971557 1 tasks_processing.go:71] worker 42 working on container_images task. I0506 20:29:55.971564 1 tasks_processing.go:71] worker 29 working on sap_datahubs task. I0506 20:29:55.971566 1 tasks_processing.go:71] worker 53 working on metrics task. I0506 20:29:55.971568 1 tasks_processing.go:69] worker 63 listening for tasks. I0506 20:29:55.971572 1 tasks_processing.go:71] worker 2 working on active_alerts task. I0506 20:29:55.971569 1 tasks_processing.go:71] worker 54 working on sap_config task. I0506 20:29:55.971580 1 tasks_processing.go:71] worker 32 working on operators_pods_and_events task. I0506 20:29:55.971584 1 tasks_processing.go:71] worker 48 working on nodes task. I0506 20:29:55.971581 1 tasks_processing.go:69] worker 57 listening for tasks. W0506 20:29:55.971597 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory W0506 20:29:55.971602 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0506 20:29:55.971413 1 tasks_processing.go:71] worker 0 working on storage_cluster task. I0506 20:29:55.971609 1 tasks_processing.go:71] worker 10 working on image task. I0506 20:29:55.971620 1 tasks_processing.go:71] worker 53 working on image_pruners task. I0506 20:29:55.971639 1 tasks_processing.go:71] worker 13 working on jaegers task. I0506 20:29:55.971508 1 tasks_processing.go:69] worker 33 listening for tasks. I0506 20:29:55.971657 1 tasks_processing.go:71] worker 33 working on nodenetworkconfigurationpolicies task. I0506 20:29:55.971663 1 tasks_processing.go:71] worker 40 working on clusterroles task. I0506 20:29:55.971518 1 tasks_processing.go:69] worker 34 listening for tasks. I0506 20:29:55.971508 1 tasks_processing.go:69] worker 25 listening for tasks. I0506 20:29:55.971527 1 tasks_processing.go:69] worker 15 listening for tasks. I0506 20:29:55.971552 1 tasks_processing.go:71] worker 41 working on machine_config_pools task. I0506 20:29:55.971774 1 tasks_processing.go:71] worker 5 working on lokistack task. I0506 20:29:55.971855 1 tasks_processing.go:71] worker 24 working on oauths task. I0506 20:29:55.971908 1 tasks_processing.go:71] worker 36 working on machine_configs task. I0506 20:29:55.971991 1 tasks_processing.go:71] worker 30 working on validating_webhook_configurations task. I0506 20:29:55.971520 1 tasks_processing.go:69] worker 21 listening for tasks. I0506 20:29:55.971566 1 tasks_processing.go:71] worker 9 working on openstack_dataplanedeployments task. I0506 20:29:55.972890 1 tasks_processing.go:71] worker 25 working on crds task. I0506 20:29:55.971391 1 tasks_processing.go:71] worker 4 working on container_runtime_configs task. I0506 20:29:55.972917 1 tasks_processing.go:71] worker 34 working on openstack_version task. I0506 20:29:55.971413 1 tasks_processing.go:69] worker 8 listening for tasks. I0506 20:29:55.973321 1 tasks_processing.go:71] worker 8 working on node_logs task. I0506 20:29:55.971823 1 tasks_processing.go:71] worker 63 working on openshift_machine_api_events task. I0506 20:29:55.971562 1 tasks_processing.go:71] worker 14 working on aggregated_monitoring_cr_names task. I0506 20:29:55.971574 1 tasks_processing.go:71] worker 26 working on pod_network_connectivity_checks task. I0506 20:29:55.971611 1 tasks_processing.go:71] worker 52 working on machines task. I0506 20:29:55.971613 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 35.065µs to process 0 records I0506 20:29:55.971830 1 tasks_processing.go:71] worker 47 working on cluster_apiserver task. I0506 20:29:55.971834 1 tasks_processing.go:71] worker 51 working on service_accounts task. I0506 20:29:55.971838 1 tasks_processing.go:71] worker 46 working on networks task. I0506 20:29:55.971842 1 tasks_processing.go:71] worker 22 working on overlapping_namespace_uids task. I0506 20:29:55.971846 1 tasks_processing.go:71] worker 27 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0506 20:29:55.971851 1 tasks_processing.go:71] worker 61 working on image_registries task. I0506 20:29:55.971882 1 tasks_processing.go:71] worker 57 working on proxies task. I0506 20:29:55.971895 1 tasks_processing.go:71] worker 35 working on qemu_kubevirt_launcher_logs task. I0506 20:29:55.971902 1 tasks_processing.go:71] worker 28 working on machine_healthchecks task. I0506 20:29:55.974062 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 30.824µs to process 0 records I0506 20:29:55.971926 1 tasks_processing.go:71] worker 23 working on operators task. I0506 20:29:55.971930 1 tasks_processing.go:71] worker 39 working on openstack_controlplanes task. I0506 20:29:55.971936 1 tasks_processing.go:71] worker 19 working on machine_autoscalers task. I0506 20:29:55.971547 1 tasks_processing.go:69] worker 62 listening for tasks. I0506 20:29:55.971945 1 tasks_processing.go:71] worker 44 working on silenced_alerts task. I0506 20:29:55.971939 1 tasks_processing.go:71] worker 38 working on monitoring_persistent_volumes task. I0506 20:29:55.971952 1 tasks_processing.go:71] worker 50 working on nodenetworkstates task. I0506 20:29:55.971951 1 tasks_processing.go:71] worker 17 working on support_secret task. I0506 20:29:55.971955 1 tasks_processing.go:71] worker 18 working on feature_gates task. I0506 20:29:55.971963 1 tasks_processing.go:71] worker 45 working on openshift_logging task. I0506 20:29:55.971965 1 tasks_processing.go:71] worker 37 working on machine_sets task. I0506 20:29:55.971963 1 tasks_processing.go:71] worker 20 working on olm_operators task. I0506 20:29:55.971974 1 tasks_processing.go:71] worker 55 working on ceph_cluster task. I0506 20:29:55.971976 1 tasks_processing.go:71] worker 56 working on pdbs task. I0506 20:29:55.971979 1 tasks_processing.go:71] worker 59 working on ingress_certificates task. I0506 20:29:55.971980 1 tasks_processing.go:71] worker 7 working on sap_pods task. I0506 20:29:55.971979 1 tasks_processing.go:71] worker 58 working on config_maps task. I0506 20:29:55.971983 1 tasks_processing.go:71] worker 60 working on ingress task. I0506 20:29:55.971497 1 tasks_processing.go:69] worker 49 listening for tasks. I0506 20:29:55.971491 1 tasks_processing.go:69] worker 31 listening for tasks. I0506 20:29:55.972925 1 tasks_processing.go:71] worker 15 working on infrastructures task. I0506 20:29:55.973001 1 tasks_processing.go:71] worker 21 working on authentication task. I0506 20:29:55.971557 1 tasks_processing.go:71] worker 3 working on mutating_webhook_configurations task. I0506 20:29:55.974606 1 tasks_processing.go:71] worker 2 working on dvo_metrics task. I0506 20:29:55.974970 1 tasks_processing.go:71] worker 62 working on openstack_dataplanenodesets task. W0506 20:29:55.974999 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0506 20:29:55.975010 1 tasks_processing.go:74] worker 44 stopped. I0506 20:29:55.975021 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 862.03µs to process 0 records I0506 20:29:55.975461 1 tasks_processing.go:74] worker 49 stopped. I0506 20:29:55.975475 1 tasks_processing.go:74] worker 31 stopped. I0506 20:29:55.976382 1 tasks_processing.go:74] worker 29 stopped. I0506 20:29:55.976395 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 4.806042ms to process 0 records I0506 20:29:55.976403 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 4.726571ms to process 0 records I0506 20:29:55.976407 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 4.817544ms to process 0 records I0506 20:29:55.976412 1 tasks_processing.go:74] worker 43 stopped. I0506 20:29:55.976415 1 tasks_processing.go:74] worker 13 stopped. I0506 20:29:55.978016 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0506 20:29:55.978032 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0506 20:29:55.978037 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0506 20:29:55.978040 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0506 20:29:55.978053 1 controller.go:489] The operator is still being initialized I0506 20:29:55.978060 1 controller.go:512] The operator is healthy I0506 20:29:55.979442 1 tasks_processing.go:74] worker 5 stopped. I0506 20:29:55.979454 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 7.578199ms to process 0 records I0506 20:29:55.979849 1 tasks_processing.go:74] worker 12 stopped. I0506 20:29:55.979924 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=04fec976503025b944cd29b58a5c40c2224b35c4356c97877630544deacc7972 I0506 20:29:55.979935 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 8.423409ms to process 1 records I0506 20:29:55.980108 1 tasks_processing.go:74] worker 11 stopped. I0506 20:29:55.980177 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=986091ab19f2722deb6c71dc59b8dabd9bb5d2479aa02982e02f00375d44c719 I0506 20:29:55.980192 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=45452c3b4458cc2936b828899fefb6639ffea8a0af4d3986e5bffe412206207f I0506 20:29:55.980200 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 8.585405ms to process 2 records I0506 20:29:55.981106 1 tasks_processing.go:74] worker 0 stopped. I0506 20:29:55.981118 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 9.486725ms to process 0 records I0506 20:29:55.981545 1 tasks_processing.go:74] worker 10 stopped. I0506 20:29:55.981627 1 recorder.go:75] Recording config/image with fingerprint=334619ebf22f14e964882fcf23c43893ff528847b2f0616bdf3e5fb00a7d9cd4 I0506 20:29:55.981638 1 gather.go:177] gatherer "clusterconfig" function "image" took 9.918405ms to process 1 records I0506 20:29:55.981922 1 tasks_processing.go:74] worker 54 stopped. I0506 20:29:55.981931 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 10.335169ms to process 0 records I0506 20:29:55.985269 1 tasks_processing.go:74] worker 9 stopped. I0506 20:29:55.985281 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 12.628196ms to process 0 records I0506 20:29:55.985529 1 tasks_processing.go:74] worker 33 stopped. I0506 20:29:55.985540 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 13.863541ms to process 0 records I0506 20:29:55.986985 1 tasks_processing.go:74] worker 34 stopped. I0506 20:29:55.986994 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 13.918802ms to process 0 records I0506 20:29:55.987342 1 tasks_processing.go:74] worker 53 stopped. I0506 20:29:55.987589 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=49d9065babd4f0433ef211e8c9fe67442c00fffeeadfea632e57d50e14631616 I0506 20:29:55.987605 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 15.708287ms to process 1 records I0506 20:29:55.989834 1 tasks_processing.go:74] worker 16 stopped. I0506 20:29:55.989847 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 18.415111ms to process 0 records I0506 20:29:55.990824 1 tasks_processing.go:74] worker 63 stopped. I0506 20:29:55.990834 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 17.451731ms to process 0 records I0506 20:29:55.991030 1 tasks_processing.go:74] worker 46 stopped. I0506 20:29:55.991152 1 recorder.go:75] Recording config/network with fingerprint=8a341ffb9a7b90a637289359f11603740a3367053f5ccef42daa4d88c72e1bab I0506 20:29:55.991163 1 gather.go:177] gatherer "clusterconfig" function "networks" took 17.008126ms to process 1 records I0506 20:29:55.991267 1 tasks_processing.go:74] worker 24 stopped. I0506 20:29:55.991346 1 recorder.go:75] Recording config/oauth with fingerprint=2f07c7be66bd2046678dc20fa10994f3fb57dd05584316d5b8046a16e6cbcb48 I0506 20:29:55.991354 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 19.232812ms to process 1 records I0506 20:29:55.996080 1 tasks_processing.go:74] worker 39 stopped. I0506 20:29:55.996095 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 21.940648ms to process 0 records I0506 20:29:55.996506 1 tasks_processing.go:74] worker 4 stopped. I0506 20:29:55.996519 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 23.48535ms to process 0 records I0506 20:29:55.996537 1 tasks_processing.go:74] worker 26 stopped. E0506 20:29:55.996549 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0506 20:29:55.996556 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 22.993465ms to process 0 records I0506 20:29:55.996760 1 tasks_processing.go:74] worker 28 stopped. E0506 20:29:55.996772 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0506 20:29:55.996782 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 22.6827ms to process 0 records I0506 20:29:55.996797 1 tasks_processing.go:74] worker 52 stopped. E0506 20:29:55.996809 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0506 20:29:55.996816 1 gather.go:177] gatherer "clusterconfig" function "machines" took 23.032802ms to process 0 records I0506 20:29:55.997007 1 tasks_processing.go:74] worker 41 stopped. I0506 20:29:55.997019 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 25.299091ms to process 0 records I0506 20:29:55.997570 1 gather_logs.go:145] no pods in namespace were found I0506 20:29:55.997576 1 tasks_processing.go:74] worker 19 stopped. I0506 20:29:55.997587 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 23.434002ms to process 0 records I0506 20:29:55.997595 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 23.520352ms to process 0 records I0506 20:29:55.997600 1 tasks_processing.go:74] worker 35 stopped. I0506 20:29:55.997652 1 tasks_processing.go:74] worker 57 stopped. I0506 20:29:55.997719 1 recorder.go:75] Recording config/proxy with fingerprint=d037e4136dcc007fb839d3237ba9ed4c85e9cbf3797dd626ab0d13e210491ef1 I0506 20:29:55.997736 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 23.588912ms to process 1 records I0506 20:29:55.997791 1 tasks_processing.go:74] worker 48 stopped. I0506 20:29:55.998120 1 recorder.go:75] Recording config/node/ip-10-0-0-169.ec2.internal with fingerprint=c217baa6321b730a0648711554656ebbd4e541b0e54505c47a06b7b1d2c3752b I0506 20:29:55.998201 1 recorder.go:75] Recording config/node/ip-10-0-1-182.ec2.internal with fingerprint=3454c50c8c894f9f108b938760b199758359e66c0798e3b4d13f58f93ee5596a I0506 20:29:55.998284 1 recorder.go:75] Recording config/node/ip-10-0-2-52.ec2.internal with fingerprint=136dd5b73ca4919d0ef54bc66e4f1d801683956e69667422dcfb3a5cf2dd1480 I0506 20:29:55.998298 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 26.18921ms to process 3 records I0506 20:29:55.998429 1 tasks_processing.go:74] worker 47 stopped. I0506 20:29:55.998455 1 recorder.go:75] Recording config/apiserver with fingerprint=36e3fab9528c2d8c56d27b2fa58b1ec9678406131b399f0c77d9fa099b2c53dc I0506 20:29:55.998476 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 23.960471ms to process 1 records I0506 20:29:56.007268 1 tasks_processing.go:74] worker 30 stopped. I0506 20:29:56.007434 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=7f9c4a8b70a979e42c15a54d7fa467f54464cafe1affa1c1e53ac1362522bcaa I0506 20:29:56.007536 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=65d5b10ecfe3beda6d98e4c551a1e5427ee176a24c9517ff22315e238a485d68 I0506 20:29:56.007574 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=bc3be26b49c82d8702fc3bd4833430ed1a6716bdc3ffd3519eb85cdcb82d6c3a I0506 20:29:56.007642 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=7def15f391692ce2f4b3b279a504e12cb91ead7414c1b15af3b2885e0d099929 I0506 20:29:56.007700 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=2ac797f15b19b32a836b1f57e83b780bf49509d95fc08d624712066ec0ca4b11 I0506 20:29:56.007751 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=f87582007682d98c1fcc97de862b2ffb1159afad644cb26688d5cbc418dbc2aa I0506 20:29:56.007794 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=02c04358f33e1ebd37714e72b976475029bcae9bd2e97f33aeb5413aa11557b4 I0506 20:29:56.007851 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=a1a99f92c331c25a186da0e66eac86b4deabcce371595fb37988ead3148dbc20 I0506 20:29:56.007891 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=5f573649bfc6c11bcb0edae766b61bb638432e3b53815bb30d88d87cbbb5da57 I0506 20:29:56.007930 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=596eecdce8b29e6d6955447d0b968ad4a0b61aa120c63bff9e7a0b80e15b5bd7 I0506 20:29:56.007973 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=5d0186d6bda495e8ac8b44578c8d3d493d6f327c99cdfa2a614168d371310c01 I0506 20:29:56.007985 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 35.214618ms to process 11 records I0506 20:29:56.008010 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 33.507181ms to process 0 records I0506 20:29:56.008022 1 tasks_processing.go:74] worker 7 stopped. I0506 20:29:56.008463 1 tasks_processing.go:74] worker 37 stopped. I0506 20:29:56.008475 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 34.278087ms to process 0 records I0506 20:29:56.008481 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 34.294497ms to process 0 records I0506 20:29:56.008486 1 tasks_processing.go:74] worker 55 stopped. I0506 20:29:56.008825 1 tasks_processing.go:74] worker 15 stopped. I0506 20:29:56.009295 1 recorder.go:75] Recording config/infrastructure with fingerprint=ab055a5dfaee326dfdc4cfd5a38b2bbb32b2a7d1ff6c5a79901b46debc1cb67d I0506 20:29:56.009308 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 34.615164ms to process 1 records I0506 20:29:56.010095 1 tasks_processing.go:74] worker 62 stopped. I0506 20:29:56.010111 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 35.105713ms to process 0 records I0506 20:29:56.010163 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 35.937559ms to process 0 records I0506 20:29:56.010171 1 tasks_processing.go:74] worker 45 stopped. I0506 20:29:56.010463 1 tasks_processing.go:74] worker 21 stopped. I0506 20:29:56.010702 1 recorder.go:75] Recording config/authentication with fingerprint=6abb3b9dcf37c6166048196699e9a1831a908d152a0cd34b71e7e5f1671575d9 I0506 20:29:56.010722 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 36.212015ms to process 1 records I0506 20:29:56.010737 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 36.563559ms to process 0 records I0506 20:29:56.010745 1 tasks_processing.go:74] worker 50 stopped. I0506 20:29:56.011318 1 tasks_processing.go:74] worker 8 stopped. I0506 20:29:56.011339 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 37.979023ms to process 0 records I0506 20:29:56.011687 1 tasks_processing.go:74] worker 1 stopped. I0506 20:29:56.012778 1 recorder.go:75] Recording config/version with fingerprint=ae4bc5fbcb899e1c242c836302773b5ed950d530cc5022888d76e5678b6d2da3 I0506 20:29:56.012844 1 recorder.go:75] Recording config/id with fingerprint=64dbd4a77b32c4a9decb47f40a0b395c61478422171f307f174f3e82d914a00d I0506 20:29:56.012878 1 gather.go:177] gatherer "clusterconfig" function "version" took 40.246113ms to process 2 records I0506 20:29:56.013015 1 tasks_processing.go:74] worker 25 stopped. I0506 20:29:56.014875 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=5623bc2b17e228077d0350728685fdb814ce099327edabf6f176d5f262b65cd8 I0506 20:29:56.015199 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=0f1614e4aa60afe5e237db3f8ef5b79c0077371855d88f3facbf146be30dd68a I0506 20:29:56.015219 1 gather.go:177] gatherer "clusterconfig" function "crds" took 38.811977ms to process 2 records I0506 20:29:56.015266 1 tasks_processing.go:74] worker 40 stopped. I0506 20:29:56.015406 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=151ed74f8180d4d8fe437aca31936bb3e17492ca588040f4385dcbec9aa5731e I0506 20:29:56.015491 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=5bcc61e3c9f0e1d41e12e4a6731e61e65a8cb80f66e1311ff529ace462aadf43 I0506 20:29:56.015499 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 40.251234ms to process 2 records I0506 20:29:56.015586 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=9bf8d1ac8bd58a1bf47421339c05cea9b9dcabfe68eb0dc22295b46128100d5f I0506 20:29:56.015597 1 tasks_processing.go:74] worker 3 stopped. I0506 20:29:56.015619 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=09b46a29e5130f55cf5d5cf42c75bd8b58fcab834853ce90ba2331a57c2436b1 I0506 20:29:56.015647 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=33131eff09b24f08f7ac0ca0c534e8b410fee98647f367d98fa432d5d9a5f151 I0506 20:29:56.015655 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 37.668352ms to process 3 records I0506 20:29:56.015702 1 recorder.go:75] Recording config/olm_operators with fingerprint=8c437fc88bb8e9b7a6d0ef562cbe554f3f9961957e1fb2eae5de1dcb78c4f702 I0506 20:29:56.015710 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 38.941176ms to process 1 records I0506 20:29:56.015731 1 tasks_processing.go:74] worker 20 stopped. I0506 20:29:56.015793 1 tasks_processing.go:74] worker 61 stopped. I0506 20:29:56.016049 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=786df48eeded20afe8f3e6f4d7eb459d2dfe9702a81e3736466273b21ed382e3 I0506 20:29:56.016058 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 40.04406ms to process 1 records I0506 20:29:56.016073 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0506 20:29:56.016077 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 40.946746ms to process 1 records I0506 20:29:56.016084 1 tasks_processing.go:74] worker 22 stopped. W0506 20:29:56.017166 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 20:29:56.019952 1 tasks_processing.go:74] worker 56 stopped. I0506 20:29:56.020003 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0506 20:29:56.020069 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=1d69d6d92173dab70d934e2bc45d7cb977227141cd9bf058615b8f027a5f227e I0506 20:29:56.020110 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=20ca92fe8ba3033c58f5a1a5f0533dc0a824fcdbd332e9533128ef5b8caa7ad6 I0506 20:29:56.020119 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0506 20:29:56.020143 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=d359d3afb72813f23305322e91d606be82486915d211350d5bffa15faff0adc7 W0506 20:29:56.020152 1 operator.go:288] started I0506 20:29:56.020152 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 45.763799ms to process 3 records I0506 20:29:56.020162 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 45.830583ms to process 0 records I0506 20:29:56.020171 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0506 20:29:56.020192 1 tasks_processing.go:74] worker 38 stopped. I0506 20:29:56.020300 1 tasks_processing.go:74] worker 18 stopped. I0506 20:29:56.020328 1 recorder.go:75] Recording config/featuregate with fingerprint=9b3fd2d84f1b59a4eba87a1efcf49fed26d27f79507d2c4e7676e0206187fd69 I0506 20:29:56.020383 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 45.820362ms to process 1 records E0506 20:29:56.020395 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0506 20:29:56.020404 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 45.827596ms to process 0 records I0506 20:29:56.020423 1 tasks_processing.go:74] worker 17 stopped. I0506 20:29:56.020513 1 tasks_processing.go:74] worker 60 stopped. I0506 20:29:56.020588 1 recorder.go:75] Recording config/ingress with fingerprint=30a57962f8cab150b942f2c54e55789da18e9acebaecde9f45391a79c4df3165 I0506 20:29:56.020611 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 46.195034ms to process 1 records I0506 20:29:56.020618 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 46.951381ms to process 0 records I0506 20:29:56.020627 1 tasks_processing.go:74] worker 14 stopped. I0506 20:29:56.028500 1 tasks_processing.go:74] worker 42 stopped. I0506 20:29:56.029915 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-pkctz with fingerprint=b633c81e3ed62abcbd51ef5f6e236f680056cf0a9825b889e2370e58d64e65fb I0506 20:29:56.030004 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-2-52.ec2.internal with fingerprint=d292e15073c404d9e58cd0187f227f534d1b01c8b466ef34a0b5345e3b1aec9b I0506 20:29:56.030049 1 recorder.go:75] Recording config/running_containers with fingerprint=acd7d157e0b2cc7d18a6326dd6948d36da0eb71c1c3c1476835a920934a4b3ab I0506 20:29:56.030058 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 56.920678ms to process 3 records I0506 20:29:56.032555 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0506 20:29:56.032586 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0506 20:29:56.032603 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0506 20:29:56.037325 1 tasks_processing.go:74] worker 36 stopped. I0506 20:29:56.037359 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0506 20:29:56.037372 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 65.3238ms to process 1 records I0506 20:29:56.037382 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 63.303891ms to process 0 records I0506 20:29:56.037390 1 tasks_processing.go:74] worker 27 stopped. I0506 20:29:56.037719 1 base_controller.go:82] Caches are synced for ConfigController I0506 20:29:56.037732 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0506 20:29:56.043440 1 prometheus_rules.go:88] Prometheus rules successfully created I0506 20:29:56.048109 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 20:29:56.048338 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0506 20:29:56.048350 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0506 20:29:56.048353 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0506 20:29:56.048357 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0506 20:29:56.048361 1 controller.go:212] Source scaController *sca.Controller is not ready I0506 20:29:56.048377 1 controller.go:489] The operator is still being initialized I0506 20:29:56.048384 1 controller.go:512] The operator is healthy E0506 20:29:56.056864 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%275fe5cec3-1171-4c19-a048-711274ea967d%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:60732->172.30.0.10:53: read: connection refused I0506 20:29:56.056881 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%275fe5cec3-1171-4c19-a048-711274ea967d%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:60732->172.30.0.10:53: read: connection refused I0506 20:29:56.066386 1 tasks_processing.go:74] worker 58 stopped. E0506 20:29:56.066401 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0506 20:29:56.066407 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0506 20:29:56.066411 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0506 20:29:56.066420 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0506 20:29:56.066445 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0506 20:29:56.066453 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0506 20:29:56.066458 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0506 20:29:56.066462 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0506 20:29:56.066499 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0506 20:29:56.066507 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0506 20:29:56.066512 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 92.183518ms to process 7 records I0506 20:29:56.067280 1 tasks_processing.go:74] worker 59 stopped. E0506 20:29:56.067293 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0506 20:29:56.067299 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q4kpnqpeeltr5llhepbddms6skg44m1-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q4kpnqpeeltr5llhepbddms6skg44m1-primary-cert-bundle-secret" not found I0506 20:29:56.067344 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=92be1f7f415ca3e2a9f7311150a6b8a9ad37c6ef6eba84ab03753c1b9d070e41 I0506 20:29:56.067356 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 93.085945ms to process 1 records I0506 20:29:56.107488 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0506 20:29:56.111058 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:44198->172.30.0.10:53: read: connection refused I0506 20:29:56.111072 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:44198->172.30.0.10:53: read: connection refused I0506 20:29:56.121044 1 base_controller.go:82] Caches are synced for LoggingSyncer I0506 20:29:56.121055 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0506 20:29:57.015462 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 20:29:57.215617 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0506 20:29:57.215632 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0506 20:29:57.216343 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-h7zds pod in namespace openshift-dns (previous: false). I0506 20:29:57.443981 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-h7zds pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-h7zds\" is waiting to start: ContainerCreating" I0506 20:29:57.444001 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-h7zds\" is waiting to start: ContainerCreating" I0506 20:29:57.444012 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-h7zds pod in namespace openshift-dns (previous: false). I0506 20:29:57.450272 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0506 20:29:57.622734 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-h7zds pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-h7zds\" is waiting to start: ContainerCreating" I0506 20:29:57.622752 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-h7zds\" is waiting to start: ContainerCreating" I0506 20:29:57.622781 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-kxv2n pod in namespace openshift-dns (previous: false). I0506 20:29:57.840518 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kxv2n pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-kxv2n\" is waiting to start: ContainerCreating" I0506 20:29:57.840539 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-kxv2n\" is waiting to start: ContainerCreating" I0506 20:29:57.840547 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-kxv2n pod in namespace openshift-dns (previous: false). W0506 20:29:58.014779 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 20:29:58.021633 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kxv2n pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-kxv2n\" is waiting to start: ContainerCreating" I0506 20:29:58.021649 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-kxv2n\" is waiting to start: ContainerCreating" I0506 20:29:58.021679 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-l9h5n pod in namespace openshift-dns (previous: false). I0506 20:29:58.052690 1 tasks_processing.go:74] worker 23 stopped. I0506 20:29:58.052741 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=7b6da413a6ba2d6f650662815bf08941a0475cc803572a9ddefcedd93646efa8 I0506 20:29:58.052774 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=1dacf1f9c3fb42f47ab07c98b56a8299af39c1d5821341ef9610f7b1c5c9f5aa I0506 20:29:58.052803 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0506 20:29:58.052844 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=097faad9aa775c3bfeaef03a04515dc9a2d6d3bfc3e979239098eb39985f9b37 I0506 20:29:58.052862 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0506 20:29:58.052886 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=18a20e60145f197a3ff8aaa8c5c13b274222afda05263c5c4b2071ef86ea913b I0506 20:29:58.052915 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=ec07463799795db06cca811317f87f2c22fec4115a527d59be0676f16730d1ba I0506 20:29:58.052939 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=99f36a4ae12cbca8ab827c02e8333958485a3c47ffb5ec8e51ad3c2628abefcd I0506 20:29:58.052952 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=ea86cdd800146707a60e523788880f031e5db58ca19e07e65106e8476ccd62f1 I0506 20:29:58.052969 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=f9c55b2315ab2f2192fbf4f137ce4ed8a8b998daf43d960d8052cf04679b9465 I0506 20:29:58.052978 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0506 20:29:58.052994 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=507dca398f10468b2ba845896b865f1932d83e701255b82ef792573eeb488018 I0506 20:29:58.053004 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0506 20:29:58.053020 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=30bd32c49744e5099488415c9bf274922ff6e1156c3c0951c3ff30afa6c2437c I0506 20:29:58.053030 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0506 20:29:58.053048 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=a2983e6ad7526e8ee9c85bbf00c02d16fd6a22ca6105e67ec9158527eb1fca1d I0506 20:29:58.053058 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0506 20:29:58.053073 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=b70b1e8351a1fa72498f0df1881ce4b098b96d131ace2a65f9276e1d9a9e9aab I0506 20:29:58.053194 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=5306acc3c5e6daa3bc8d26d982ba594d1e2e8bdbec6141069c2475869b60ba41 I0506 20:29:58.053205 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0506 20:29:58.053218 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0506 20:29:58.053239 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0506 20:29:58.053275 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=009c4a2e044e7f429702f6e0967746a08239df95ccc51c736df00d07c3cd5be9 I0506 20:29:58.053301 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=59151ca62851d8fd5816313096d9d53bf87c0dd36a61d1f32732908e1efe1657 I0506 20:29:58.053310 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0506 20:29:58.053325 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=8d39cb29ef896bc0235d3b3eedd4e068cb67840fe03715f807c041d3ca4994ae I0506 20:29:58.053335 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0506 20:29:58.053349 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=4b6e3671c5b2e95b917d3808f122ac3432f237e4b053f8602e251e371a849d09 I0506 20:29:58.053364 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=1aca82832f45e6711e5ad900012b5c002bc69fbcd10f7de2da3d7cb875268e92 I0506 20:29:58.053379 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=da9b4dc9b72cd49a5056ec035bec6d4651fd66494a11f172544477bdfac77081 I0506 20:29:58.053394 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=29f79edc89051493cfcea6f9a12199431726a4dc5fb66ac80fd1a259a2a18406 I0506 20:29:58.053414 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=e0f500f4690fbdabfa7a1e1ce1bf7a1db58691f0fec06b80441aaff12f9ede1a I0506 20:29:58.053424 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0506 20:29:58.053451 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=dfc8a7726acb536450d0de0eb07a520bd7e5dd923703a8fba1960f91e144d078 I0506 20:29:58.053467 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0506 20:29:58.053475 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0506 20:29:58.053482 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.078548308s to process 36 records I0506 20:29:58.247787 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-l9h5n pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-l9h5n\" is waiting to start: ContainerCreating" I0506 20:29:58.247804 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-l9h5n\" is waiting to start: ContainerCreating" I0506 20:29:58.247812 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-l9h5n pod in namespace openshift-dns (previous: false). I0506 20:29:58.421441 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-l9h5n pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-l9h5n\" is waiting to start: ContainerCreating" I0506 20:29:58.421457 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-l9h5n\" is waiting to start: ContainerCreating" I0506 20:29:58.421467 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8f726 pod in namespace openshift-dns (previous: false). I0506 20:29:58.621516 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 20:29:58.621531 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8lw6l pod in namespace openshift-dns (previous: false). I0506 20:29:58.828172 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 20:29:58.828187 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-klnkb pod in namespace openshift-dns (previous: false). W0506 20:29:59.014711 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 20:29:59.029328 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 20:29:59.029378 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-84dbbfbc5d-7kctw pod in namespace openshift-image-registry (previous: false). I0506 20:29:59.221393 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-84dbbfbc5d-7kctw pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-84dbbfbc5d-7kctw\" is waiting to start: ContainerCreating" I0506 20:29:59.221408 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-84dbbfbc5d-7kctw\" is waiting to start: ContainerCreating" I0506 20:29:59.221452 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-b9d9bbcd5-6clcv pod in namespace openshift-image-registry (previous: false). I0506 20:29:59.423179 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-b9d9bbcd5-6clcv pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-b9d9bbcd5-6clcv\" is waiting to start: ContainerCreating" I0506 20:29:59.423196 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-b9d9bbcd5-6clcv\" is waiting to start: ContainerCreating" I0506 20:29:59.423229 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-b9d9bbcd5-ntrpk pod in namespace openshift-image-registry (previous: false). I0506 20:29:59.626093 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-b9d9bbcd5-ntrpk pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-b9d9bbcd5-ntrpk\" is waiting to start: ContainerCreating" I0506 20:29:59.626108 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-b9d9bbcd5-ntrpk\" is waiting to start: ContainerCreating" I0506 20:29:59.626119 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-hjv9h pod in namespace openshift-image-registry (previous: false). I0506 20:29:59.821721 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 20:29:59.821737 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-vxg7w pod in namespace openshift-image-registry (previous: false). W0506 20:30:00.014913 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 20:30:00.022676 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 20:30:00.022691 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-wsmnn pod in namespace openshift-image-registry (previous: false). I0506 20:30:00.235841 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 20:30:00.235858 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6588857547-sc6pl pod in namespace openshift-ingress (previous: false). I0506 20:30:00.422126 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6588857547-sc6pl pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6588857547-sc6pl\" is waiting to start: ContainerCreating" I0506 20:30:00.422145 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6588857547-sc6pl\" is waiting to start: ContainerCreating" I0506 20:30:00.422155 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6bdc599f75-bphmd pod in namespace openshift-ingress (previous: false). I0506 20:30:00.625271 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6bdc599f75-bphmd pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6bdc599f75-bphmd\" is waiting to start: ContainerCreating" I0506 20:30:00.625290 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6bdc599f75-bphmd\" is waiting to start: ContainerCreating" I0506 20:30:00.625301 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6bdc599f75-cfqqq pod in namespace openshift-ingress (previous: false). I0506 20:30:00.821605 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6bdc599f75-cfqqq pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6bdc599f75-cfqqq\" is waiting to start: ContainerCreating" I0506 20:30:00.821623 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6bdc599f75-cfqqq\" is waiting to start: ContainerCreating" I0506 20:30:00.821649 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-8h2wk pod in namespace openshift-ingress-canary (previous: false). W0506 20:30:01.011954 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0506 20:30:01.011976 1 tasks_processing.go:74] worker 2 stopped. E0506 20:30:01.011994 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0506 20:30:01.012011 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0506 20:30:01.012026 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0506 20:30:01.012039 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.037352879s to process 1 records I0506 20:30:01.023129 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-8h2wk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-8h2wk\" is waiting to start: ContainerCreating" I0506 20:30:01.023143 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-8h2wk\" is waiting to start: ContainerCreating" I0506 20:30:01.023170 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dh9zd pod in namespace openshift-ingress-canary (previous: false). I0506 20:30:01.221472 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dh9zd pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dh9zd\" is waiting to start: ContainerCreating" I0506 20:30:01.221488 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dh9zd\" is waiting to start: ContainerCreating" I0506 20:30:01.221514 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-p8vmr pod in namespace openshift-ingress-canary (previous: false). I0506 20:30:01.421975 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-p8vmr pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-p8vmr\" is waiting to start: ContainerCreating" I0506 20:30:01.421992 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-p8vmr\" is waiting to start: ContainerCreating" I0506 20:30:01.422005 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-r68ln pod in namespace openshift-kube-storage-version-migrator (previous: false). I0506 20:30:01.622560 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-r68ln pod in namespace openshift-kube-storage-version-migrator (previous: false). I0506 20:30:01.821176 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-jchcd pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0506 20:30:02.023440 1 tasks_processing.go:74] worker 32 stopped. I0506 20:30:02.023528 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=d95c6422c9da24baa2edbec52ab90b339ca2b7a044dc54c22654194d8f0fa358 I0506 20:30:02.023580 1 recorder.go:75] Recording events/openshift-dns with fingerprint=7bb5d209467faf2c399f1bc68290dc6cdef669bca5a9bc52027929499b5784d8 I0506 20:30:02.023670 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=4a404351c2d331fe075db9473c0c0a8c1386c5eb2d4b8198bea2a78d334b5059 I0506 20:30:02.023698 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=02759cc4115c58549c8bc2db36458f26d92750c73410eaed39252459224b6c2f I0506 20:30:02.023742 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=f1ad6c13205916ac2b518671178c9ca72fb483695434b35fc633d132a40ca283 I0506 20:30:02.023759 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=ba48a39f5b8f10ae11a6a517acb3eef2f52e3d985283c3066205f260d51acdc8 I0506 20:30:02.023772 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=1d638f2d434e2054f5d13c065163dfdc6353a3fe2117840fadc8423a67eeb9fb I0506 20:30:02.023819 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=c443671509f8374823af14c21722a86944b9e7fe3984681f20badf537b580292 I0506 20:30:02.023963 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-h7zds with fingerprint=9e31cd5a3c3276b5e7a3dcb4d8e19c84ec9d6c73e75e4a8fb87efc3a3a2e4502 I0506 20:30:02.024049 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-kxv2n with fingerprint=2644e81107da40c7604ebed8ad9f7b48e46ae60dd54ee0fc4729f329764e35e8 I0506 20:30:02.024129 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-l9h5n with fingerprint=4d29ca0503c39f24593341696bce326e8f4199352cb7713c25f34a2274f973ec I0506 20:30:02.024233 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-84dbbfbc5d-7kctw with fingerprint=1ec6ba674bc2966be516362bf4acd235feac2201ecf49701d0f9364f5b1b0ad4 I0506 20:30:02.024340 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-b9d9bbcd5-6clcv with fingerprint=aef767832f4f10ff9b1cc144acf554a9fc351df1d0a9d297696a3bf2f6d6c397 I0506 20:30:02.024428 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-b9d9bbcd5-ntrpk with fingerprint=e4117274d251a507655e0f5bb3c6b544fb1833202904bddb68de8d4e92402a68 I0506 20:30:02.024484 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-8h2wk with fingerprint=cfb3fb728d564a8775ac732f873aec18f79bd7f51f7c276e4e393fab415640c8 I0506 20:30:02.024540 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-dh9zd with fingerprint=c26d80f4dffefff2a0cde3176936e0e04748c2ed54ae713000600be21775ae91 I0506 20:30:02.024594 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-p8vmr with fingerprint=6b42fcfd2b750219a17260c6640afb7f95551c2b0d2f77a272bb120829d2460f I0506 20:30:02.024608 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-r68ln/migrator_current.log with fingerprint=e971960bf38337c47f3cb80599a274183a8b80632081571ebf766225d3eedc61 I0506 20:30:02.024613 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-r68ln/graceful-termination_current.log with fingerprint=1f83fa2269cdd234a2af7554437f47f98674d04e674fe6eeb9707ead2432d813 I0506 20:30:02.024681 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-jchcd/kube-storage-version-migrator-operator_current.log with fingerprint=e0161ff53c22d4b2f2d08003397aeeb87c32614a2d2c0e8198fff7d4c71b2d81 I0506 20:30:02.024690 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.051842834s to process 20 records I0506 20:30:07.128057 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 20:30:07.880800 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 20:30:08.080338 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 20:30:08.793596 1 tasks_processing.go:74] worker 6 stopped. I0506 20:30:08.793638 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0506 20:30:08.793649 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.822146138s to process 1 records I0506 20:30:09.581125 1 tasks_processing.go:74] worker 51 stopped. I0506 20:30:09.581437 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=6f7ec23b175ef6978e72e155e10475689a15847bf4d5aede6cec1cd3ba8d2466 I0506 20:30:09.581477 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.607118917s to process 1 records E0506 20:30:09.581573 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.61s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0506 20:30:09.582695 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0506 20:30:09.582716 1 periodic.go:209] Running workloads gatherer I0506 20:30:09.582733 1 tasks_processing.go:45] number of workers: 2 I0506 20:30:09.582741 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 20:30:09.582745 1 tasks_processing.go:71] worker 1 working on workload_info task. I0506 20:30:09.582753 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 20:30:09.582771 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0506 20:30:09.606548 1 tasks_processing.go:74] worker 0 stopped. I0506 20:30:09.606581 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 23.736625ms to process 0 records I0506 20:30:09.608966 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0506 20:30:09.617733 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0506 20:30:09.628287 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (11ms) I0506 20:30:09.638670 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0506 20:30:09.647958 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (9ms) I0506 20:30:09.657663 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (10ms) I0506 20:30:09.667380 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (10ms) I0506 20:30:09.677019 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (10ms) I0506 20:30:09.686058 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (9ms) I0506 20:30:09.695618 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (10ms) I0506 20:30:09.704912 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (9ms) I0506 20:30:09.717337 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (12ms) I0506 20:30:09.817457 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0506 20:30:09.918162 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (101ms) I0506 20:30:10.018383 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0506 20:30:10.121052 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (103ms) I0506 20:30:10.218684 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (98ms) I0506 20:30:10.318751 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0506 20:30:10.418082 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (99ms) I0506 20:30:10.517757 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0506 20:30:10.618618 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0506 20:30:10.718183 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0506 20:30:10.817636 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0506 20:30:10.919144 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (101ms) I0506 20:30:11.020042 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (101ms) I0506 20:30:11.120572 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (101ms) I0506 20:30:11.223924 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (103ms) I0506 20:30:11.322410 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (98ms) I0506 20:30:11.422114 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0506 20:30:11.520223 1 gather_workloads_info.go:387] No image sha256:f68c0da8268f23078f5f760a91a3c8212f2195acf18c231f6e034207a67adeec (98ms) I0506 20:30:11.620481 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0506 20:30:11.725225 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (105ms) I0506 20:30:11.818595 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (93ms) I0506 20:30:11.919874 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (101ms) I0506 20:30:11.919906 1 tasks_processing.go:74] worker 1 stopped. E0506 20:30:11.919918 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0506 20:30:11.920290 1 recorder.go:75] Recording config/workload_info with fingerprint=bd4f6bfaace2d06287dde4b159508755c88c25125e865c5869d69c46d6a33634 I0506 20:30:11.920325 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.337152379s to process 1 records E0506 20:30:11.920356 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.337s with: function \"workload_info\" failed with an error" I0506 20:30:11.921462 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0506 20:30:11.921477 1 periodic.go:209] Running conditional gatherer I0506 20:30:11.927455 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0506 20:30:11.934876 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.10:36031->172.30.0.10:53: read: connection refused E0506 20:30:11.935144 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0506 20:30:11.935208 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0506 20:30:11.941332 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0506 20:30:11.941349 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941355 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941359 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941362 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941366 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941370 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941373 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941376 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 20:30:11.941379 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0506 20:30:11.941396 1 tasks_processing.go:45] number of workers: 3 I0506 20:30:11.941411 1 tasks_processing.go:69] worker 2 listening for tasks. I0506 20:30:11.941416 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0506 20:30:11.941424 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 20:30:11.941435 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 20:30:11.941450 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0506 20:30:11.941465 1 tasks_processing.go:74] worker 1 stopped. I0506 20:30:11.941471 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0506 20:30:11.941574 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0506 20:30:11.941589 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.466µs to process 1 records I0506 20:30:11.941625 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0506 20:30:11.941633 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.498µs to process 1 records I0506 20:30:11.941640 1 tasks_processing.go:74] worker 0 stopped. I0506 20:30:11.941683 1 tasks_processing.go:74] worker 2 stopped. I0506 20:30:11.941696 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 262.039µs to process 0 records I0506 20:30:11.941725 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.10:36031->172.30.0.10:53: read: connection refused I0506 20:30:11.941746 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0506 20:30:11.962702 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=f80cb61623669792e1763b107019c9cae9761db07d8fba236f5a3d6bd4fc2f28 I0506 20:30:11.962847 1 diskrecorder.go:70] Writing 117 records to /var/lib/insights-operator/insights-2026-05-06-203011.tar.gz I0506 20:30:11.970571 1 diskrecorder.go:51] Wrote 117 records to disk in 7ms I0506 20:30:11.970602 1 periodic.go:278] Gathering cluster info every 2h0m0s I0506 20:30:11.970617 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0506 20:30:23.937437 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 20:31:20.224092 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="936141d9529b314572e18a41fd204898829e14ec312291a8f7fe1a005d9cd378") W0506 20:31:20.224130 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0506 20:31:20.224176 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="f6f175f92a58607f4d561ac4f3c06f5d014d3d64f338680f69bcbb31076f9438") I0506 20:31:20.224181 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0506 20:31:20.224260 1 base_controller.go:181] Shutting down ConfigController ... I0506 20:31:20.224280 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="6fded6d5212f32813ee608fdb759b7f766839a51701233cd3cca4324d94aee7c") I0506 20:31:20.224280 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0506 20:31:20.224287 1 periodic.go:170] Shutting down I0506 20:31:20.224296 1 base_controller.go:113] All ConfigController workers have been terminated E0506 20:31:20.224303 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled I0506 20:31:20.224305 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0506 20:31:20.224306 1 base_controller.go:181] Shutting down LoggingSyncer ... I0506 20:31:20.224322 1 base_controller.go:113] All LoggingSyncer workers have been terminated