W0705 21:51:07.252780 1 cmd.go:257] Using insecure, self-signed certificates I0705 21:51:07.815832 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 21:51:07.816212 1 observer_polling.go:159] Starting file observer I0705 21:51:08.461079 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0705 21:51:08.461297 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0705 21:51:08.461938 1 secure_serving.go:57] Forcing use of http/1.1 only W0705 21:51:08.461962 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0705 21:51:08.461969 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0705 21:51:08.461977 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0705 21:51:08.461981 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0705 21:51:08.461985 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0705 21:51:08.461988 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0705 21:51:08.462216 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0705 21:51:08.468062 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"6017f7a4-3b5c-4b6f-9052-acd0955cba3a", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0705 21:51:08.468076 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0705 21:51:08.468960 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0705 21:51:08.468978 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0705 21:51:08.468976 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0705 21:51:08.468991 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0705 21:51:08.469004 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0705 21:51:08.468996 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0705 21:51:08.469517 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3265963187/tls.crt::/tmp/serving-cert-3265963187/tls.key" I0705 21:51:08.469967 1 secure_serving.go:213] Serving securely on [::]:8443 I0705 21:51:08.470002 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0705 21:51:08.474743 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0705 21:51:08.474771 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0705 21:51:08.474809 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0705 21:51:08.480980 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 21:51:08.481000 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0705 21:51:08.487489 1 secretconfigobserver.go:119] support secret does not exist I0705 21:51:08.494472 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 21:51:08.500461 1 secretconfigobserver.go:119] support secret does not exist I0705 21:51:08.502856 1 recorder.go:161] Pruning old reports every 8h38m2s, max age is 288h0m0s I0705 21:51:08.509409 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0705 21:51:08.509443 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0705 21:51:08.509410 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0705 21:51:08.509462 1 periodic.go:209] Running clusterconfig gatherer I0705 21:51:08.509467 1 insightsreport.go:296] Starting report retriever I0705 21:51:08.509475 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0705 21:51:08.509508 1 tasks_processing.go:45] number of workers: 32 I0705 21:51:08.509541 1 tasks_processing.go:69] worker 31 listening for tasks. I0705 21:51:08.509543 1 tasks_processing.go:69] worker 3 listening for tasks. I0705 21:51:08.509550 1 tasks_processing.go:71] worker 31 working on storage_classes task. I0705 21:51:08.509551 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 21:51:08.509558 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 21:51:08.509560 1 tasks_processing.go:69] worker 2 listening for tasks. I0705 21:51:08.509568 1 tasks_processing.go:69] worker 10 listening for tasks. I0705 21:51:08.509571 1 tasks_processing.go:69] worker 24 listening for tasks. I0705 21:51:08.509579 1 tasks_processing.go:69] worker 4 listening for tasks. I0705 21:51:08.509565 1 tasks_processing.go:69] worker 17 listening for tasks. I0705 21:51:08.509579 1 tasks_processing.go:69] worker 18 listening for tasks. I0705 21:51:08.509587 1 tasks_processing.go:69] worker 19 listening for tasks. I0705 21:51:08.509592 1 tasks_processing.go:69] worker 5 listening for tasks. I0705 21:51:08.509594 1 tasks_processing.go:69] worker 11 listening for tasks. I0705 21:51:08.509597 1 tasks_processing.go:69] worker 20 listening for tasks. I0705 21:51:08.509602 1 tasks_processing.go:69] worker 6 listening for tasks. I0705 21:51:08.509605 1 tasks_processing.go:69] worker 21 listening for tasks. I0705 21:51:08.509603 1 tasks_processing.go:69] worker 12 listening for tasks. I0705 21:51:08.509609 1 tasks_processing.go:69] worker 13 listening for tasks. I0705 21:51:08.509612 1 tasks_processing.go:69] worker 7 listening for tasks. I0705 21:51:08.509616 1 tasks_processing.go:69] worker 14 listening for tasks. I0705 21:51:08.509620 1 tasks_processing.go:69] worker 8 listening for tasks. I0705 21:51:08.509622 1 tasks_processing.go:69] worker 15 listening for tasks. I0705 21:51:08.509625 1 tasks_processing.go:69] worker 22 listening for tasks. I0705 21:51:08.509631 1 tasks_processing.go:69] worker 9 listening for tasks. I0705 21:51:08.509633 1 tasks_processing.go:69] worker 23 listening for tasks. I0705 21:51:08.509631 1 tasks_processing.go:69] worker 16 listening for tasks. I0705 21:51:08.509639 1 tasks_processing.go:69] worker 27 listening for tasks. I0705 21:51:08.509641 1 tasks_processing.go:69] worker 25 listening for tasks. I0705 21:51:08.509644 1 tasks_processing.go:69] worker 29 listening for tasks. I0705 21:51:08.509648 1 tasks_processing.go:69] worker 26 listening for tasks. I0705 21:51:08.509652 1 tasks_processing.go:69] worker 28 listening for tasks. I0705 21:51:08.509663 1 tasks_processing.go:69] worker 30 listening for tasks. I0705 21:51:08.509722 1 tasks_processing.go:71] worker 30 working on ingress task. I0705 21:51:08.509734 1 tasks_processing.go:71] worker 21 working on storage_cluster task. I0705 21:51:08.509750 1 tasks_processing.go:71] worker 9 working on clusterroles task. I0705 21:51:08.509883 1 tasks_processing.go:71] worker 23 working on openstack_controlplanes task. I0705 21:51:08.509903 1 tasks_processing.go:71] worker 3 working on sap_pods task. I0705 21:51:08.510022 1 tasks_processing.go:71] worker 17 working on machines task. I0705 21:51:08.510113 1 tasks_processing.go:71] worker 16 working on jaegers task. I0705 21:51:08.510261 1 tasks_processing.go:71] worker 1 working on tsdb_status task. I0705 21:51:08.510325 1 tasks_processing.go:71] worker 27 working on machine_sets task. I0705 21:51:08.510344 1 tasks_processing.go:71] worker 0 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0705 21:51:08.510407 1 tasks_processing.go:71] worker 12 working on cost_management_metrics_configs task. I0705 21:51:08.510498 1 tasks_processing.go:71] worker 18 working on operators_pods_and_events task. I0705 21:51:08.510546 1 tasks_processing.go:71] worker 25 working on container_images task. I0705 21:51:08.510594 1 tasks_processing.go:71] worker 19 working on dvo_metrics task. I0705 21:51:08.510698 1 tasks_processing.go:71] worker 13 working on cluster_apiserver task. I0705 21:51:08.510854 1 tasks_processing.go:71] worker 29 working on validating_webhook_configurations task. I0705 21:51:08.510868 1 tasks_processing.go:71] worker 7 working on networks task. I0705 21:51:08.510942 1 tasks_processing.go:71] worker 2 working on openshift_logging task. I0705 21:51:08.511018 1 tasks_processing.go:71] worker 5 working on feature_gates task. I0705 21:51:08.511112 1 tasks_processing.go:71] worker 10 working on image_registries task. I0705 21:51:08.511150 1 tasks_processing.go:71] worker 26 working on machine_autoscalers task. I0705 21:51:08.511255 1 tasks_processing.go:71] worker 14 working on ceph_cluster task. W0705 21:51:08.510336 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 21:51:08.511357 1 tasks_processing.go:71] worker 6 working on nodenetworkconfigurationpolicies task. I0705 21:51:08.511419 1 tasks_processing.go:71] worker 28 working on openstack_dataplanenodesets task. I0705 21:51:08.511368 1 tasks_processing.go:71] worker 1 working on crds task. I0705 21:51:08.511335 1 tasks_processing.go:71] worker 11 working on qemu_kubevirt_launcher_logs task. I0705 21:51:08.511343 1 tasks_processing.go:71] worker 15 working on install_plans task. I0705 21:51:08.511347 1 tasks_processing.go:71] worker 8 working on image_pruners task. I0705 21:51:08.511352 1 tasks_processing.go:71] worker 22 working on aggregated_monitoring_cr_names task. I0705 21:51:08.511317 1 tasks_processing.go:71] worker 20 working on node_logs task. I0705 21:51:08.511380 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 1.042747ms to process 0 records I0705 21:51:08.511387 1 tasks_processing.go:71] worker 24 working on active_alerts task. W0705 21:51:08.512450 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 21:51:08.512465 1 tasks_processing.go:71] worker 24 working on operators task. I0705 21:51:08.511392 1 tasks_processing.go:71] worker 4 working on proxies task. I0705 21:51:08.512627 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 45.059µs to process 0 records I0705 21:51:08.515362 1 tasks_processing.go:71] worker 21 working on config_maps task. I0705 21:51:08.515378 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 5.612264ms to process 0 records I0705 21:51:08.516120 1 tasks_processing.go:71] worker 23 working on oauths task. I0705 21:51:08.516133 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 6.222592ms to process 0 records I0705 21:51:08.516607 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0705 21:51:08.516624 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0705 21:51:08.516629 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0705 21:51:08.516632 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0705 21:51:08.516650 1 controller.go:489] The operator is still being initialized I0705 21:51:08.516656 1 controller.go:512] The operator is healthy I0705 21:51:08.517592 1 tasks_processing.go:71] worker 17 working on container_runtime_configs task. E0705 21:51:08.517603 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0705 21:51:08.517620 1 gather.go:177] gatherer "clusterconfig" function "machines" took 7.550242ms to process 0 records I0705 21:51:08.518194 1 tasks_processing.go:71] worker 26 working on infrastructures task. I0705 21:51:08.518210 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 7.02697ms to process 0 records I0705 21:51:08.518416 1 tasks_processing.go:71] worker 16 working on support_secret task. I0705 21:51:08.518432 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 8.288121ms to process 0 records I0705 21:51:08.518724 1 tasks_processing.go:71] worker 12 working on pdbs task. I0705 21:51:08.518737 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 8.304177ms to process 0 records I0705 21:51:08.518848 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 8.824963ms to process 0 records I0705 21:51:08.518853 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 7.779139ms to process 0 records I0705 21:51:08.518860 1 tasks_processing.go:71] worker 2 working on machine_config_pools task. I0705 21:51:08.518962 1 tasks_processing.go:71] worker 3 working on olm_operators task. I0705 21:51:08.519086 1 tasks_processing.go:71] worker 27 working on authentication task. I0705 21:51:08.519095 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 8.7326ms to process 0 records I0705 21:51:08.519960 1 tasks_processing.go:71] worker 7 working on metrics task. W0705 21:51:08.519994 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 21:51:08.520145 1 recorder.go:75] Recording config/network with fingerprint=d9a4b6229e8a507a57d19a237b3a3fc26f6c8f830722d359d77c79621b368d73 I0705 21:51:08.520180 1 gather.go:177] gatherer "clusterconfig" function "networks" took 9.076927ms to process 1 records I0705 21:51:08.520197 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.579µs to process 0 records I0705 21:51:08.520206 1 tasks_processing.go:71] worker 7 working on nodenetworkstates task. I0705 21:51:08.520715 1 tasks_processing.go:71] worker 31 working on schedulers task. I0705 21:51:08.520821 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=68482c42aa9743bf38e86060b145593246a5f39a2db17348d85cf526b07b75d5 I0705 21:51:08.520844 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=6db71267de4119d99a8b21267b31e2c97295614d1ea5ca2133ab75bdd8ad3ff9 I0705 21:51:08.520854 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 11.152894ms to process 2 records I0705 21:51:08.520934 1 tasks_processing.go:71] worker 30 working on service_accounts task. I0705 21:51:08.521026 1 recorder.go:75] Recording config/ingress with fingerprint=a1002af2851189a0475de8162c606e2c13e939ed9cb1b9e77d832a5c2008595b I0705 21:51:08.521038 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 11.045717ms to process 1 records I0705 21:51:08.522602 1 tasks_processing.go:71] worker 13 working on pod_network_connectivity_checks task. I0705 21:51:08.522756 1 recorder.go:75] Recording config/apiserver with fingerprint=28c8572d7f25d651a4b45fbeac5887c48599c9d229ac08b3f7e8f381a8b27cb5 I0705 21:51:08.522770 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 11.887811ms to process 1 records I0705 21:51:08.522776 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 11.480241ms to process 0 records I0705 21:51:08.522784 1 tasks_processing.go:71] worker 14 working on monitoring_persistent_volumes task. I0705 21:51:08.526102 1 tasks_processing.go:71] worker 17 working on machine_healthchecks task. I0705 21:51:08.526115 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 8.492261ms to process 0 records I0705 21:51:08.526838 1 tasks_processing.go:71] worker 5 working on version task. I0705 21:51:08.526953 1 recorder.go:75] Recording config/featuregate with fingerprint=4c7bc493bd30e4b27fe48dcdedf88bb7019677fff2c36138e4b6e804fa6be32e I0705 21:51:08.526969 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 15.800527ms to process 1 records I0705 21:51:08.527209 1 tasks_processing.go:71] worker 10 working on sap_config task. I0705 21:51:08.527668 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=751f0f5535eb8f6770e460383dfa20d1f9b7d1f97a8c2cc1ccc7f31cbc44ff1f I0705 21:51:08.527684 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 16.078791ms to process 1 records I0705 21:51:08.527770 1 tasks_processing.go:71] worker 6 working on overlapping_namespace_uids task. I0705 21:51:08.527791 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 16.360238ms to process 0 records I0705 21:51:08.527831 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 16.346462ms to process 0 records I0705 21:51:08.527843 1 tasks_processing.go:71] worker 28 working on ingress_certificates task. I0705 21:51:08.528305 1 tasks_processing.go:71] worker 4 working on silenced_alerts task. W0705 21:51:08.528339 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 21:51:08.528456 1 recorder.go:75] Recording config/proxy with fingerprint=0fcc52b5f1c4b78768e3c4f0a140a319a5784a8c92ea522950b9e593b8082678 I0705 21:51:08.528475 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 15.656442ms to process 1 records I0705 21:51:08.528488 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 29.78µs to process 0 records I0705 21:51:08.528506 1 tasks_processing.go:71] worker 4 working on image task. I0705 21:51:08.530390 1 gather_logs.go:145] no pods in namespace were found I0705 21:51:08.530446 1 tasks_processing.go:71] worker 11 working on nodes task. I0705 21:51:08.530490 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 18.950524ms to process 0 records I0705 21:51:08.533994 1 tasks_processing.go:71] worker 13 working on mutating_webhook_configurations task. E0705 21:51:08.534218 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0705 21:51:08.534268 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 11.334271ms to process 0 records I0705 21:51:08.534300 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 13.828797ms to process 0 records I0705 21:51:08.534329 1 tasks_processing.go:71] worker 7 working on openstack_dataplanedeployments task. I0705 21:51:08.535372 1 tasks_processing.go:71] worker 8 working on sap_datahubs task. I0705 21:51:08.536104 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=ac3ff9c0a940fca23828e72e3e33a25fa78e2ad8b55bd7cf393c5a32a9037c3d I0705 21:51:08.536199 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 23.675401ms to process 1 records W0705 21:51:08.536365 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 21:51:08.536456 1 tasks_processing.go:71] worker 23 working on openshift_machine_api_events task. I0705 21:51:08.537085 1 recorder.go:75] Recording config/oauth with fingerprint=a178b18203401c1ab88c9d10da7b2898059d884548bfeba8f45f5246a773c6d4 I0705 21:51:08.537139 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 19.286188ms to process 1 records E0705 21:51:08.537216 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0705 21:51:08.537267 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 10.138183ms to process 0 records I0705 21:51:08.537297 1 tasks_processing.go:71] worker 17 working on lokistack task. I0705 21:51:08.537957 1 tasks_processing.go:71] worker 29 working on openstack_version task. I0705 21:51:08.538235 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=8bf6cbd36d5a2278f001a7719b7fc8fcf56f6c33c58c51d901399cbdd7914fd3 I0705 21:51:08.538344 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=5b15c5f07e053246c0453547d90ca58408b3c1be577f0c3cb593e3b46916eb10 I0705 21:51:08.538373 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=f8f8ced115d1d75e35f366e6eac1b7cdfb127c05b0b873de1ea8c69cec914f3b I0705 21:51:08.538403 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=3623e6d18a8fbaf56e9bbad89455c28d40c731199dae42fa567555208d6770bc I0705 21:51:08.538433 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=cf99dc0b4de1264773c6083960f056ed7da06f64da3cc7210ae37e3ba098d55b I0705 21:51:08.538466 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=e4e9eb330e95244a382795c96608232d4d26b00503bc9f8e66c3b1da36d26c4d I0705 21:51:08.538494 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=b51054f8ee5e4a0bdc1129e1d1af42dd80c8155e80b8f238af4e8f211d3fdb6f I0705 21:51:08.538532 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=a401ef87f336466151f19fcb397e407af3b298ffe0e915ca7498aa1f2af4e156 I0705 21:51:08.538560 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=d5de2627a6e66c34d7085f6747a26d04deed8d28689afba8a029c09ff4af64de I0705 21:51:08.538590 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=a649061af97613706214c660acd5763e7e18d6fc80a353d9593b5eb5f73a65f0 I0705 21:51:08.538618 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=f58e868496db3ddabc490e020e63e705b1878bedbb1757093440fa8a5ce260a4 I0705 21:51:08.538626 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 27.080941ms to process 11 records I0705 21:51:08.544182 1 tasks_processing.go:71] worker 10 working on machine_configs task. I0705 21:51:08.544189 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 16.941972ms to process 0 records I0705 21:51:08.544526 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 32.046419ms to process 0 records I0705 21:51:08.544562 1 tasks_processing.go:71] worker 20 working on certificate_signing_requests task. W0705 21:51:08.544714 1 operator.go:288] started I0705 21:51:08.544798 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0705 21:51:08.544841 1 recorder.go:75] Recording config/authentication with fingerprint=bab6588fd68c51dcd18c37a53653159ba897818f29851a48e6ebf89eef8a029d I0705 21:51:08.544866 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 25.490715ms to process 1 records I0705 21:51:08.544991 1 tasks_processing.go:74] worker 2 stopped. I0705 21:51:08.544992 1 tasks_processing.go:74] worker 27 stopped. I0705 21:51:08.545023 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 26.124712ms to process 0 records I0705 21:51:08.545038 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0705 21:51:08.545039 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0705 21:51:08.545422 1 tasks_processing.go:74] worker 7 stopped. I0705 21:51:08.545477 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 11.063248ms to process 0 records I0705 21:51:08.546609 1 tasks_processing.go:74] worker 31 stopped. I0705 21:51:08.546757 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=38c3d81a1bea2ca9dab456969ec9ddb6d2f3c1410c2946879e1c27229ddcae06 I0705 21:51:08.546776 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 25.877985ms to process 1 records I0705 21:51:08.546837 1 tasks_processing.go:74] worker 16 stopped. E0705 21:51:08.546850 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0705 21:51:08.546860 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 28.408921ms to process 0 records I0705 21:51:08.547395 1 tasks_processing.go:74] worker 9 stopped. I0705 21:51:08.547827 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=dd80997c19c41b96b509d70e3b6260c8a7e706b4dba7227130283298d186228a I0705 21:51:08.548037 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=743c19bea002237589a8306503755cdb5b047b8d6ebd14ff2ed03f7fc270d8f0 I0705 21:51:08.548076 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 37.631417ms to process 2 records I0705 21:51:08.548205 1 recorder.go:75] Recording config/running_containers with fingerprint=ee528a3c94bdbc1f74b9c78b56618de74156199f7d0820e8a09da73ee46bcd4f I0705 21:51:08.548242 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 37.349819ms to process 1 records I0705 21:51:08.548850 1 tasks_processing.go:74] worker 25 stopped. I0705 21:51:08.549357 1 tasks_processing.go:74] worker 26 stopped. I0705 21:51:08.549528 1 recorder.go:75] Recording config/infrastructure with fingerprint=ad46c5bbc7d9603e33c2e03773ed606fc64088863cb60deb4c0e5c05ab013c65 I0705 21:51:08.549577 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 29.733023ms to process 1 records I0705 21:51:08.549606 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 29.871401ms to process 0 records I0705 21:51:08.549635 1 tasks_processing.go:74] worker 3 stopped. I0705 21:51:08.550659 1 tasks_processing.go:74] worker 8 stopped. I0705 21:51:08.550720 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 15.270097ms to process 0 records I0705 21:51:08.550766 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 13.359891ms to process 0 records I0705 21:51:08.550794 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 12.680586ms to process 0 records I0705 21:51:08.550820 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 27.944662ms to process 0 records I0705 21:51:08.550853 1 tasks_processing.go:74] worker 14 stopped. I0705 21:51:08.550876 1 tasks_processing.go:74] worker 17 stopped. I0705 21:51:08.550900 1 tasks_processing.go:74] worker 29 stopped. I0705 21:51:08.551075 1 tasks_processing.go:74] worker 12 stopped. I0705 21:51:08.551345 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=9385eacafec8a1da550b8f6cfbeaf065cb693f4f20a42cf5bcce1d8b7e9af781 I0705 21:51:08.551458 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=28acf21c84eaf31108d81ce3e6202b1f55daa391c9b9b98ba010df23bc3b92aa I0705 21:51:08.551550 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=9bb206acc2e15d66a2654f78666ae6b7b612b8139132769134e03d6b60ee9ae2 I0705 21:51:08.551587 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 32.2965ms to process 3 records I0705 21:51:08.552284 1 tasks_processing.go:74] worker 1 stopped. I0705 21:51:08.552789 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=084446d9ac8bc40787ea308bfbf1225255d82bb324bef50386aec08273fc9488 I0705 21:51:08.553445 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=8a59835e15b1a76b002bd295b8de7eed1f8448375825c4743bc50c85a1995c68 I0705 21:51:08.553486 1 gather.go:177] gatherer "clusterconfig" function "crds" took 39.746376ms to process 2 records I0705 21:51:08.554321 1 tasks_processing.go:74] worker 4 stopped. I0705 21:51:08.554582 1 recorder.go:75] Recording config/image with fingerprint=e74c29d636610a12b30bc0d2404431dbd4f8ce6896cf819ba7301046a3bb8b31 I0705 21:51:08.554612 1 gather.go:177] gatherer "clusterconfig" function "image" took 25.803605ms to process 1 records I0705 21:51:08.554978 1 tasks_processing.go:74] worker 13 stopped. I0705 21:51:08.555153 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=5635ef57ae3eba6387c4525e8b13eee829432fdee493779421223513125aa0ce I0705 21:51:08.555234 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=2c513f29409f3441f9ce04ffe3199478318ea491ca5c6e6ed5db3fece87b6077 I0705 21:51:08.555288 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=bcbdc9379c6e7c1db8442e2833f31a713f1a1b21c18ff5262f01d7803f1f68ba I0705 21:51:08.555302 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 20.924518ms to process 3 records I0705 21:51:08.563275 1 tasks_processing.go:74] worker 23 stopped. I0705 21:51:08.563291 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 26.784135ms to process 0 records I0705 21:51:08.563301 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 52.940392ms to process 0 records I0705 21:51:08.563309 1 tasks_processing.go:74] worker 0 stopped. I0705 21:51:08.563504 1 tasks_processing.go:74] worker 11 stopped. I0705 21:51:08.563893 1 recorder.go:75] Recording config/node/ip-10-0-0-35.ec2.internal with fingerprint=8bc4b8b7aa8fec0019d5d94855afd1ccfc5eb3e2e68269c46ea9201f32252838 I0705 21:51:08.563908 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 33.023571ms to process 1 records I0705 21:51:08.565680 1 prometheus_rules.go:88] Prometheus rules successfully created I0705 21:51:08.565766 1 tasks_processing.go:74] worker 22 stopped. I0705 21:51:08.565797 1 controller.go:212] Source scaController *sca.Controller is not ready I0705 21:51:08.565825 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0705 21:51:08.565830 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0705 21:51:08.565834 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0705 21:51:08.565837 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0705 21:51:08.565866 1 controller.go:489] The operator is still being initialized I0705 21:51:08.565934 1 controller.go:512] The operator is healthy I0705 21:51:08.565809 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 53.826196ms to process 0 records I0705 21:51:08.566810 1 tasks_processing.go:74] worker 20 stopped. I0705 21:51:08.566827 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 22.204427ms to process 0 records I0705 21:51:08.569033 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0705 21:51:08.569040 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0705 21:51:08.569047 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file E0705 21:51:08.571969 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2737dee443-ef17-414a-b411-6bbd0783b179%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.14:40010->172.30.0.10:53: read: connection refused I0705 21:51:08.571982 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2737dee443-ef17-414a-b411-6bbd0783b179%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.14:40010->172.30.0.10:53: read: connection refused I0705 21:51:08.575581 1 base_controller.go:82] Caches are synced for ConfigController I0705 21:51:08.575595 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0705 21:51:08.581867 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 21:51:08.583393 1 tasks_processing.go:74] worker 21 stopped. E0705 21:51:08.583411 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0705 21:51:08.583422 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0705 21:51:08.583427 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0705 21:51:08.583443 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0705 21:51:08.583497 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0705 21:51:08.583506 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0705 21:51:08.583513 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0705 21:51:08.583519 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0705 21:51:08.583580 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0705 21:51:08.583590 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0705 21:51:08.583598 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 68.010554ms to process 7 records I0705 21:51:08.586621 1 tasks_processing.go:74] worker 28 stopped. E0705 21:51:08.586636 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0705 21:51:08.586643 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rc80blalc4pj5c2hirn6qrh369a4h15-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rc80blalc4pj5c2hirn6qrh369a4h15-primary-cert-bundle-secret" not found I0705 21:51:08.586707 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=403fe52afec8e153bc4b4d8ed103179948dc7dfae12fc1d3970043e960266fe1 I0705 21:51:08.586723 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 58.764513ms to process 1 records I0705 21:51:08.624786 1 tasks_processing.go:74] worker 5 stopped. I0705 21:51:08.625081 1 recorder.go:75] Recording config/version with fingerprint=ef002d6c529136888b05278558e5f5e84381d26b89c530cd240c39d7f735245f I0705 21:51:08.625096 1 recorder.go:75] Recording config/id with fingerprint=fb9082caf345808f8b8406ccb1f0dc3d02b3b88b5fdb631dce0b9de8c257cff4 I0705 21:51:08.625105 1 gather.go:177] gatherer "clusterconfig" function "version" took 97.929282ms to process 2 records I0705 21:51:08.635972 1 tasks_processing.go:74] worker 6 stopped. I0705 21:51:08.636054 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0705 21:51:08.636089 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 108.187288ms to process 1 records I0705 21:51:08.645056 1 base_controller.go:82] Caches are synced for LoggingSyncer I0705 21:51:08.645070 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0705 21:51:08.649635 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0705 21:51:08.653521 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.14:50186->172.30.0.10:53: read: connection refused I0705 21:51:08.653533 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.14:50186->172.30.0.10:53: read: connection refused I0705 21:51:08.653587 1 tasks_processing.go:74] worker 10 stopped. I0705 21:51:08.653621 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0705 21:51:08.653637 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 109.3424ms to process 1 records I0705 21:51:08.959788 1 gather_cluster_operator_pods_and_events.go:121] Found 10 pods with 11 containers I0705 21:51:08.959800 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 2287802 bytes I0705 21:51:08.959828 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-lmqxm pod in namespace openshift-dns (previous: false). I0705 21:51:09.192618 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-lmqxm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-lmqxm\" is waiting to start: ContainerCreating" I0705 21:51:09.192637 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-lmqxm\" is waiting to start: ContainerCreating" I0705 21:51:09.192645 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-lmqxm pod in namespace openshift-dns (previous: false). I0705 21:51:09.365388 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-lmqxm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-lmqxm\" is waiting to start: ContainerCreating" I0705 21:51:09.365407 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-lmqxm\" is waiting to start: ContainerCreating" I0705 21:51:09.365416 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-25rg7 pod in namespace openshift-dns (previous: false). W0705 21:51:09.531682 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 21:51:09.565741 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 21:51:09.565755 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-58bd79854-fpdxz pod in namespace openshift-image-registry (previous: false). I0705 21:51:09.771288 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-58bd79854-fpdxz pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-58bd79854-fpdxz\" is waiting to start: ContainerCreating" I0705 21:51:09.771305 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-58bd79854-fpdxz\" is waiting to start: ContainerCreating" I0705 21:51:09.771314 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-58bd79854-qhbm8 pod in namespace openshift-image-registry (previous: false). I0705 21:51:09.959696 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 21:51:09.959712 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6c4cc698df-pcmjl pod in namespace openshift-image-registry (previous: false). I0705 21:51:09.980083 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0705 21:51:10.159940 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 21:51:10.159958 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-btn62 pod in namespace openshift-image-registry (previous: false). I0705 21:51:10.365935 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 21:51:10.365954 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-597bfd7b6f-86tzc pod in namespace openshift-ingress (previous: false). I0705 21:51:10.383402 1 tasks_processing.go:74] worker 24 stopped. I0705 21:51:10.383448 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=7137e657a16b0f0e140d9457beead666cd683095de0d3daa9de2a8753efee145 I0705 21:51:10.383481 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=1a995d423a30cd8b2d276820daf8eec302ceee675edab61e8894e9e00c3e210f I0705 21:51:10.383511 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0705 21:51:10.383537 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=e49f2274743c60f611d99c507a168d2b54f466dcc44d54b8bafa42d8ad6265ec I0705 21:51:10.383554 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0705 21:51:10.383577 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=1bbe44cd3d319337c195ba690e5dd1e981fb32ba75b0654be7edd0becc53152e I0705 21:51:10.383614 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=a25fe8edeeb66b8313512f8e7cd49f9394924cef83d1a18d710975baf4f2a849 I0705 21:51:10.383637 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=2bef1f23a905033c920ad4930c8d3c53af0c191567b7c88616b58bddf4a2c082 I0705 21:51:10.383666 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=e9fee6a76a56e65fcd5e93acd27a32a188559e85978a2f75582b2b060ba318fc I0705 21:51:10.383684 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=692a0ccf17acd1f758de579a45ba516d57f983cd3b7ee017d163e89d7e7ed8d7 I0705 21:51:10.383691 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0705 21:51:10.383707 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=392907c339e2fd5f64ec8f7b6fd897f242d8476c5e420c2c7f5519b47e2c6e42 I0705 21:51:10.383718 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0705 21:51:10.383733 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=22bb8cfd1be8fa26830030442042efbe8764f735fa142f9877f61e40940e6ee1 I0705 21:51:10.383742 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0705 21:51:10.383755 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=6309b9f42ffbdad3fd47b567e910f5ee844b6fba5078bf64ce0318a7bb7391d2 I0705 21:51:10.383764 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0705 21:51:10.383779 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=012cffd5786c88e4f370d92507a3b2295f6b7d885b0ae4b178d038128d1d71f4 I0705 21:51:10.383903 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=286ef8cdb3989a3a7618926193e4e55c99f052f0c2d4e9d71998888b25b44f4c I0705 21:51:10.383913 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0705 21:51:10.383920 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0705 21:51:10.383939 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0705 21:51:10.383960 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=3e1a5bea8b33441489cb0a7b8ccd94900e0cb782d17a8b90ebbaa3a4bbc9f213 I0705 21:51:10.383984 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=c4854116f40c26f6d1c2c229eb45c7eb9d7743c56332eda08fd683bec7979dfa I0705 21:51:10.383994 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0705 21:51:10.384011 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=889477d58932f0e9f339bdbcc65e83abb42d51c658742ebd33d0dd6afc98ec7e I0705 21:51:10.384020 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0705 21:51:10.384033 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=68e2ef83821f6e2f13183a9b61cf8b33755de8500f25b84bb4a405e20a7e7e43 I0705 21:51:10.384048 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=3dff44a3d911867c9d0237aa06f7d81af936f75f3172ea3c8f5ffa40061009a5 I0705 21:51:10.384062 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=3d933f524d27e3a837669bd1b381b7b3285d6eb58a3711306c3854b5f208862e I0705 21:51:10.384077 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=bf686c36f3d0a44d8646ecf4d2e3425ee8f747a33b7268cb2eac65b902ac7307 I0705 21:51:10.384089 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=f970ce0d6ccee17d1f3e84e39643bda8f4f10495b23a85516631e6db0ec6b1a1 I0705 21:51:10.384112 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=d5754f94b204898a10c306584b59744c165f477398d4983717bb2c47a2a96e08 I0705 21:51:10.384126 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0705 21:51:10.384133 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0705 21:51:10.384140 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.870923033s to process 35 records W0705 21:51:10.532324 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 21:51:10.566338 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-597bfd7b6f-86tzc pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-597bfd7b6f-86tzc\" is waiting to start: ContainerCreating" I0705 21:51:10.566354 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-597bfd7b6f-86tzc\" is waiting to start: ContainerCreating" I0705 21:51:10.566372 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6fb5964b47-pdhcr pod in namespace openshift-ingress (previous: false). I0705 21:51:10.760193 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 21:51:10.760210 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6fb5964b47-qnwt8 pod in namespace openshift-ingress (previous: false). I0705 21:51:10.965782 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6fb5964b47-qnwt8 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6fb5964b47-qnwt8\" is waiting to start: ContainerCreating" I0705 21:51:10.965799 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6fb5964b47-qnwt8\" is waiting to start: ContainerCreating" I0705 21:51:10.965812 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-6wj7g pod in namespace openshift-ingress-canary (previous: false). I0705 21:51:11.165609 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-6wj7g pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-6wj7g\" is waiting to start: ContainerCreating" I0705 21:51:11.165625 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-6wj7g\" is waiting to start: ContainerCreating" I0705 21:51:11.165639 1 tasks_processing.go:74] worker 18 stopped. I0705 21:51:11.165707 1 recorder.go:75] Recording events/openshift-dns with fingerprint=a59b5f882de43ac15f5b72f9191c62e6ca3d82875a6b282629452c521553d616 I0705 21:51:11.165782 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=ce565a3f547345addf66a52cfb8d4f67e5baa47f804edcb539ee041f2dc5098e I0705 21:51:11.165809 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=a4547c32d485662ad9fe2991a551af37d2ac36189a62c9af4fbefba272558b82 I0705 21:51:11.165851 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=96ac71c9526a5454cf09a5eb513b7fd82ea585bf3d1e73eabe6aad138934325f I0705 21:51:11.165862 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=612f96f855150dd65661f8dbdb88fe831b0308e73bb8c4184af7b81d385b47a5 I0705 21:51:11.165869 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 2.655123885s to process 5 records W0705 21:51:11.532208 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0705 21:51:12.531958 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0705 21:51:13.528027 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0705 21:51:13.528051 1 tasks_processing.go:74] worker 19 stopped. E0705 21:51:13.528061 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0705 21:51:13.528072 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0705 21:51:13.528086 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0705 21:51:13.528095 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.017442229s to process 1 records I0705 21:51:21.153655 1 tasks_processing.go:74] worker 15 stopped. I0705 21:51:21.153691 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0705 21:51:21.153704 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.642063722s to process 1 records I0705 21:51:21.929714 1 tasks_processing.go:74] worker 30 stopped. I0705 21:51:21.929983 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=6eea9bca9c2d016e4128bf2b374fefdd9610f4d08552d21add7f6bb399a3af40 I0705 21:51:21.929999 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.408759181s to process 1 records E0705 21:51:21.930054 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.42s with: function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0705 21:51:21.931187 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0705 21:51:21.931207 1 periodic.go:209] Running workloads gatherer I0705 21:51:21.931227 1 tasks_processing.go:45] number of workers: 2 I0705 21:51:21.931238 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 21:51:21.931244 1 tasks_processing.go:71] worker 1 working on workload_info task. I0705 21:51:21.931252 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 21:51:21.931268 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0705 21:51:21.955246 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 19s for image data I0705 21:51:21.966961 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (12ms) I0705 21:51:21.975001 1 tasks_processing.go:74] worker 0 stopped. I0705 21:51:21.975020 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 43.719014ms to process 0 records I0705 21:51:21.979553 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (13ms) I0705 21:51:22.001589 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (22ms) I0705 21:51:22.010334 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (9ms) I0705 21:51:22.019859 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (10ms) I0705 21:51:22.029417 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0705 21:51:22.038445 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (9ms) I0705 21:51:22.047857 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (9ms) I0705 21:51:22.057123 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (9ms) I0705 21:51:22.066783 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (10ms) I0705 21:51:22.076106 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (9ms) I0705 21:51:22.165406 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (89ms) I0705 21:51:22.265064 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0705 21:51:22.365613 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (101ms) I0705 21:51:22.465311 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0705 21:51:22.565327 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0705 21:51:22.666150 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0705 21:51:22.766241 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0705 21:51:22.865538 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (99ms) I0705 21:51:22.953378 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 21:51:22.964299 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (99ms) I0705 21:51:23.064927 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (101ms) I0705 21:51:23.165195 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (100ms) I0705 21:51:23.265136 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0705 21:51:23.367014 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (102ms) I0705 21:51:23.468342 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0705 21:51:23.566004 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (98ms) I0705 21:51:23.627757 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 21:51:23.665260 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0705 21:51:23.767923 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (103ms) I0705 21:51:23.831364 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 21:51:23.865911 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (98ms) I0705 21:51:23.965687 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0705 21:51:24.065558 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0705 21:51:24.164979 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (99ms) I0705 21:51:24.165016 1 tasks_processing.go:74] worker 1 stopped. E0705 21:51:24.165028 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0705 21:51:24.165351 1 recorder.go:75] Recording config/workload_info with fingerprint=b1da8dd2efc56b6bed282a64bf1779915f305e5692c7eae8ac8d46250b4f4d9f I0705 21:51:24.165370 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.233763013s to process 1 records E0705 21:51:24.165396 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.234s with: function \"workload_info\" failed with an error" I0705 21:51:24.166511 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0705 21:51:24.166529 1 periodic.go:209] Running conditional gatherer I0705 21:51:24.173416 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0705 21:51:24.182905 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.14:42071->172.30.0.10:53: read: connection refused E0705 21:51:24.183260 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 21:51:24.183386 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0705 21:51:24.190293 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0705 21:51:24.190311 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190318 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190323 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190327 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190331 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190337 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190342 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190346 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 21:51:24.190350 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0705 21:51:24.190376 1 tasks_processing.go:45] number of workers: 3 I0705 21:51:24.190396 1 tasks_processing.go:69] worker 2 listening for tasks. I0705 21:51:24.190402 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0705 21:51:24.190452 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 21:51:24.190513 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0705 21:51:24.190461 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 21:51:24.190583 1 tasks_processing.go:74] worker 1 stopped. I0705 21:51:24.190470 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0705 21:51:24.190515 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0705 21:51:24.190778 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.282µs to process 1 records I0705 21:51:24.190856 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0705 21:51:24.190868 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.481µs to process 1 records I0705 21:51:24.190879 1 tasks_processing.go:74] worker 0 stopped. I0705 21:51:24.190940 1 tasks_processing.go:74] worker 2 stopped. I0705 21:51:24.190955 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 295.772µs to process 0 records I0705 21:51:24.190976 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.14:42071->172.30.0.10:53: read: connection refused I0705 21:51:24.190997 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0705 21:51:24.212539 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=24d27c338270fd91856dc98ed5e45c9aae05364a998431f4e1288a314c134ff0 I0705 21:51:24.212675 1 diskrecorder.go:70] Writing 96 records to /var/lib/insights-operator/insights-2026-07-05-215124.tar.gz I0705 21:51:24.221208 1 diskrecorder.go:51] Wrote 96 records to disk in 8ms I0705 21:51:24.221284 1 periodic.go:278] Gathering cluster info every 2h0m0s I0705 21:51:24.221331 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0705 21:51:37.274819 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 21:52:27.817371 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="d81fa0652a1618e858f823b4128213fa5524623f9cb8c121b5d725f24d2e42d8") W0705 21:52:27.817404 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0705 21:52:27.817448 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="18cf7bc7db777eed9aec65057915ec059ffe75e90ee63bbe168ccb97660c5ac0") I0705 21:52:27.817459 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0705 21:52:27.817473 1 base_controller.go:181] Shutting down ConfigController ... I0705 21:52:27.817504 1 periodic.go:170] Shutting down I0705 21:52:27.817523 1 base_controller.go:181] Shutting down LoggingSyncer ... I0705 21:52:27.817540 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0705 21:52:27.817537 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0705 21:52:27.817550 1 base_controller.go:113] All LoggingSyncer workers have been terminated I0705 21:52:27.817555 1 base_controller.go:113] All ConfigController workers have been terminated