W1002 02:41:29.979177 1 cmd.go:257] Using insecure, self-signed certificates I1002 02:41:30.149577 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I1002 02:41:30.149840 1 observer_polling.go:159] Starting file observer W1002 02:41:30.169770 1 builder.go:272] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-5ff5cb4f99" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I1002 02:41:30.290951 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I1002 02:41:30.291134 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I1002 02:41:30.291567 1 secure_serving.go:57] Forcing use of http/1.1 only I1002 02:41:30.291581 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W1002 02:41:30.291591 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W1002 02:41:30.291595 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W1002 02:41:30.291599 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W1002 02:41:30.291602 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W1002 02:41:30.291605 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W1002 02:41:30.291640 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I1002 02:41:30.294951 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes CPMSMachineNamePrefix ChunkSizeMiB CloudDualStackNodeIPs ConsolePluginContentSecurityPolicy DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags GatewayAPI GatewayAPIController HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy OnClusterBuild PersistentIPsForVirtualization PrivateHostedZoneAWS RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController VSphereDriverConfiguration VSphereMultiVCenters ValidatingAdmissionPolicy AWSClusterHostedDNS AutomatedEtcdBackup BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNS GCPCustomAPIEndpoints HighlyAvailableArbiter ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements SELinuxChangePolicy SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerification SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I1002 02:41:30.294968 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ChunkSizeMiB", "CloudDualStackNodeIPs", "ConsolePluginContentSecurityPolicy", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "GatewayAPI", "GatewayAPIController", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "OnClusterBuild", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "VSphereDriverConfiguration", "VSphereMultiVCenters", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AutomatedEtcdBackup", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "HighlyAvailableArbiter", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "SELinuxChangePolicy", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I1002 02:41:30.296027 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I1002 02:41:30.296038 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I1002 02:41:30.296052 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I1002 02:41:30.296057 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I1002 02:41:30.296068 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I1002 02:41:30.296077 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I1002 02:41:30.296543 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1433288152/tls.crt::/tmp/serving-cert-1433288152/tls.key" I1002 02:41:30.297024 1 secure_serving.go:213] Serving securely on [::]:8443 I1002 02:41:30.297061 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" E1002 02:41:30.298193 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18da9625a71e49d9 openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdditionalRoutingCapabilities\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"CPMSMachineNamePrefix\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"ConsolePluginContentSecurityPolicy\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"GatewayAPI\", \"GatewayAPIController\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MetricsCollectionProfiles\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"OnClusterBuild\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SetEIPForNLBIngressController\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ClusterVersionOperatorConfiguration\", \"DNSNameResolver\", \"DualReplica\", \"DyanmicServiceEndpointIBMCloud\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"Example2\", \"ExternalOIDCWithUIDAndExtraClaimMappings\", \"GCPClusterHostedDNS\", \"GCPCustomAPIEndpoints\", \"HighlyAvailableArbiter\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"KMSEncryptionProvider\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NewOLMCatalogdAPIV1Metas\", \"NewOLMOwnSingleNamespace\", \"NewOLMPreflightPermissionChecks\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"SELinuxChangePolicy\", \"SELinuxMount\", \"ShortCertRotation\", \"SignatureStores\", \"SigstoreImageVerification\", \"SigstoreImageVerificationPKI\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereConfigurableMaxAllowedBlockVolumesPerNode\", \"VSphereHostVMGroupZonal\", \"VSphereMultiDisk\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-10-02 02:41:30.294929881 +0000 UTC m=+0.345032631,LastTimestamp:2026-10-02 02:41:30.294929881 +0000 UTC m=+0.345032631,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W1002 02:41:30.300593 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I1002 02:41:30.300621 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I1002 02:41:30.300709 1 base_controller.go:76] Waiting for caches to sync for ConfigController I1002 02:41:30.306552 1 secretconfigobserver.go:249] Found cloud.openshift.com token I1002 02:41:30.306568 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I1002 02:41:30.314434 1 secretconfigobserver.go:119] support secret does not exist I1002 02:41:30.319220 1 secretconfigobserver.go:249] Found cloud.openshift.com token I1002 02:41:30.323415 1 secretconfigobserver.go:119] support secret does not exist I1002 02:41:30.324920 1 recorder.go:156] Pruning old reports every 8h19m53s, max age is 288h0m0s I1002 02:41:30.330971 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I1002 02:41:30.330996 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I1002 02:41:30.331004 1 insightsreport.go:296] Starting report retriever I1002 02:41:30.331009 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I1002 02:41:30.330996 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I1002 02:41:30.330981 1 periodic.go:212] Running clusterconfig gatherer I1002 02:41:30.331079 1 tasks_processing.go:45] number of workers: 64 I1002 02:41:30.331108 1 tasks_processing.go:69] worker 3 listening for tasks. I1002 02:41:30.331119 1 tasks_processing.go:69] worker 2 listening for tasks. I1002 02:41:30.331122 1 tasks_processing.go:69] worker 1 listening for tasks. I1002 02:41:30.331128 1 tasks_processing.go:69] worker 0 listening for tasks. I1002 02:41:30.331128 1 tasks_processing.go:69] worker 23 listening for tasks. I1002 02:41:30.331132 1 tasks_processing.go:69] worker 13 listening for tasks. I1002 02:41:30.331134 1 tasks_processing.go:71] worker 23 working on sap_pods task. I1002 02:41:30.331139 1 tasks_processing.go:69] worker 9 listening for tasks. I1002 02:41:30.331141 1 tasks_processing.go:69] worker 4 listening for tasks. I1002 02:41:30.331142 1 tasks_processing.go:69] worker 24 listening for tasks. I1002 02:41:30.331146 1 tasks_processing.go:69] worker 10 listening for tasks. I1002 02:41:30.331135 1 tasks_processing.go:69] worker 8 listening for tasks. I1002 02:41:30.331169 1 tasks_processing.go:69] worker 49 listening for tasks. I1002 02:41:30.331175 1 tasks_processing.go:69] worker 11 listening for tasks. I1002 02:41:30.331176 1 tasks_processing.go:69] worker 12 listening for tasks. I1002 02:41:30.331177 1 tasks_processing.go:69] worker 26 listening for tasks. I1002 02:41:30.331151 1 tasks_processing.go:69] worker 5 listening for tasks. I1002 02:41:30.331155 1 tasks_processing.go:69] worker 47 listening for tasks. I1002 02:41:30.331155 1 tasks_processing.go:69] worker 18 listening for tasks. I1002 02:41:30.331159 1 tasks_processing.go:69] worker 6 listening for tasks. I1002 02:41:30.331189 1 tasks_processing.go:69] worker 39 listening for tasks. I1002 02:41:30.331192 1 tasks_processing.go:69] worker 43 listening for tasks. I1002 02:41:30.331193 1 tasks_processing.go:69] worker 52 listening for tasks. I1002 02:41:30.331195 1 tasks_processing.go:69] worker 28 listening for tasks. I1002 02:41:30.331197 1 tasks_processing.go:69] worker 40 listening for tasks. I1002 02:41:30.331195 1 tasks_processing.go:69] worker 15 listening for tasks. I1002 02:41:30.331203 1 tasks_processing.go:69] worker 53 listening for tasks. I1002 02:41:30.331205 1 tasks_processing.go:69] worker 35 listening for tasks. I1002 02:41:30.331205 1 tasks_processing.go:69] worker 37 listening for tasks. I1002 02:41:30.331207 1 tasks_processing.go:69] worker 29 listening for tasks. I1002 02:41:30.331167 1 tasks_processing.go:69] worker 14 listening for tasks. I1002 02:41:30.331209 1 tasks_processing.go:69] worker 45 listening for tasks. I1002 02:41:30.331213 1 tasks_processing.go:69] worker 30 listening for tasks. I1002 02:41:30.331166 1 tasks_processing.go:69] worker 7 listening for tasks. I1002 02:41:30.331216 1 tasks_processing.go:69] worker 31 listening for tasks. I1002 02:41:30.331219 1 tasks_processing.go:69] worker 22 listening for tasks. I1002 02:41:30.331149 1 tasks_processing.go:69] worker 25 listening for tasks. I1002 02:41:30.331222 1 tasks_processing.go:69] worker 20 listening for tasks. I1002 02:41:30.331169 1 tasks_processing.go:69] worker 46 listening for tasks. I1002 02:41:30.331225 1 tasks_processing.go:71] worker 3 working on oauths task. I1002 02:41:30.331226 1 tasks_processing.go:71] worker 2 working on image task. I1002 02:41:30.331229 1 tasks_processing.go:71] worker 20 working on clusterroles task. I1002 02:41:30.331229 1 tasks_processing.go:69] worker 63 listening for tasks. I1002 02:41:30.331231 1 tasks_processing.go:71] worker 46 working on nodes task. I1002 02:41:30.331172 1 tasks_processing.go:69] worker 17 listening for tasks. I1002 02:41:30.331180 1 tasks_processing.go:69] worker 36 listening for tasks. I1002 02:41:30.331235 1 tasks_processing.go:69] worker 33 listening for tasks. I1002 02:41:30.331240 1 tasks_processing.go:71] worker 43 working on storage_cluster task. I1002 02:41:30.331241 1 tasks_processing.go:69] worker 59 listening for tasks. I1002 02:41:30.331238 1 tasks_processing.go:69] worker 19 listening for tasks. I1002 02:41:30.331248 1 tasks_processing.go:71] worker 28 working on support_secret task. I1002 02:41:30.331247 1 tasks_processing.go:71] worker 15 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I1002 02:41:30.331254 1 tasks_processing.go:71] worker 35 working on olm_operators task. I1002 02:41:30.331255 1 tasks_processing.go:71] worker 11 working on mutating_webhook_configurations task. I1002 02:41:30.331260 1 tasks_processing.go:71] worker 12 working on dvo_metrics task. I1002 02:41:30.331296 1 tasks_processing.go:71] worker 14 working on image_registries task. I1002 02:41:30.331302 1 tasks_processing.go:71] worker 63 working on machine_config_pools task. I1002 02:41:30.331301 1 tasks_processing.go:71] worker 18 working on aggregated_monitoring_cr_names task. I1002 02:41:30.331316 1 tasks_processing.go:71] worker 37 working on proxies task. I1002 02:41:30.331402 1 tasks_processing.go:71] worker 24 working on metrics task. I1002 02:41:30.331405 1 tasks_processing.go:71] worker 36 working on version task. I1002 02:41:30.331615 1 tasks_processing.go:71] worker 26 working on nodenetworkconfigurationpolicies task. I1002 02:41:30.331643 1 tasks_processing.go:71] worker 10 working on openstack_dataplanenodesets task. W1002 02:41:30.331694 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I1002 02:41:30.331733 1 tasks_processing.go:71] worker 24 working on crds task. I1002 02:41:30.331836 1 tasks_processing.go:71] worker 8 working on config_maps task. I1002 02:41:30.331255 1 tasks_processing.go:71] worker 47 working on operators_pods_and_events task. I1002 02:41:30.331935 1 tasks_processing.go:71] worker 30 working on nodenetworkstates task. I1002 02:41:30.331976 1 tasks_processing.go:71] worker 13 working on infrastructures task. I1002 02:41:30.332014 1 tasks_processing.go:71] worker 52 working on networks task. I1002 02:41:30.332050 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 322.604µs to process 0 records I1002 02:41:30.332059 1 tasks_processing.go:71] worker 4 working on machine_healthchecks task. I1002 02:41:30.331633 1 tasks_processing.go:71] worker 17 working on machine_sets task. I1002 02:41:30.332180 1 tasks_processing.go:71] worker 25 working on machine_autoscalers task. I1002 02:41:30.331649 1 tasks_processing.go:71] worker 59 working on service_accounts task. I1002 02:41:30.331245 1 tasks_processing.go:71] worker 29 working on node_logs task. I1002 02:41:30.331986 1 tasks_processing.go:71] worker 49 working on openstack_controlplanes task. I1002 02:41:30.332364 1 tasks_processing.go:71] worker 22 working on validating_webhook_configurations task. I1002 02:41:30.331935 1 tasks_processing.go:71] worker 9 working on ingress task. I1002 02:41:30.331833 1 tasks_processing.go:71] worker 39 working on openstack_version task. I1002 02:41:30.332494 1 tasks_processing.go:71] worker 31 working on authentication task. I1002 02:41:30.331187 1 tasks_processing.go:69] worker 42 listening for tasks. I1002 02:41:30.332524 1 tasks_processing.go:71] worker 42 working on cluster_apiserver task. I1002 02:41:30.331188 1 tasks_processing.go:69] worker 51 listening for tasks. I1002 02:41:30.331161 1 tasks_processing.go:69] worker 48 listening for tasks. I1002 02:41:30.331201 1 tasks_processing.go:69] worker 32 listening for tasks. I1002 02:41:30.331184 1 tasks_processing.go:69] worker 16 listening for tasks. I1002 02:41:30.331188 1 tasks_processing.go:69] worker 27 listening for tasks. I1002 02:41:30.331214 1 tasks_processing.go:69] worker 60 listening for tasks. I1002 02:41:30.331297 1 tasks_processing.go:71] worker 6 working on cost_management_metrics_configs task. I1002 02:41:30.332653 1 tasks_processing.go:71] worker 32 working on jaegers task. I1002 02:41:30.332677 1 tasks_processing.go:71] worker 51 working on sap_datahubs task. I1002 02:41:30.332727 1 tasks_processing.go:71] worker 48 working on ceph_cluster task. I1002 02:41:30.331183 1 tasks_processing.go:69] worker 41 listening for tasks. I1002 02:41:30.331135 1 tasks_processing.go:71] worker 0 working on schedulers task. I1002 02:41:30.332652 1 tasks_processing.go:71] worker 60 working on openstack_dataplanedeployments task. I1002 02:41:30.331719 1 tasks_processing.go:71] worker 40 working on openshift_machine_api_events task. I1002 02:41:30.332712 1 tasks_processing.go:71] worker 27 working on image_pruners task. I1002 02:41:30.331217 1 tasks_processing.go:69] worker 21 listening for tasks. I1002 02:41:30.331220 1 tasks_processing.go:69] worker 55 listening for tasks. I1002 02:41:30.331222 1 tasks_processing.go:69] worker 62 listening for tasks. I1002 02:41:30.331226 1 tasks_processing.go:69] worker 56 listening for tasks. I1002 02:41:30.331220 1 tasks_processing.go:69] worker 34 listening for tasks. I1002 02:41:30.331209 1 tasks_processing.go:69] worker 38 listening for tasks. I1002 02:41:30.331233 1 tasks_processing.go:69] worker 57 listening for tasks. I1002 02:41:30.331238 1 tasks_processing.go:69] worker 58 listening for tasks. I1002 02:41:30.333153 1 tasks_processing.go:71] worker 38 working on operators task. I1002 02:41:30.333158 1 tasks_processing.go:71] worker 58 working on silenced_alerts task. I1002 02:41:30.331250 1 tasks_processing.go:71] worker 53 working on container_images task. I1002 02:41:30.333177 1 tasks_processing.go:71] worker 56 working on storage_classes task. I1002 02:41:30.331243 1 tasks_processing.go:71] worker 1 working on install_plans task. I1002 02:41:30.331298 1 tasks_processing.go:71] worker 19 working on openshift_logging task. I1002 02:41:30.331408 1 tasks_processing.go:71] worker 33 working on overlapping_namespace_uids task. I1002 02:41:30.331182 1 tasks_processing.go:69] worker 50 listening for tasks. I1002 02:41:30.331850 1 tasks_processing.go:71] worker 7 working on machine_configs task. I1002 02:41:30.331905 1 tasks_processing.go:71] worker 45 working on certificate_signing_requests task. I1002 02:41:30.331932 1 tasks_processing.go:71] worker 5 working on sap_config task. I1002 02:41:30.331203 1 tasks_processing.go:69] worker 44 listening for tasks. I1002 02:41:30.331211 1 tasks_processing.go:69] worker 54 listening for tasks. I1002 02:41:30.331217 1 tasks_processing.go:69] worker 61 listening for tasks. I1002 02:41:30.332659 1 tasks_processing.go:71] worker 16 working on pod_network_connectivity_checks task. I1002 02:41:30.332876 1 tasks_processing.go:71] worker 41 working on pdbs task. I1002 02:41:30.333806 1 tasks_processing.go:71] worker 34 working on active_alerts task. W1002 02:41:30.333832 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I1002 02:41:30.333841 1 tasks_processing.go:74] worker 34 stopped. I1002 02:41:30.333848 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 24.211µs to process 0 records I1002 02:41:30.334216 1 tasks_processing.go:71] worker 21 working on lokistack task. I1002 02:41:30.334302 1 tasks_processing.go:71] worker 55 working on container_runtime_configs task. I1002 02:41:30.334351 1 tasks_processing.go:71] worker 62 working on machines task. I1002 02:41:30.334415 1 tasks_processing.go:71] worker 57 working on tsdb_status task. W1002 02:41:30.334436 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I1002 02:41:30.334444 1 tasks_processing.go:74] worker 57 stopped. I1002 02:41:30.334448 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 19.129µs to process 0 records W1002 02:41:30.334450 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I1002 02:41:30.334458 1 tasks_processing.go:74] worker 58 stopped. I1002 02:41:30.334462 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.293007ms to process 0 records I1002 02:41:30.334670 1 tasks_processing.go:71] worker 50 working on monitoring_persistent_volumes task. I1002 02:41:30.335039 1 tasks_processing.go:71] worker 44 working on feature_gates task. I1002 02:41:30.335120 1 tasks_processing.go:71] worker 54 working on ingress_certificates task. I1002 02:41:30.335161 1 tasks_processing.go:74] worker 26 stopped. I1002 02:41:30.335171 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 3.511594ms to process 0 records I1002 02:41:30.335177 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 3.92175ms to process 0 records I1002 02:41:30.335180 1 tasks_processing.go:74] worker 43 stopped. I1002 02:41:30.335201 1 tasks_processing.go:74] worker 4 stopped. I1002 02:41:30.335209 1 tasks_processing.go:74] worker 61 stopped. E1002 02:41:30.335211 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I1002 02:41:30.335217 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 3.097291ms to process 0 records I1002 02:41:30.336311 1 tasks_processing.go:74] worker 30 stopped. I1002 02:41:30.336340 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 4.327857ms to process 0 records I1002 02:41:30.336491 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I1002 02:41:30.336509 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I1002 02:41:30.336518 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I1002 02:41:30.336522 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I1002 02:41:30.336534 1 controller.go:458] The operator is still being initialized I1002 02:41:30.336541 1 controller.go:481] The operator is healthy I1002 02:41:30.344159 1 tasks_processing.go:74] worker 49 stopped. I1002 02:41:30.344168 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 11.809254ms to process 0 records I1002 02:41:30.344178 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 12.477902ms to process 0 records I1002 02:41:30.344183 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 11.973239ms to process 0 records I1002 02:41:30.344188 1 tasks_processing.go:74] worker 10 stopped. I1002 02:41:30.344187 1 tasks_processing.go:74] worker 25 stopped. I1002 02:41:30.344296 1 tasks_processing.go:74] worker 51 stopped. I1002 02:41:30.344310 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 11.56091ms to process 0 records I1002 02:41:30.344316 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 12.113003ms to process 0 records I1002 02:41:30.344350 1 tasks_processing.go:74] worker 17 stopped. I1002 02:41:30.344411 1 tasks_processing.go:74] worker 2 stopped. I1002 02:41:30.344478 1 recorder.go:70] Recording config/image with fingerprint=02911a88beb3fc208cd942b53682dadc2aa510505ad97210849a83631b0fb6a7 I1002 02:41:30.344492 1 gather.go:177] gatherer "clusterconfig" function "image" took 13.080422ms to process 1 records I1002 02:41:30.344532 1 tasks_processing.go:74] worker 3 stopped. I1002 02:41:30.344794 1 recorder.go:70] Recording config/oauth with fingerprint=0b6b78bb064216e25f717eb2f025e9c9c8cb5fbe468bd7f1b0c5a397615e2b0e I1002 02:41:30.344805 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 13.293798ms to process 1 records I1002 02:41:30.345851 1 tasks_processing.go:74] worker 39 stopped. I1002 02:41:30.345860 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 13.443729ms to process 0 records I1002 02:41:30.348205 1 tasks_processing.go:74] worker 60 stopped. I1002 02:41:30.348223 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 15.275865ms to process 0 records I1002 02:41:30.348233 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 16.905981ms to process 0 records I1002 02:41:30.348237 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 17.085578ms to process 0 records I1002 02:41:30.348243 1 tasks_processing.go:74] worker 23 stopped. I1002 02:41:30.348246 1 tasks_processing.go:74] worker 63 stopped. I1002 02:41:30.348507 1 tasks_processing.go:74] worker 9 stopped. I1002 02:41:30.348684 1 recorder.go:70] Recording config/ingress with fingerprint=57d45ecd076f75a186fe9e8310d73c5a2a99cd13c5ae497fd53155ed2fb0959d I1002 02:41:30.348698 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 16.104481ms to process 1 records I1002 02:41:30.348790 1 tasks_processing.go:74] worker 52 stopped. I1002 02:41:30.348833 1 recorder.go:70] Recording config/network with fingerprint=8afb3faf6838505ca428b5820c49c9e8768c79c79c0312b5fe25bfafebc1fb2a I1002 02:41:30.348844 1 gather.go:177] gatherer "clusterconfig" function "networks" took 16.58667ms to process 1 records I1002 02:41:30.348923 1 tasks_processing.go:74] worker 31 stopped. I1002 02:41:30.349002 1 recorder.go:70] Recording config/authentication with fingerprint=8c0e3c257211cf855153dae9eeae4219331ff16086735faad2a72424961d6878 I1002 02:41:30.349012 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 16.288935ms to process 1 records I1002 02:41:30.349101 1 tasks_processing.go:74] worker 14 stopped. I1002 02:41:30.349448 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=f69e60293cf34f10d0725c5a02219a6791e3102fe1ee5a21dec35f659dcc6481 I1002 02:41:30.349461 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 17.627991ms to process 1 records I1002 02:41:30.350561 1 tasks_processing.go:74] worker 28 stopped. E1002 02:41:30.350573 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I1002 02:41:30.350579 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 19.304005ms to process 0 records I1002 02:41:30.350669 1 tasks_processing.go:74] worker 27 stopped. I1002 02:41:30.350742 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=7fed2f26534aad8274d12fc7685621cbaf7ff251c52e5f98b22356dc44cc0158 I1002 02:41:30.350755 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 17.616144ms to process 1 records I1002 02:41:30.352345 1 tasks_processing.go:74] worker 32 stopped. I1002 02:41:30.352359 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 19.633703ms to process 0 records I1002 02:41:30.352672 1 tasks_processing.go:74] worker 6 stopped. I1002 02:41:30.352685 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 20.069727ms to process 0 records I1002 02:41:30.364313 1 tasks_processing.go:74] worker 55 stopped. I1002 02:41:30.364345 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 29.995337ms to process 0 records I1002 02:41:30.364353 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 31.128369ms to process 0 records I1002 02:41:30.364357 1 tasks_processing.go:74] worker 19 stopped. I1002 02:41:30.364445 1 tasks_processing.go:74] worker 16 stopped. E1002 02:41:30.364457 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I1002 02:41:30.364476 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 31.039216ms to process 0 records I1002 02:41:30.364535 1 tasks_processing.go:74] worker 48 stopped. I1002 02:41:30.364550 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 31.76822ms to process 0 records I1002 02:41:30.364572 1 tasks_processing.go:74] worker 21 stopped. I1002 02:41:30.364582 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 30.345036ms to process 0 records I1002 02:41:30.364624 1 tasks_processing.go:74] worker 62 stopped. E1002 02:41:30.364636 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I1002 02:41:30.364644 1 gather.go:177] gatherer "clusterconfig" function "machines" took 30.255869ms to process 0 records I1002 02:41:30.365282 1 tasks_processing.go:74] worker 11 stopped. I1002 02:41:30.365462 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=5ba9bc646fc8d44e3fd807c9d5b058c15ac962abb97b9a073d7820fa594ecf5f I1002 02:41:30.365491 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=6e4389f70ae972810f9d380b0ba78a537095d8e862f549b48fd449851412b28d I1002 02:41:30.365516 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=be21a64e7325d7970c964d434d0e8682ea283f26faaf6551db74a365cb1e66fa I1002 02:41:30.365525 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 34.014014ms to process 3 records I1002 02:41:30.365606 1 tasks_processing.go:74] worker 13 stopped. I1002 02:41:30.366020 1 recorder.go:70] Recording config/infrastructure with fingerprint=761a5ab052092e59d6f2a3721de7239e465e886717519dbc0845f62d275280d7 I1002 02:41:30.366029 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 33.444437ms to process 1 records I1002 02:41:30.368309 1 tasks_processing.go:74] worker 40 stopped. I1002 02:41:30.368322 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 35.335126ms to process 0 records I1002 02:41:30.368370 1 tasks_processing.go:74] worker 42 stopped. I1002 02:41:30.368514 1 recorder.go:70] Recording config/apiserver with fingerprint=a11d45823af55b051484de7e1b052ac7b62cfa1224d23fa3b14a04203b7b26f5 I1002 02:41:30.368529 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 35.839792ms to process 1 records I1002 02:41:30.368595 1 recorder.go:70] Recording config/proxy with fingerprint=1ab6078f4779fc45daa23c5ccbcc24028e71935b56a2cba1f56ad9302415e768 I1002 02:41:30.368609 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 37.141727ms to process 1 records I1002 02:41:30.368616 1 tasks_processing.go:74] worker 37 stopped. I1002 02:41:30.368676 1 tasks_processing.go:74] worker 5 stopped. I1002 02:41:30.368689 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 35.360989ms to process 0 records I1002 02:41:30.370300 1 tasks_processing.go:74] worker 50 stopped. I1002 02:41:30.370323 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 35.618885ms to process 0 records I1002 02:41:30.370372 1 tasks_processing.go:74] worker 56 stopped. I1002 02:41:30.370572 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=72b01e44e543dbf642df5a439ef018c06f9d3128a43c21adfc1291bfdc06ded4 I1002 02:41:30.370594 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=3236ad83bc4b4103ef4abd7c3a61bfa8dd9279a494860c8b958664b4e20f437e I1002 02:41:30.370599 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 37.152124ms to process 2 records I1002 02:41:30.371699 1 tasks_processing.go:74] worker 24 stopped. I1002 02:41:30.372982 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=2afeb266e31c6feb00958186e0fc4b4c00056c49672a3d85328264823206a464 I1002 02:41:30.373154 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=58b7f83dcd4cfa63b2740b2bc8ace9334433a15d247515a7362435ce9901fba4 I1002 02:41:30.373164 1 gather.go:177] gatherer "clusterconfig" function "crds" took 39.946684ms to process 2 records I1002 02:41:30.373245 1 tasks_processing.go:74] worker 44 stopped. I1002 02:41:30.373248 1 recorder.go:70] Recording config/featuregate with fingerprint=8055ed155eba97f91ee45c3c24a851dd01a3b4d9a523e5ccb02cd25e31d6ad75 I1002 02:41:30.373265 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 36.655332ms to process 1 records I1002 02:41:30.373347 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=8237d6ec187f63a2ffd4a3a1c303525057238dd00f2154e586c35e64a90cfcc4 I1002 02:41:30.373348 1 tasks_processing.go:74] worker 0 stopped. I1002 02:41:30.373354 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 38.850359ms to process 1 records I1002 02:41:30.373431 1 tasks_processing.go:74] worker 22 stopped. I1002 02:41:30.373438 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=db3666d490acb57b9066d8112c348d048f677863d27ba33e3443ed4cb0f07016 I1002 02:41:30.373479 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=6703b334541ecea2b853b96fe72782ae8d3d80f2d0c6b820ef1735be8ace87fc I1002 02:41:30.373496 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=980b1c2253bde8f4d4f92934b315b3195a6921b1c6347cc8a4b85420da14544b I1002 02:41:30.373516 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=2339f920820292e975c962734f2a3c54e51336424aee35105d67b2b2afa03db6 I1002 02:41:30.373533 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=8eef731e43a7d1e8ffa6f7c8a6a28fe8ff5de18c2b2fd0b58779bc77714f1c50 I1002 02:41:30.373548 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=7b37e05747b4a0311410d887389e96da7e5f3d1876178d3d88fdbcf49c591b81 I1002 02:41:30.373566 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=8443f8008e2c707c7f6b8e8f5127db08ada18fc455badc886e5207fa802f2e09 I1002 02:41:30.373595 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=cf264da95d06d502cd034c09c05a9d39edddf7f1b302375f53fe1a3d578ad905 I1002 02:41:30.373615 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=93b4afc0bea71f5a446369d39a2676b9d0c3a2e698462062ecfed165f962e604 I1002 02:41:30.373637 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=51eec6f1e2f0c193768f5a79a39e30a6c326d5a4d98eed1917851f5aaeb67df4 I1002 02:41:30.373665 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=1a64911396953ae07b3c57933fca5385dd763ec3fef5ff95009de1866c476897 I1002 02:41:30.373674 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 39.762864ms to process 11 records E1002 02:41:30.373687 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I1002 02:41:30.373706 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I1002 02:41:30.373713 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 39.767233ms to process 1 records I1002 02:41:30.373717 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 40.01278ms to process 0 records I1002 02:41:30.373722 1 tasks_processing.go:74] worker 45 stopped. I1002 02:41:30.373727 1 tasks_processing.go:74] worker 7 stopped. I1002 02:41:30.375703 1 tasks_processing.go:74] worker 29 stopped. I1002 02:41:30.375717 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 43.449869ms to process 0 records I1002 02:41:30.375733 1 tasks_processing.go:74] worker 41 stopped. I1002 02:41:30.375844 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=85960b13896ba77dcda91aef6b194fec7a5503ce04ee33e7f714998f8f0ecda2 I1002 02:41:30.375866 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=eaef0ed0db2cee0db09e11b60b82aef820b048a3d70071065b711ef5bd3390d7 I1002 02:41:30.375885 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=e06e02eb679b920fd534069df890f0ebe2d2cef5c3b2960ac70d860b1ccb116a I1002 02:41:30.375898 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 42.261079ms to process 3 records I1002 02:41:30.376437 1 tasks_processing.go:74] worker 33 stopped. I1002 02:41:30.376506 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I1002 02:41:30.376518 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 43.203141ms to process 1 records I1002 02:41:30.377131 1 tasks_processing.go:74] worker 46 stopped. I1002 02:41:30.377974 1 recorder.go:70] Recording config/node/ip-10-0-128-203.ec2.internal with fingerprint=5e1d3b804aa5a33bb197d2e058cc7bc8b618c77bef61f928c3facb36bf869849 I1002 02:41:30.378088 1 recorder.go:70] Recording config/node/ip-10-0-158-191.ec2.internal with fingerprint=0b257639b7fe858bfde9930935c6e19dbb5276047e80a6970b33225cfe6eaed0 I1002 02:41:30.378178 1 recorder.go:70] Recording config/node/ip-10-0-162-3.ec2.internal with fingerprint=d894be44b2de2cd53e7ff8725c1aad27d2ecb8c8ecd0c11475229aba35ab43a2 I1002 02:41:30.378192 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 45.886054ms to process 3 records W1002 02:41:30.380999 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I1002 02:41:30.383396 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I1002 02:41:30.383466 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W1002 02:41:30.383519 1 operator.go:287] started I1002 02:41:30.383545 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I1002 02:41:30.392247 1 tasks_processing.go:74] worker 15 stopped. I1002 02:41:30.392262 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 60.987047ms to process 0 records I1002 02:41:30.392390 1 tasks_processing.go:74] worker 20 stopped. I1002 02:41:30.392565 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=cbf46ebbed4603dfcf1b8bb2002dc60ab2cea8f01c979e819c4e0b642314f40a I1002 02:41:30.392619 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=a9ac36e1d3e28503e1aa8cf7d8cd15dca3d7e470dd43726efe45d96ca5da3fdf I1002 02:41:30.392626 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 61.150661ms to process 2 records I1002 02:41:30.392803 1 tasks_processing.go:74] worker 35 stopped. I1002 02:41:30.392847 1 recorder.go:70] Recording config/olm_operators with fingerprint=61f9411c7f81ca84473e306064f88c9b78c791db54b02b1d23ea7c4d386bb1a4 I1002 02:41:30.392857 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 61.537584ms to process 1 records I1002 02:41:30.396165 1 tasks_processing.go:74] worker 53 stopped. I1002 02:41:30.396180 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I1002 02:41:30.396183 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I1002 02:41:30.396198 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I1002 02:41:30.396219 1 recorder.go:70] Recording config/running_containers with fingerprint=a9fa2281da783ca1704315c9086df187aefa1094df443280476e6aa33a237c95 I1002 02:41:30.396231 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 62.975395ms to process 1 records I1002 02:41:30.397476 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I1002 02:41:30.397487 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I1002 02:41:30.397491 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I1002 02:41:30.397497 1 controller.go:203] Source scaController *sca.Controller is not ready I1002 02:41:30.397501 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I1002 02:41:30.397516 1 controller.go:458] The operator is still being initialized I1002 02:41:30.397523 1 controller.go:481] The operator is healthy I1002 02:41:30.400283 1 tasks_processing.go:74] worker 18 stopped. I1002 02:41:30.400298 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 68.967911ms to process 0 records I1002 02:41:30.401099 1 base_controller.go:82] Caches are synced for ConfigController I1002 02:41:30.401109 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I1002 02:41:30.401644 1 prometheus_rules.go:88] Prometheus rules successfully created I1002 02:41:30.408137 1 configmapobserver.go:84] configmaps "insights-config" not found E1002 02:41:30.409478 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27e6f53516-59e0-4bdc-9929-181931082ae5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:46246->172.30.0.10:53: read: connection refused I1002 02:41:30.409490 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27e6f53516-59e0-4bdc-9929-181931082ae5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:46246->172.30.0.10:53: read: connection refused I1002 02:41:30.418987 1 tasks_processing.go:74] worker 8 stopped. E1002 02:41:30.418999 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E1002 02:41:30.419003 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E1002 02:41:30.419007 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I1002 02:41:30.419013 1 recorder.go:70] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=295a2bc7810c628251f1f312c209b5eb4da888326706c635ac0c12aec0969e17 I1002 02:41:30.419039 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I1002 02:41:30.419045 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I1002 02:41:30.419049 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=fe7abfa7f7aea852d8bca6b7df9b4d5de32045254ff2774f9bd30f0b6dcb7dc4 I1002 02:41:30.419053 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I1002 02:41:30.419069 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I1002 02:41:30.419075 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I1002 02:41:30.419079 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 87.097172ms to process 7 records I1002 02:41:30.452586 1 tasks_processing.go:74] worker 36 stopped. I1002 02:41:30.452973 1 recorder.go:70] Recording config/version with fingerprint=3161b8feb2e2b0a5deeffb65acdbdc99bd630b95a2b2c0458cf79c251578e109 I1002 02:41:30.452986 1 recorder.go:70] Recording config/id with fingerprint=750d4841f17c8deec452fe9a9d5b4ca704e540cba2db09b69c1805eac992fb94 I1002 02:41:30.452992 1 gather.go:177] gatherer "clusterconfig" function "version" took 120.977116ms to process 2 records I1002 02:41:30.483793 1 base_controller.go:82] Caches are synced for LoggingSyncer I1002 02:41:30.483808 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I1002 02:41:30.485562 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W1002 02:41:30.488375 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:49021->172.30.0.10:53: read: connection refused I1002 02:41:30.488391 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:49021->172.30.0.10:53: read: connection refused I1002 02:41:30.494424 1 tasks_processing.go:74] worker 54 stopped. E1002 02:41:30.494438 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E1002 02:41:30.494443 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2t6cdgqk3js83u7ba6jeliv9o2kgm8c6-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2t6cdgqk3js83u7ba6jeliv9o2kgm8c6-primary-cert-bundle-secret" not found I1002 02:41:30.494513 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=674248ffa6db65051d4695a6c3d4c2fb78951a6a90d03fbfd60bf721b5fdf201 I1002 02:41:30.494526 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 159.284009ms to process 1 records I1002 02:41:30.778226 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I1002 02:41:30.778240 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I1002 02:41:30.778583 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-gzvwg pod in namespace openshift-dns (previous: false). I1002 02:41:30.802309 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I1002 02:41:31.001756 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I1002 02:41:31.007510 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-gzvwg pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-gzvwg\" is waiting to start: ContainerCreating" I1002 02:41:31.007523 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-gzvwg\" is waiting to start: ContainerCreating" I1002 02:41:31.007530 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-gzvwg pod in namespace openshift-dns (previous: false). I1002 02:41:31.193515 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-gzvwg pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-gzvwg\" is waiting to start: ContainerCreating" I1002 02:41:31.193533 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-gzvwg\" is waiting to start: ContainerCreating" I1002 02:41:31.193560 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-kt9gr pod in namespace openshift-dns (previous: false). W1002 02:41:31.378925 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I1002 02:41:31.436974 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kt9gr pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-kt9gr\" is waiting to start: ContainerCreating" I1002 02:41:31.436992 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-kt9gr\" is waiting to start: ContainerCreating" I1002 02:41:31.436999 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-kt9gr pod in namespace openshift-dns (previous: false). I1002 02:41:31.584361 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kt9gr pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-kt9gr\" is waiting to start: ContainerCreating" I1002 02:41:31.584379 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-kt9gr\" is waiting to start: ContainerCreating" I1002 02:41:31.584390 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-zt5zt pod in namespace openshift-dns (previous: false). I1002 02:41:31.798523 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-zt5zt pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-zt5zt\" is waiting to start: ContainerCreating" I1002 02:41:31.798565 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-zt5zt\" is waiting to start: ContainerCreating" I1002 02:41:31.798575 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-zt5zt pod in namespace openshift-dns (previous: false). I1002 02:41:31.803584 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I1002 02:41:31.989363 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-zt5zt pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-zt5zt\" is waiting to start: ContainerCreating" I1002 02:41:31.989379 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-zt5zt\" is waiting to start: ContainerCreating" I1002 02:41:31.989387 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-9lqlj pod in namespace openshift-dns (previous: false). I1002 02:41:32.182003 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I1002 02:41:32.182022 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-qqkrc pod in namespace openshift-dns (previous: false). I1002 02:41:32.206423 1 tasks_processing.go:74] worker 38 stopped. I1002 02:41:32.206499 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=8f8c847630fe3c956d24551eb417f0bc065a6d86ae804d5f895d9fcf227939cb I1002 02:41:32.206541 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=80c1e26b982d87b8141b782c5b749d7f8103db0e97c8e6ccd0461e0299e31c1d I1002 02:41:32.206576 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I1002 02:41:32.206608 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=1fcf5a5abc273661827da7b214e1918691e498cde91a8a9beac18501c64e64d2 I1002 02:41:32.206638 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I1002 02:41:32.206668 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=b9a2c6a726ae40d210f0d7960ca1b2c4ab0666c11f48f31530f499fe3a259da4 I1002 02:41:32.206700 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=bd81bce6edca8f8745b186322186cf995135d23697c93e69382c79f359f4049f I1002 02:41:32.206725 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=f06e729f695d78b817c4b51b0c3d88a822a09c970ccc34ebb1da084f2b68ef66 I1002 02:41:32.206745 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=47bbe53882d3d453279216cd8bd866e0a2db790d89a2b2de26ddab8912ce4d5c I1002 02:41:32.206766 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=8c66ef9a3ae8b8d8b2405fb43be3eaad1e7b79ee2e8477a0bba8fad340f71f0b I1002 02:41:32.206778 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I1002 02:41:32.206795 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=27cdfc4e7a92088ba813ddb5537f319c2ef4940d77f13706ad3b2ec0cf745327 I1002 02:41:32.206809 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I1002 02:41:32.206824 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=df51da007cd111787428e8ee3aa3920b2eb996063d49650b6bf2281a4d15ad66 I1002 02:41:32.206835 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I1002 02:41:32.206850 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=be51a683326aa628b6fbcdf08ce9721846877575559bf17c251bddc297ef17bf I1002 02:41:32.206862 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I1002 02:41:32.206878 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=bcbc2b4ccdbb2a32ddfca09116f27ec90515d23b1963befae9fd80bfb26c3856 I1002 02:41:32.206956 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=2e846cafa03e718eeeddcd70499e1861e9e668ab96060350e15f824c1baf3fd6 I1002 02:41:32.206981 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I1002 02:41:32.207004 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=09b3fb1b909e2a9b274b87473305272d998091d1dca516813955b7ba4e484527 I1002 02:41:32.207027 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=756f27a7d63ea6057626f7d2a22c64b00556161468503455e8011b80f62a34f6 I1002 02:41:32.207040 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I1002 02:41:32.207056 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=2040c0bd6296209d992a666de7a170fe3fdf782b48a5db18c69d904e6294a1c2 I1002 02:41:32.207068 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I1002 02:41:32.207103 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=7ed40c4d9080834d8fea0220ce341a30bcd3719865b5b38cb5422a39b414b4bc I1002 02:41:32.207142 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=c7d9a87c62b15bfd186a954fa865baf40a6119e5fb69cdcf7c3a90776817cbe3 I1002 02:41:32.207161 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=436be27b53368e61c0701bc240007d59ddbaec1661e216488ed0abd935f98a9c I1002 02:41:32.207182 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=2a8c8aa5010f175313a4b6b3e6da3999fe57332f8d5d81e91f2d73411db02d39 I1002 02:41:32.207200 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=96f014e6c00b96b4797b296a7edcf3ed66a6505feaf3a76fa3b47404f223d20b I1002 02:41:32.207225 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=45fad49d15237210f902224cd606e5bc2d734cdd2bd5aeac4f03209b407876a6 I1002 02:41:32.207245 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I1002 02:41:32.207257 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I1002 02:41:32.207266 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.873255179s to process 33 records W1002 02:41:32.378870 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I1002 02:41:32.385408 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I1002 02:41:32.385422 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-spj58 pod in namespace openshift-dns (previous: false). I1002 02:41:32.585819 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I1002 02:41:32.585871 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5687596d5d-lc9jc pod in namespace openshift-image-registry (previous: false). I1002 02:41:32.782717 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5687596d5d-lc9jc pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5687596d5d-lc9jc\" is waiting to start: ContainerCreating" I1002 02:41:32.782734 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5687596d5d-lc9jc\" is waiting to start: ContainerCreating" I1002 02:41:32.782763 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5687596d5d-p2xd8 pod in namespace openshift-image-registry (previous: false). I1002 02:41:32.983866 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5687596d5d-p2xd8 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5687596d5d-p2xd8\" is waiting to start: ContainerCreating" I1002 02:41:32.983880 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5687596d5d-p2xd8\" is waiting to start: ContainerCreating" I1002 02:41:32.983909 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-664db85869-gtl47 pod in namespace openshift-image-registry (previous: false). I1002 02:41:33.184450 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-664db85869-gtl47 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-664db85869-gtl47\" is waiting to start: ContainerCreating" I1002 02:41:33.184478 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-664db85869-gtl47\" is waiting to start: ContainerCreating" I1002 02:41:33.184491 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-47s67 pod in namespace openshift-image-registry (previous: false). W1002 02:41:33.379237 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I1002 02:41:33.384357 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I1002 02:41:33.384371 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-hhp7f pod in namespace openshift-image-registry (previous: false). I1002 02:41:33.584269 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I1002 02:41:33.584286 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-lkxv5 pod in namespace openshift-image-registry (previous: false). I1002 02:41:33.781586 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I1002 02:41:33.781603 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6758585f4c-dhsmn pod in namespace openshift-ingress (previous: false). I1002 02:41:33.982548 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6758585f4c-dhsmn pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6758585f4c-dhsmn\" is waiting to start: ContainerCreating" I1002 02:41:33.982563 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6758585f4c-dhsmn\" is waiting to start: ContainerCreating" I1002 02:41:33.982572 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6758585f4c-tmbnf pod in namespace openshift-ingress (previous: false). I1002 02:41:34.189632 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6758585f4c-tmbnf pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6758585f4c-tmbnf\" is waiting to start: ContainerCreating" I1002 02:41:34.189649 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6758585f4c-tmbnf\" is waiting to start: ContainerCreating" I1002 02:41:34.189657 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7c578c9bd5-gspcf pod in namespace openshift-ingress (previous: false). W1002 02:41:34.378971 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I1002 02:41:34.382482 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7c578c9bd5-gspcf pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7c578c9bd5-gspcf\" is waiting to start: ContainerCreating" I1002 02:41:34.382498 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7c578c9bd5-gspcf\" is waiting to start: ContainerCreating" I1002 02:41:34.382507 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-4hnqp pod in namespace openshift-ingress-canary (previous: false). I1002 02:41:34.583559 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-4hnqp pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-4hnqp\" is waiting to start: ContainerCreating" I1002 02:41:34.583574 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-4hnqp\" is waiting to start: ContainerCreating" I1002 02:41:34.583601 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-csl7d pod in namespace openshift-ingress-canary (previous: false). I1002 02:41:34.784440 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-csl7d pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-csl7d\" is waiting to start: ContainerCreating" I1002 02:41:34.784456 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-csl7d\" is waiting to start: ContainerCreating" I1002 02:41:34.784473 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-ksm4f pod in namespace openshift-ingress-canary (previous: false). I1002 02:41:34.982931 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-ksm4f pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-ksm4f\" is waiting to start: ContainerCreating" I1002 02:41:34.982946 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-ksm4f\" is waiting to start: ContainerCreating" I1002 02:41:34.982961 1 tasks_processing.go:74] worker 47 stopped. I1002 02:41:34.983038 1 recorder.go:70] Recording events/openshift-dns with fingerprint=8d30efa1871f072f3d95fce51ac923147018ab7f93ab101c30e91c4d7671fd63 I1002 02:41:34.983092 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=8923bb4b0fd121dda32aa39e400dd86260982fdcb647e887c1390ee59ad297d3 I1002 02:41:34.983110 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=f595660b1d6fcfe57be0dfde7327eead9543c3e0f4995b031fed9c81efe1e4f9 I1002 02:41:34.983139 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=ea0584bfd5727ee72e5d1683ed5e34aa65ffb8a1ec4ed34991046efa015ff539 I1002 02:41:34.983151 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=8add5241bf09f5afefa4db221d99d138b8892363ff36a5d17ae2d3ac9e1c14bb I1002 02:41:34.984310 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-kt9gr with fingerprint=b1d1696c8555f9da4e3a8f18daedc035cf61a56283d616ad43f825a996170893 I1002 02:41:34.984400 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-5687596d5d-lc9jc with fingerprint=9984945cdcea65261675ff39ee663b918e4453a6fbcd13f972f46a4a2912e540 I1002 02:41:34.984457 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-5687596d5d-p2xd8 with fingerprint=3b5157cfc664606927991b4913e8913e5a153c2524114c4f1d4a7d4044f7d5f1 I1002 02:41:34.984517 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-664db85869-gtl47 with fingerprint=cbdc1ac3be97e525fcf8e88d8d7ee47d87521c0381b0b6856232600288e9efa8 I1002 02:41:34.984557 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-csl7d with fingerprint=03ab562cc6ccf57ac111fb7165dd5be94a425d74e41a80cf50844130096a1742 I1002 02:41:34.984565 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.65103191s to process 10 records W1002 02:41:35.376558 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I1002 02:41:35.376584 1 tasks_processing.go:74] worker 12 stopped. E1002 02:41:35.376595 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I1002 02:41:35.376607 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W1002 02:41:35.376618 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I1002 02:41:35.376627 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.045295884s to process 1 records I1002 02:41:37.546549 1 configmapobserver.go:84] configmaps "insights-config" not found I1002 02:41:42.981666 1 tasks_processing.go:74] worker 1 stopped. I1002 02:41:42.981706 1 recorder.go:70] Recording config/installplans with fingerprint=6f9781df213f917044fe07f29e6836f4c98c6ccf7740910e145b68b90ff5234d I1002 02:41:42.981717 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.648436495s to process 1 records I1002 02:41:43.738775 1 tasks_processing.go:74] worker 59 stopped. I1002 02:41:43.738980 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=cf9c12fe8737f3a6d4cb9da132f43361fa8e8d22eb1cc0d002bd7e06234de064 I1002 02:41:43.738994 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.406562455s to process 1 records E1002 02:41:43.739046 1 periodic.go:250] "Unhandled Error" err="clusterconfig failed after 13.408s with: function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I1002 02:41:43.740146 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I1002 02:41:43.740159 1 periodic.go:212] Running workloads gatherer I1002 02:41:43.740173 1 tasks_processing.go:45] number of workers: 2 I1002 02:41:43.740179 1 tasks_processing.go:69] worker 1 listening for tasks. I1002 02:41:43.740182 1 tasks_processing.go:71] worker 1 working on workload_info task. I1002 02:41:43.740189 1 tasks_processing.go:69] worker 0 listening for tasks. I1002 02:41:43.740276 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I1002 02:41:43.762222 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I1002 02:41:43.768231 1 tasks_processing.go:74] worker 0 stopped. I1002 02:41:43.768246 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 27.939514ms to process 0 records I1002 02:41:43.769621 1 gather_workloads_info.go:387] No image sha256:bae7a33f8db8a3d4b3c4c05498aba85a0ce463f85322067c48e79663710e616e (8ms) I1002 02:41:43.778246 1 gather_workloads_info.go:387] No image sha256:a7c71f3c9714cf1717c45e12ec817be6d6209f00989e6a31d634e527f0c8147d (9ms) I1002 02:41:43.785858 1 gather_workloads_info.go:387] No image sha256:798001e24f1ed6a5621c442729a840ba2a2abc43433901e8450caa12b4bad5c5 (8ms) I1002 02:41:43.793639 1 gather_workloads_info.go:387] No image sha256:ccdda82b4adfa8ee4a84b3bad19693be6a42a9881a7b83788d835912129c49a4 (8ms) I1002 02:41:43.804339 1 gather_workloads_info.go:387] No image sha256:8b1e32ce43eb1247b0558230b6e0baa85ac62f1a5e2a2089d40b4fea90538529 (11ms) I1002 02:41:43.812051 1 gather_workloads_info.go:387] No image sha256:14703f73bb1ca69ea03b726e340b3b68f8e294e1f22c80d1a16666ea3d4a88a3 (8ms) I1002 02:41:43.819675 1 gather_workloads_info.go:387] No image sha256:18956d5fbc8d0369a26cb1e00eb5c3aff1973ae8cbd3983920a0cf2851f05358 (8ms) I1002 02:41:43.827017 1 gather_workloads_info.go:387] No image sha256:9697bc2258bdfa9ae8c1866cc7eb0b3b46851998db827b121bdf77417a881eb3 (7ms) I1002 02:41:43.834368 1 gather_workloads_info.go:387] No image sha256:87efe06afd1f04426fca7f86c0f74c4ee75c311ba199ddabd5b849b877bc59fa (7ms) I1002 02:41:43.841772 1 gather_workloads_info.go:387] No image sha256:099b3af427b8e1eb44765283a640446c3225151baa06eed789623206151a60cd (7ms) I1002 02:41:43.870077 1 gather_workloads_info.go:387] No image sha256:ceab8a8c340801fcebf800620d3fef5493d8a443077a21c6e56984051bd3abde (28ms) I1002 02:41:43.970370 1 gather_workloads_info.go:387] No image sha256:7941b5d9a758b8667e99cd9236f7f96ec036af61df5cbc96e16077d36700d7c7 (100ms) I1002 02:41:44.071156 1 gather_workloads_info.go:387] No image sha256:76f820bd9bd138d29305d545d7d49bfe63e2923f1fb1ec2e8eac81a388359024 (101ms) I1002 02:41:44.173436 1 gather_workloads_info.go:387] No image sha256:0a92dd43975f972e3dc707fb37854e046813531553038147e9c90d54a8d9df73 (102ms) I1002 02:41:44.271251 1 gather_workloads_info.go:387] No image sha256:f2c46054a8f64a0e949cbf30295f31b4a35a0203c6ad03fa7ec922b4101dcbf8 (98ms) I1002 02:41:44.370934 1 gather_workloads_info.go:387] No image sha256:2d077e73dd76873ee2a1583aa3b3e76a1b408737f0af2e22b2aa055604d89e81 (100ms) I1002 02:41:44.470676 1 gather_workloads_info.go:387] No image sha256:3d0634fe58641d5242649c44ddba70ed67fdd6d2dcb5c2261df5cee8b33de9fd (100ms) I1002 02:41:44.570137 1 gather_workloads_info.go:387] No image sha256:23c4fe84047a4e6cbe7f75f470e7d6fe0e61e7910dd17945b1b61bc4b72f3f2a (99ms) I1002 02:41:44.670856 1 gather_workloads_info.go:387] No image sha256:6e67c980b7300e769fb4b2adaaf006d0f8274e43b10586701b204ea5153f15fc (101ms) I1002 02:41:44.770363 1 gather_workloads_info.go:387] No image sha256:420c12f61c53e54eab2d99476759c0de339fca98ce0a3a782bc6545cc0e97a9c (99ms) I1002 02:41:44.870516 1 gather_workloads_info.go:387] No image sha256:752be69b2262be713df12c47f4bac8c2dafed272c401e9a89f8060f053d68054 (100ms) I1002 02:41:44.970655 1 gather_workloads_info.go:387] No image sha256:944d9261ba7a143131fe8267c172defc5f37acc1cea3d4d373ec6fc5d8bfcc31 (100ms) I1002 02:41:45.070605 1 gather_workloads_info.go:387] No image sha256:48404bd61d05dc738ccce4d22e36b30dbcdc6015b06b4afb604fd1baeee35bf2 (100ms) I1002 02:41:45.170415 1 gather_workloads_info.go:387] No image sha256:8ef4bdf07dda423fab73484dbc66d527ce8ba8cc2d4f99210bc1bc24ea08c0cb (100ms) I1002 02:41:45.270891 1 gather_workloads_info.go:387] No image sha256:d28734effaadd66434e77a1a5fbe2e8a4ca2066cd9f8868c22ade9475539bfd7 (100ms) I1002 02:41:45.370800 1 gather_workloads_info.go:387] No image sha256:824a3b8f78e19aa21a9f6444aefe8e0b624886cca18fe828b13acbab55e6e868 (100ms) I1002 02:41:45.470376 1 gather_workloads_info.go:387] No image sha256:2fac754deaeade3456361eed52e344318ff16d04819384432759f0ea35530114 (100ms) I1002 02:41:45.571032 1 gather_workloads_info.go:387] No image sha256:a9e885d6f0456a2cc10f9e5da71fe9403f5e4c639b9b7ea15bd403b272ccb824 (101ms) I1002 02:41:45.670767 1 gather_workloads_info.go:387] No image sha256:373614619b9420b110d1508c5f17e066ce69c4c226fd04d02fbb959d9ba41eb6 (100ms) I1002 02:41:45.769938 1 gather_workloads_info.go:387] No image sha256:6d6ebff54b8adac74f4d1b12ac8aa0f16cd7b28370e0d6aa847d1c457e03a5b6 (99ms) I1002 02:41:45.806147 1 configmapobserver.go:84] configmaps "insights-config" not found I1002 02:41:45.870773 1 gather_workloads_info.go:387] No image sha256:4f175ee49f51dc4379e9993fecd1657c7a9c4c3dc096772cb198d0212b9eea47 (101ms) I1002 02:41:45.970524 1 gather_workloads_info.go:387] No image sha256:0a8a0473029ace3adbb66f490dbe560f0e7782a38cdf32c1f7dd3e092e1d191e (100ms) I1002 02:41:45.970552 1 tasks_processing.go:74] worker 1 stopped. E1002 02:41:45.970563 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I1002 02:41:45.970793 1 recorder.go:70] Recording config/workload_info with fingerprint=21ec50b12a301e7fd91ef839830035b97a548de1a590273d2d286c0432984269 I1002 02:41:45.970807 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.230363622s to process 1 records E1002 02:41:45.970828 1 periodic.go:250] "Unhandled Error" err="workloads failed after 2.23s with: function \"workload_info\" failed with an error" I1002 02:41:45.971924 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I1002 02:41:45.971938 1 periodic.go:212] Running conditional gatherer I1002 02:41:45.980162 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules I1002 02:41:45.985048 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.10:54076->172.30.0.10:53: read: connection refused E1002 02:41:45.985284 1 conditional_gatherer.go:320] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I1002 02:41:45.985358 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I1002 02:41:45.994133 1 conditional_gatherer.go:390] cluster version is '4.19.9' E1002 02:41:45.994145 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994149 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994152 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994154 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994157 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994159 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994161 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994163 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E1002 02:41:45.994165 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I1002 02:41:45.994176 1 tasks_processing.go:45] number of workers: 3 I1002 02:41:45.994186 1 tasks_processing.go:69] worker 2 listening for tasks. I1002 02:41:45.994189 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I1002 02:41:45.994196 1 tasks_processing.go:69] worker 0 listening for tasks. I1002 02:41:45.994209 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I1002 02:41:45.994210 1 tasks_processing.go:69] worker 1 listening for tasks. I1002 02:41:45.994213 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I1002 02:41:45.994218 1 tasks_processing.go:74] worker 1 stopped. I1002 02:41:45.994256 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I1002 02:41:45.994266 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 469ns to process 1 records I1002 02:41:45.994295 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I1002 02:41:45.994303 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 877ns to process 1 records I1002 02:41:45.994307 1 tasks_processing.go:74] worker 0 stopped. I1002 02:41:45.994424 1 tasks_processing.go:74] worker 2 stopped. I1002 02:41:45.994434 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 205.226µs to process 0 records I1002 02:41:45.994448 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.10:54076->172.30.0.10:53: read: connection refused I1002 02:41:45.994460 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I1002 02:41:46.007098 1 configmapobserver.go:84] configmaps "insights-config" not found I1002 02:41:46.021862 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=20f15e71018240efa54df784200f4faeb4e4e628594098e022a9e09f1f145013 I1002 02:41:46.021971 1 diskrecorder.go:70] Writing 102 records to /var/lib/insights-operator/insights-2026-10-02-024146.tar.gz I1002 02:41:46.027273 1 diskrecorder.go:51] Wrote 102 records to disk in 5ms I1002 02:41:46.027298 1 periodic.go:281] Gathering cluster info every 2h0m0s I1002 02:41:46.027311 1 periodic.go:282] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I1002 02:41:56.657962 1 configmapobserver.go:84] configmaps "insights-config" not found I1002 02:43:00.150718 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="919980c59400356571d3a95e7cb472782707cc7c2591c1f1b8264303a01490f4") W1002 02:43:00.150750 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.key was created I1002 02:43:00.150827 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="005c97b117166829c2566b3aae1f9c5ede09c4a2dc0290dabe9b8a69e4a66569") I1002 02:43:00.150864 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="c0a994724a935ea66de43a2ffb242a8978225331a9b8af53560b1082a6cd1880") I1002 02:43:00.150871 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I1002 02:43:00.150884 1 base_controller.go:181] Shutting down LoggingSyncer ... I1002 02:43:00.150929 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I1002 02:43:00.150937 1 periodic.go:173] Shutting down I1002 02:43:00.150938 1 base_controller.go:181] Shutting down ConfigController ... I1002 02:43:00.150951 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I1002 02:43:00.150958 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I1002 02:43:00.150965 1 base_controller.go:113] All ConfigController workers have been terminated I1002 02:43:00.150992 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"