I0325 11:00:56.340255 1 cmd.go:253] Using service-serving-cert provided certificates I0325 11:00:56.340749 1 observer_polling.go:159] Starting file observer I0325 11:00:56.755895 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0325 11:00:56.756080 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0325 11:00:56.756650 1 secure_serving.go:57] Forcing use of http/1.1 only W0325 11:00:56.756672 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0325 11:00:56.756681 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0325 11:00:56.756688 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0325 11:00:56.756692 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0325 11:00:56.756696 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0325 11:00:56.756701 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0325 11:00:56.756772 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0325 11:00:56.761019 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0325 11:00:56.761111 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"254a5404-5787-4c1b-8391-a866dff122e8", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0325 11:00:56.761912 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0325 11:00:56.761930 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0325 11:00:56.761927 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0325 11:00:56.761934 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0325 11:00:56.761967 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0325 11:00:56.761955 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0325 11:00:56.762328 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0325 11:00:56.762750 1 secure_serving.go:213] Serving securely on [::]:8443 I0325 11:00:56.762784 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0325 11:00:56.770284 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0325 11:00:56.770324 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0325 11:00:56.770431 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0325 11:00:56.776428 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:00:56.776462 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0325 11:00:56.782224 1 secretconfigobserver.go:119] support secret does not exist I0325 11:00:56.787154 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:00:56.792163 1 secretconfigobserver.go:119] support secret does not exist I0325 11:00:56.795620 1 recorder.go:161] Pruning old reports every 6h30m55s, max age is 288h0m0s I0325 11:00:56.801373 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0325 11:00:56.801398 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0325 11:00:56.801409 1 periodic.go:209] Running clusterconfig gatherer I0325 11:00:56.801398 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0325 11:00:56.801424 1 insightsreport.go:296] Starting report retriever I0325 11:00:56.801431 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0325 11:00:56.801453 1 tasks_processing.go:45] number of workers: 32 I0325 11:00:56.801483 1 tasks_processing.go:69] worker 5 listening for tasks. I0325 11:00:56.801493 1 tasks_processing.go:69] worker 6 listening for tasks. I0325 11:00:56.801493 1 tasks_processing.go:69] worker 2 listening for tasks. I0325 11:00:56.801500 1 tasks_processing.go:69] worker 0 listening for tasks. I0325 11:00:56.801501 1 tasks_processing.go:71] worker 6 working on openstack_version task. I0325 11:00:56.801504 1 tasks_processing.go:71] worker 2 working on version task. I0325 11:00:56.801507 1 tasks_processing.go:69] worker 1 listening for tasks. I0325 11:00:56.801513 1 tasks_processing.go:69] worker 3 listening for tasks. I0325 11:00:56.801511 1 tasks_processing.go:69] worker 4 listening for tasks. I0325 11:00:56.801518 1 tasks_processing.go:69] worker 25 listening for tasks. I0325 11:00:56.801527 1 tasks_processing.go:69] worker 28 listening for tasks. I0325 11:00:56.801527 1 tasks_processing.go:69] worker 20 listening for tasks. I0325 11:00:56.801531 1 tasks_processing.go:69] worker 21 listening for tasks. I0325 11:00:56.801537 1 tasks_processing.go:69] worker 22 listening for tasks. I0325 11:00:56.801536 1 tasks_processing.go:69] worker 19 listening for tasks. I0325 11:00:56.801545 1 tasks_processing.go:69] worker 27 listening for tasks. I0325 11:00:56.801545 1 tasks_processing.go:69] worker 7 listening for tasks. I0325 11:00:56.801550 1 tasks_processing.go:69] worker 23 listening for tasks. I0325 11:00:56.801553 1 tasks_processing.go:69] worker 8 listening for tasks. I0325 11:00:56.801556 1 tasks_processing.go:69] worker 24 listening for tasks. I0325 11:00:56.801485 1 tasks_processing.go:69] worker 31 listening for tasks. I0325 11:00:56.801554 1 tasks_processing.go:69] worker 26 listening for tasks. I0325 11:00:56.801494 1 tasks_processing.go:71] worker 5 working on dvo_metrics task. I0325 11:00:56.801567 1 tasks_processing.go:69] worker 29 listening for tasks. I0325 11:00:56.801565 1 tasks_processing.go:69] worker 9 listening for tasks. I0325 11:00:56.801573 1 tasks_processing.go:69] worker 15 listening for tasks. I0325 11:00:56.801580 1 tasks_processing.go:69] worker 16 listening for tasks. I0325 11:00:56.801576 1 tasks_processing.go:69] worker 30 listening for tasks. I0325 11:00:56.801588 1 tasks_processing.go:69] worker 17 listening for tasks. I0325 11:00:56.801590 1 tasks_processing.go:69] worker 18 listening for tasks. I0325 11:00:56.801588 1 tasks_processing.go:69] worker 14 listening for tasks. I0325 11:00:56.801596 1 tasks_processing.go:69] worker 10 listening for tasks. I0325 11:00:56.801598 1 tasks_processing.go:69] worker 11 listening for tasks. I0325 11:00:56.801604 1 tasks_processing.go:69] worker 13 listening for tasks. I0325 11:00:56.801597 1 tasks_processing.go:69] worker 12 listening for tasks. I0325 11:00:56.801668 1 tasks_processing.go:71] worker 12 working on oauths task. I0325 11:00:56.801676 1 tasks_processing.go:71] worker 8 working on machines task. I0325 11:00:56.801678 1 tasks_processing.go:71] worker 16 working on cluster_apiserver task. I0325 11:00:56.801690 1 tasks_processing.go:71] worker 20 working on silenced_alerts task. I0325 11:00:56.801693 1 tasks_processing.go:71] worker 14 working on proxies task. I0325 11:00:56.801748 1 tasks_processing.go:71] worker 0 working on lokistack task. I0325 11:00:56.801808 1 tasks_processing.go:71] worker 1 working on ingress task. I0325 11:00:56.801880 1 tasks_processing.go:71] worker 21 working on container_runtime_configs task. I0325 11:00:56.801914 1 tasks_processing.go:71] worker 30 working on machine_autoscalers task. I0325 11:00:56.801951 1 tasks_processing.go:71] worker 24 working on nodes task. I0325 11:00:56.801962 1 tasks_processing.go:71] worker 3 working on storage_cluster task. I0325 11:00:56.802033 1 tasks_processing.go:71] worker 17 working on monitoring_persistent_volumes task. I0325 11:00:56.802078 1 tasks_processing.go:71] worker 18 working on schedulers task. I0325 11:00:56.802154 1 tasks_processing.go:71] worker 22 working on networks task. I0325 11:00:56.802190 1 tasks_processing.go:71] worker 4 working on overlapping_namespace_uids task. I0325 11:00:56.802274 1 tasks_processing.go:71] worker 29 working on clusterroles task. I0325 11:00:56.802392 1 tasks_processing.go:71] worker 19 working on config_maps task. I0325 11:00:56.802454 1 tasks_processing.go:71] worker 10 working on pod_network_connectivity_checks task. I0325 11:00:56.802482 1 tasks_processing.go:71] worker 9 working on openstack_dataplanenodesets task. I0325 11:00:56.802523 1 tasks_processing.go:71] worker 31 working on aggregated_monitoring_cr_names task. I0325 11:00:56.802702 1 tasks_processing.go:71] worker 11 working on openstack_dataplanedeployments task. I0325 11:00:56.802725 1 tasks_processing.go:71] worker 26 working on feature_gates task. I0325 11:00:56.802731 1 tasks_processing.go:71] worker 25 working on machine_healthchecks task. I0325 11:00:56.802802 1 tasks_processing.go:71] worker 15 working on operators_pods_and_events task. I0325 11:00:56.802861 1 tasks_processing.go:71] worker 28 working on openshift_machine_api_events task. I0325 11:00:56.802948 1 tasks_processing.go:71] worker 13 working on ingress_certificates task. I0325 11:00:56.803117 1 tasks_processing.go:71] worker 23 working on ceph_cluster task. I0325 11:00:56.802953 1 tasks_processing.go:71] worker 27 working on nodenetworkconfigurationpolicies task. I0325 11:00:56.802959 1 tasks_processing.go:71] worker 7 working on openshift_logging task. I0325 11:00:56.805175 1 tasks_processing.go:71] worker 6 working on image_registries task. I0325 11:00:56.805186 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 3.659628ms to process 0 records I0325 11:00:56.805361 1 tasks_processing.go:71] worker 8 working on tsdb_status task. E0325 11:00:56.805398 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0325 11:00:56.805448 1 gather.go:177] gatherer "clusterconfig" function "machines" took 3.66678ms to process 0 records E0325 11:00:56.809058 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:54448->172.30.0.10:53: read: connection refused I0325 11:00:56.809076 1 tasks_processing.go:71] worker 20 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. E0325 11:00:56.809151 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:54448->172.30.0.10:53: read: connection refused I0325 11:00:56.809174 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 7.376577ms to process 0 records E0325 11:00:56.809555 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:34178->172.30.0.10:53: read: connection refused I0325 11:00:56.809568 1 tasks_processing.go:71] worker 8 working on sap_datahubs task. E0325 11:00:56.809586 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:34178->172.30.0.10:53: read: connection refused I0325 11:00:56.809615 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.194468ms to process 0 records I0325 11:00:56.809815 1 tasks_processing.go:71] worker 21 working on authentication task. I0325 11:00:56.809830 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 7.925071ms to process 0 records I0325 11:00:56.809844 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 7.844018ms to process 0 records I0325 11:00:56.809852 1 tasks_processing.go:71] worker 3 working on crds task. I0325 11:00:56.810075 1 tasks_processing.go:71] worker 10 working on metrics task. E0325 11:00:56.810115 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0325 11:00:56.810130 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 7.606067ms to process 0 records I0325 11:00:56.810245 1 tasks_processing.go:71] worker 14 working on pdbs task. I0325 11:00:56.810466 1 recorder.go:75] Recording config/proxy with fingerprint=1eda79ca2e0450f3cf59106be49563e6ba48b3534ec181b7033f51e4db95008c I0325 11:00:56.810487 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 8.532993ms to process 1 records E0325 11:00:56.810500 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0325 11:00:56.810514 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 7.539015ms to process 0 records I0325 11:00:56.810525 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 7.931329ms to process 0 records I0325 11:00:56.810536 1 tasks_processing.go:71] worker 9 working on storage_classes task. I0325 11:00:56.810567 1 tasks_processing.go:71] worker 25 working on service_accounts task. I0325 11:00:56.812999 1 tasks_processing.go:71] worker 27 working on install_plans task. I0325 11:00:56.813009 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 9.577906ms to process 0 records I0325 11:00:56.813232 1 tasks_processing.go:71] worker 16 working on node_logs task. I0325 11:00:56.813473 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0325 11:00:56.813493 1 controller.go:326] The initial operator extension status is healthy I0325 11:00:56.813502 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0325 11:00:56.813506 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0325 11:00:56.813509 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0325 11:00:56.813525 1 controller.go:489] The operator is still being initialized I0325 11:00:56.813530 1 controller.go:512] The operator is healthy I0325 11:00:56.813607 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0325 11:00:56.813700 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0325 11:00:56.813820 1 operator.go:288] started I0325 11:00:56.813843 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0325 11:00:56.814017 1 recorder.go:75] Recording config/apiserver with fingerprint=aa9c82666e99d2e2bbddda26590a665231fb27bccfd1078ed4a8c6f764752cbb I0325 11:00:56.814037 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 11.533016ms to process 1 records I0325 11:00:56.814052 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 10.69234ms to process 0 records I0325 11:00:56.814368 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 11.122129ms to process 0 records I0325 11:00:56.814422 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 12.007478ms to process 0 records I0325 11:00:56.814551 1 tasks_processing.go:71] worker 30 working on cost_management_metrics_configs task. I0325 11:00:56.814615 1 tasks_processing.go:71] worker 11 working on nodenetworkstates task. E0325 11:00:56.814942 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:34620->172.30.0.10:53: read: connection refused I0325 11:00:56.815042 1 tasks_processing.go:71] worker 0 working on certificate_signing_requests task. I0325 11:00:56.815050 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 13.279871ms to process 0 records I0325 11:00:56.814573 1 tasks_processing.go:71] worker 23 working on mutating_webhook_configurations task. I0325 11:00:56.817117 1 tasks_processing.go:71] worker 12 working on machine_config_pools task. I0325 11:00:56.817463 1 recorder.go:75] Recording config/oauth with fingerprint=a1d74a932b399351a2bb6639df9a7d8acf54551c12f920f14350bb2e017f92a4 I0325 11:00:56.817516 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 14.001711ms to process 1 records I0325 11:00:56.817606 1 tasks_processing.go:71] worker 1 working on sap_pods task. I0325 11:00:56.817754 1 recorder.go:75] Recording config/ingress with fingerprint=69b8da89fdba4a25f77b8e611c5fbc95fe65ffe48fc1d311af791f7a17e0ad62 I0325 11:00:56.817770 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 14.29516ms to process 1 records I0325 11:00:56.817783 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 12.755802ms to process 0 records E0325 11:00:56.817792 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:34620->172.30.0.10:53: read: connection refused I0325 11:00:56.817809 1 tasks_processing.go:71] worker 7 working on olm_operators task. I0325 11:00:56.817815 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.609037ms to process 0 records I0325 11:00:56.817952 1 tasks_processing.go:71] worker 24 working on sap_config task. I0325 11:00:56.817995 1 tasks_processing.go:71] worker 10 working on machine_configs task. I0325 11:00:56.818303 1 recorder.go:75] Recording config/node/ip-10-0-2-29.ec2.internal with fingerprint=2a3a4e9126498d0add0ed7c145ddf03c1f88a8a928f6063fae3f1e0173ba2efa I0325 11:00:56.818319 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 15.345048ms to process 1 records I0325 11:00:56.818413 1 tasks_processing.go:71] worker 22 working on infrastructures task. I0325 11:00:56.818471 1 recorder.go:75] Recording config/network with fingerprint=2a2da455cc9707e180e3074ee7ed681cb0a0ddeed1a41a25738eb4e2958fc472 I0325 11:00:56.818483 1 gather.go:177] gatherer "clusterconfig" function "networks" took 15.824518ms to process 1 records I0325 11:00:56.818536 1 tasks_processing.go:71] worker 8 working on image task. I0325 11:00:56.818588 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 8.960411ms to process 0 records W0325 11:00:56.820647 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0325 11:00:56.828418 1 tasks_processing.go:71] worker 28 working on support_secret task. I0325 11:00:56.828470 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 25.540386ms to process 0 records I0325 11:00:56.828488 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 13.87483ms to process 0 records I0325 11:00:56.828497 1 tasks_processing.go:71] worker 30 working on jaegers task. I0325 11:00:56.828723 1 tasks_processing.go:71] worker 11 working on active_alerts task. I0325 11:00:56.828733 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 14.090521ms to process 0 records I0325 11:00:56.828914 1 tasks_processing.go:71] worker 26 working on operators task. I0325 11:00:56.828963 1 recorder.go:75] Recording config/featuregate with fingerprint=7921807ca5dcfc3bd454eaa933414e065f11ab5e1781a1b1ef83a2f1afb85576 I0325 11:00:56.828983 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 25.999561ms to process 1 records I0325 11:00:56.828992 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 15.58788ms to process 0 records I0325 11:00:56.829066 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=c75b60c7cd9626360c4daf0a263f5998027551d96a8adb89643cc0d7c1922d51 I0325 11:00:56.829080 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 26.824666ms to process 1 records I0325 11:00:56.829084 1 tasks_processing.go:71] worker 16 working on qemu_kubevirt_launcher_logs task. I0325 11:00:56.829307 1 tasks_processing.go:71] worker 18 working on machine_sets task. I0325 11:00:56.829373 1 recorder.go:75] Recording config/version with fingerprint=6c8ca54f88055352d0941d088c28003476a270a7f477bab0311927125beee2c4 I0325 11:00:56.829426 1 recorder.go:75] Recording config/id with fingerprint=9b22bb3a31d107e9555eddb220c3d6716cee2c597e4184df6690135d07bb4edc I0325 11:00:56.829439 1 gather.go:177] gatherer "clusterconfig" function "version" took 27.43765ms to process 2 records I0325 11:00:56.829376 1 tasks_processing.go:71] worker 2 working on image_pruners task. I0325 11:00:56.830451 1 tasks_processing.go:71] worker 17 working on openstack_controlplanes task. I0325 11:00:56.830473 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 28.408327ms to process 0 records I0325 11:00:56.831146 1 tasks_processing.go:71] worker 1 working on validating_webhook_configurations task. I0325 11:00:56.831198 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 13.488503ms to process 0 records I0325 11:00:56.831526 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=d477e6ed8ac74bfc019a9c77515cf1f68fe0055cd07f68e77326af4f1dbc6779 I0325 11:00:56.831589 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=380524e2ba46d309b5b56f1ce782d656bc6c038fc2a00959c0acbad4090b4e1e I0325 11:00:56.831632 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 20.646319ms to process 2 records I0325 11:00:56.831662 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 13.239862ms to process 0 records I0325 11:00:56.831668 1 tasks_processing.go:71] worker 9 working on container_images task. I0325 11:00:56.831863 1 tasks_processing.go:74] worker 21 stopped. I0325 11:00:56.831697 1 tasks_processing.go:74] worker 24 stopped. I0325 11:00:56.832140 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0325 11:00:56.832195 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0325 11:00:56.832220 1 recorder.go:75] Recording config/authentication with fingerprint=c725b31e3169044d1576edcdf4822cbeb0dd6db5699d692ed2aaaa93e5e06ec3 I0325 11:00:56.832279 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 22.032589ms to process 1 records I0325 11:00:56.832224 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0325 11:00:56.832343 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0325 11:00:56.832358 1 controller.go:212] Source scaController *sca.Controller is not ready I0325 11:00:56.832385 1 controller.go:489] The operator is still being initialized I0325 11:00:56.832394 1 controller.go:512] The operator is healthy I0325 11:00:56.832475 1 tasks_processing.go:74] worker 6 stopped. I0325 11:00:56.833338 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=c2e15ebe7297dbd9323ff1ee4ee8c57f127b31b8d6168195bff09b6fcb6339ab I0325 11:00:56.833403 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 27.065758ms to process 1 records E0325 11:00:56.834978 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:47267->172.30.0.10:53: read: connection refused I0325 11:00:56.835043 1 tasks_processing.go:74] worker 11 stopped. E0325 11:00:56.835074 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:47267->172.30.0.10:53: read: connection refused I0325 11:00:56.835138 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 6.281339ms to process 0 records I0325 11:00:56.835793 1 tasks_processing.go:74] worker 4 stopped. I0325 11:00:56.835886 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0325 11:00:56.835953 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 33.583263ms to process 1 records I0325 11:00:56.836482 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=ac08be3c526fdf9a78b75f9614a31818814d5125dde8de990c1071a278cf8ff3 I0325 11:00:56.836501 1 tasks_processing.go:74] worker 29 stopped. I0325 11:00:56.837316 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=21b85148366579950a1af7be40628e3f6d3afae1d3aff0b37a66b31cb45c41e2 I0325 11:00:56.837335 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 33.626236ms to process 2 records I0325 11:00:56.837438 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=10927275706fc811667af311b5c925ae4ea567e7ef096278c22af1b8f197f192 I0325 11:00:56.837441 1 tasks_processing.go:74] worker 14 stopped. I0325 11:00:56.837461 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=7131c9b535f79407f5771aa5f65b2d7eaaf3ab6cf73a4489505babc361d756b9 I0325 11:00:56.837479 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=d79151ea554f97a4881dadaa8639549b856f53b82268708fd72ca36c0ac1ed6b I0325 11:00:56.837498 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=fc345dc30caeccf55c54bc5e131d5abf2b2a8f6b8823027d4d171860eaa1c41f I0325 11:00:56.837517 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=0667394c01379efce5e03390d2ad5b6049d8673456ded97bf18ba0a457d57e35 I0325 11:00:56.837537 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=7739b1d1decf587888cc99f7b9e1d393584c4148b980f75c4a87f0ed84f15f86 I0325 11:00:56.837549 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 25.780224ms to process 6 records I0325 11:00:56.837614 1 tasks_processing.go:74] worker 23 stopped. I0325 11:00:56.837768 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=f3034c8f911dba5d0617adf27a75e93e2e66783bb4d76d908a708f583d632b8d I0325 11:00:56.837872 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=a114a2eadaebebe8e9c3fae0fb5a51595c78e63fea25b459a6e49e66316c852a I0325 11:00:56.837949 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=090ebfd57b2d86361e9a29deae4b1b00cfe7652655cc00fd47a800ed9f23bf5c I0325 11:00:56.837967 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 20.355746ms to process 3 records I0325 11:00:56.837981 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 9.253043ms to process 0 records I0325 11:00:56.837990 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 20.724497ms to process 0 records I0325 11:00:56.838017 1 tasks_processing.go:74] worker 12 stopped. I0325 11:00:56.838026 1 tasks_processing.go:74] worker 30 stopped. I0325 11:00:56.838879 1 tasks_processing.go:74] worker 22 stopped. I0325 11:00:56.839771 1 recorder.go:75] Recording config/infrastructure with fingerprint=8332308226e6b3326485d07012266bdf3a2acff35c52a24f549421672ae59fa8 I0325 11:00:56.839798 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 20.448944ms to process 1 records I0325 11:00:56.839918 1 recorder.go:75] Recording config/image with fingerprint=e3960b53433555b5b56314e3d3e487c70858305bc02dc1c1a8bd4f2f3b69b871 I0325 11:00:56.839930 1 gather.go:177] gatherer "clusterconfig" function "image" took 20.690298ms to process 1 records I0325 11:00:56.839942 1 tasks_processing.go:74] worker 8 stopped. I0325 11:00:56.839966 1 tasks_processing.go:74] worker 0 stopped. I0325 11:00:56.839987 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 23.229389ms to process 0 records I0325 11:00:56.840046 1 tasks_processing.go:74] worker 18 stopped. I0325 11:00:56.840061 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 10.723615ms to process 0 records I0325 11:00:56.840127 1 tasks_processing.go:74] worker 17 stopped. I0325 11:00:56.840145 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 9.659121ms to process 0 records E0325 11:00:56.840157 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0325 11:00:56.840166 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 11.691365ms to process 0 records I0325 11:00:56.840175 1 tasks_processing.go:74] worker 28 stopped. I0325 11:00:56.842456 1 tasks_processing.go:74] worker 2 stopped. I0325 11:00:56.842590 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=2a2f6db830ccb33c05391778549ce2bd79afd6628b1e60ab450b907b0d3422c9 I0325 11:00:56.842643 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 12.993365ms to process 1 records E0325 11:00:56.846186 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2704077b84-4229-48f6-8321-d40a95220cf8%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:56483->172.30.0.10:53: read: connection refused I0325 11:00:56.846203 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2704077b84-4229-48f6-8321-d40a95220cf8%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:56483->172.30.0.10:53: read: connection refused I0325 11:00:56.846929 1 gather_logs.go:145] no pods in namespace were found I0325 11:00:56.846949 1 tasks_processing.go:74] worker 16 stopped. I0325 11:00:56.846959 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 17.833165ms to process 0 records I0325 11:00:56.847637 1 tasks_processing.go:74] worker 31 stopped. I0325 11:00:56.847664 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 45.095334ms to process 0 records I0325 11:00:56.849135 1 tasks_processing.go:74] worker 1 stopped. I0325 11:00:56.849639 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=6be0647745a375b24c1ec61ee5de89f6eb1232dd1142335dc4445cbece1c7dfd I0325 11:00:56.849994 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=bf2a3e82915d5c7df3d24c148569934d66a664a862515f41e078484e718699e2 I0325 11:00:56.850076 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=ee757c5c5e1051e201b76c679fecf0921767d36e5fc608ea4e7c3dec0118c6c0 I0325 11:00:56.850163 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=d41cd9932d6fcc47da701ab9e0e06bb1aea4ed470265bd4a9bb678cb40ab0b7a I0325 11:00:56.850278 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=7efbb3bd2a29d5707691f7f074bc2421208f6ffa038e8c4e93c01ec698854725 I0325 11:00:56.850361 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=db9d282e6c47323bae3208f0ff7b7a2c682cc62da0d3ea74ac968c82860e935b I0325 11:00:56.850464 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=a00cff2696a0308139178c39eaf63e62b2e0bbfacaa8f4e913544aeca78a76f3 I0325 11:00:56.850573 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=e09d44e9b27f641406b6c36de8891b3adba7246db58a35af9e1c60398d78c1fc I0325 11:00:56.850656 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=f7ce86fd88474cbda7dc9395b1539df856ba450feb64ae68b72bbc0a4e02875e I0325 11:00:56.850732 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=5392dc3561219b5dcf612a06462784b2112156a1ad506abf41d24d6239420d24 I0325 11:00:56.850838 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=9933ec444017b9ae35f5e63254bf0e96b4b92ac082e3c7875a4306bf21d2124c I0325 11:00:56.850870 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 17.929786ms to process 11 records I0325 11:00:56.851008 1 recorder.go:75] Recording config/olm_operators with fingerprint=51b90f4e47c71d16a296c7d1e55d1e2da5c38b402c12a0e86dc7cb28851d7761 I0325 11:00:56.851046 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 31.516472ms to process 1 records I0325 11:00:56.851558 1 tasks_processing.go:74] worker 7 stopped. I0325 11:00:56.851700 1 tasks_processing.go:74] worker 3 stopped. I0325 11:00:56.852738 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=50368f4b694eef8830655904898d0a3855f704cca10efe9a8a2165f601e5debe I0325 11:00:56.853374 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=f43306a72ab7e964d7c1b6d7433e7f1c196e059ae23d6ccd9abf0a350b6d4215 I0325 11:00:56.853401 1 gather.go:177] gatherer "clusterconfig" function "crds" took 39.889255ms to process 2 records I0325 11:00:56.853683 1 tasks_processing.go:74] worker 10 stopped. I0325 11:00:56.853805 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0325 11:00:56.853848 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 35.662194ms to process 1 records I0325 11:00:56.856570 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0325 11:00:56.857340 1 tasks_processing.go:74] worker 9 stopped. I0325 11:00:56.858943 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8 with fingerprint=e480c72b2477cc341619ca1421d4667aec871783d90d599df041c57fd1b91613 I0325 11:00:56.859022 1 recorder.go:75] Recording config/running_containers with fingerprint=216d54fe7dc13240963ac9d0da74a13fc68351de04d28a7fc47db72681e53bcd I0325 11:00:56.859036 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 25.619524ms to process 2 records I0325 11:00:56.859047 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 49.30233ms to process 0 records I0325 11:00:56.859133 1 tasks_processing.go:74] worker 20 stopped. W0325 11:00:56.860618 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:49079->172.30.0.10:53: read: connection refused I0325 11:00:56.860635 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:49079->172.30.0.10:53: read: connection refused I0325 11:00:56.862280 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0325 11:00:56.862284 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0325 11:00:56.862296 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0325 11:00:56.866120 1 tasks_processing.go:74] worker 13 stopped. E0325 11:00:56.866137 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0325 11:00:56.866146 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2p8mq24lsf2m58q3itf4qvcrijjq5nr6-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2p8mq24lsf2m58q3itf4qvcrijjq5nr6-primary-cert-bundle-secret" not found I0325 11:00:56.866205 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=311dc42b88a78c3793bcf7cfd35011ea7d52b66fa983d20d2d98c70dfe36ec4b I0325 11:00:56.866221 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 63.150974ms to process 1 records I0325 11:00:56.866793 1 tasks_processing.go:74] worker 19 stopped. E0325 11:00:56.866808 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0325 11:00:56.866817 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0325 11:00:56.866822 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0325 11:00:56.866836 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0325 11:00:56.866870 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0325 11:00:56.866880 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0325 11:00:56.866886 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0325 11:00:56.866902 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0325 11:00:56.866947 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0325 11:00:56.866957 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0325 11:00:56.866965 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 64.382543ms to process 7 records I0325 11:00:56.870537 1 base_controller.go:82] Caches are synced for ConfigController I0325 11:00:56.870551 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0325 11:00:56.876254 1 configmapobserver.go:84] configmaps "insights-config" not found I0325 11:00:56.914419 1 base_controller.go:82] Caches are synced for LoggingSyncer I0325 11:00:56.914439 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0325 11:00:56.917026 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found W0325 11:00:57.820054 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0325 11:00:58.449332 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 19 containers I0325 11:00:58.449351 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1324517 bytes I0325 11:00:58.449963 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-j8n2x pod in namespace openshift-network-console (previous: false). I0325 11:00:58.674311 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-mzqpk pod in namespace openshift-network-console (previous: false). I0325 11:00:58.676935 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0325 11:00:58.820443 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0325 11:00:58.855371 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-vwrr8 pod in namespace openshift-console-operator (previous: true). I0325 11:00:59.053806 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-vwrr8 pod in namespace openshift-console-operator (previous: false). I0325 11:00:59.257379 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-6df455974c-g9djq pod in namespace openshift-console (previous: false). I0325 11:00:59.275949 1 tasks_processing.go:74] worker 26 stopped. I0325 11:00:59.276067 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=bb27bab911e928fcfde22ef08941ae8e1070cea2d3afd61ed6a39e76ee97c7b8 I0325 11:00:59.276134 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=b1d3a57201f472fba248e6e22732d03f4ebac77f3d7204455295e9c6f31f376a I0325 11:00:59.276160 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=da384aeb2eaba2b5190e12e5ce9d3b0da56ca0e3b9c57eeff390d4e205f1c24f I0325 11:00:59.276186 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0325 11:00:59.276211 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=b316885e624a0592ac4472c59038bac3e964e1266d81e9aa3db13af0b5cb3426 I0325 11:00:59.276227 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0325 11:00:59.276248 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=64007a98d491325e45ffe6d27efca077bc87449cd9de0bb10dcf5b1ea5bce2ca I0325 11:00:59.276276 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=aa497d6c33053e5ef4797fe9db3732a6a261e3a7511f28276e7548833d1d9cc0 I0325 11:00:59.276300 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=da18cd1ba4e5ef5896ddb33291981e862f19eb7ddb9c0aa2e228cec73dc220bf I0325 11:00:59.276329 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=32c64f05985d4844ddceabd7d774964412186f850f3674924134812e51488edc I0325 11:00:59.276610 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=137788eeb3b717f13c3fe488c35bc2bd9fb03df4f7e6cfe2ca352e74e883ec97 I0325 11:00:59.276629 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=09ea89f79a61ede3a225cccec101e3679b85d9558ceb2647ed23a641ea5c2e0d I0325 11:00:59.276639 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0325 11:00:59.276655 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=5f502f278912398197fb56397c88dcea7ab2d2c7a2fe8876271529b1b493f142 I0325 11:00:59.276666 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0325 11:00:59.276681 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=525ed72a8d631076fb74e155b6584cfadf9f538a5e566b0888f6cbb45072dee0 I0325 11:00:59.276691 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0325 11:00:59.276710 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=c8037b135e07e4b80b28b5461c826de1098eab2d4b874102a318033da1abb729 I0325 11:00:59.276719 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0325 11:00:59.276738 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=6d9048a4481754b71704cf343c31dd7ee2acf5dbbc8c9c039f61e390abed9df3 I0325 11:00:59.276839 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=19de6deafd4e4f537a3434a15ffca8cb3fb48bf634bcafb061b3fecff2a4e6f6 I0325 11:00:59.276851 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0325 11:00:59.276858 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0325 11:00:59.276878 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0325 11:00:59.276898 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=352e8360e79de01505e511afd74791cbfda115d682e524a18fda842495e8f90f I0325 11:00:59.276924 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=228ed78cb39268852bf18bd52319200890ba3418cb77e0c24d29ef73cfba8864 I0325 11:00:59.276933 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0325 11:00:59.276948 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=7c6f54ed98b2c12f30113336f9fb7672b88b156df06d41ea6a5dba6b6480c35d I0325 11:00:59.276958 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0325 11:00:59.276972 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=336a097586e7767fd5a569772c4ce9067a03c3b85a9cb264bf8083e37ae21e9b I0325 11:00:59.276986 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=081b990a080f2975471cd8b2750528e8ec382959647703ac1afc956dc2d5538f I0325 11:00:59.277000 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=220d0da3d4222df6d27868787be5128be629102b00f082711c820da960843f36 I0325 11:00:59.277015 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=56040e4f7dd3fca7408c2970678f97559ee113fd07534ef1413aa22b84088871 I0325 11:00:59.277036 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=0155fe44050b23f8adeab57a7a9a2dac10c3281b57b731b5fed67be61d050af3 I0325 11:00:59.277045 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0325 11:00:59.277068 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=2d32ff3ca2e2aa3b9ba28af5decdb0eeb9ea2f8a3e9197e30a4afa6af9b9314d I0325 11:00:59.277084 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0325 11:00:59.277104 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0325 11:00:59.277111 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.446975146s to process 38 records I0325 11:00:59.454742 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-7f9644b479-csmsj pod in namespace openshift-console (previous: false). I0325 11:00:59.648655 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:00:59.648675 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-7f9644b479-q7kkl pod in namespace openshift-console (previous: false). W0325 11:00:59.820552 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0325 11:00:59.848589 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:00:59.848607 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-dt66l pod in namespace openshift-console (previous: false). I0325 11:01:00.053817 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-t85br pod in namespace openshift-console (previous: false). I0325 11:01:00.249371 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:01:00.249408 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-knvzq pod in namespace openshift-dns (previous: false). I0325 11:01:00.453115 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-knvzq pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-knvzq\" is waiting to start: ContainerCreating" I0325 11:01:00.453137 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-knvzq\" is waiting to start: ContainerCreating" I0325 11:01:00.453148 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-knvzq pod in namespace openshift-dns (previous: false). I0325 11:01:00.655131 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-knvzq pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-knvzq\" is waiting to start: ContainerCreating" I0325 11:01:00.655150 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-knvzq\" is waiting to start: ContainerCreating" I0325 11:01:00.655160 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-frhk9 pod in namespace openshift-dns (previous: false). W0325 11:01:00.821444 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0325 11:01:00.854510 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:01:00.854529 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5474746cdf-v457c pod in namespace openshift-image-registry (previous: false). I0325 11:01:01.055957 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-57779d6d6f-2892v pod in namespace openshift-image-registry (previous: false). I0325 11:01:01.249449 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:01:01.249465 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-57779d6d6f-ghg6q pod in namespace openshift-image-registry (previous: false). I0325 11:01:01.449083 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:01:01.449122 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-6kl8n pod in namespace openshift-image-registry (previous: false). I0325 11:01:01.655738 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5867d65d4f-9djh8 pod in namespace openshift-ingress (previous: false). W0325 11:01:01.820629 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0325 11:01:01.820657 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0325 11:01:01.820677 1 tasks_processing.go:74] worker 5 stopped. E0325 11:01:01.820686 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0325 11:01:01.820696 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0325 11:01:01.820706 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.019099146s to process 1 records I0325 11:01:01.856346 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6c7f47974-2jk55 pod in namespace openshift-ingress (previous: false). I0325 11:01:02.057702 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6c7f47974-fffbx pod in namespace openshift-ingress (previous: false). I0325 11:01:02.250479 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0325 11:01:02.250518 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-xdq6k pod in namespace openshift-ingress-canary (previous: false). I0325 11:01:02.462147 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-xdq6k pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-xdq6k\" is waiting to start: ContainerCreating" I0325 11:01:02.462167 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-xdq6k\" is waiting to start: ContainerCreating" I0325 11:01:02.462185 1 tasks_processing.go:74] worker 15 stopped. I0325 11:01:02.462281 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=e5c7d301c3ce2b5295b05af2eb30157e0f2274b148144f5b2c96814799043ba5 I0325 11:01:02.462402 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=16a5dd63f96a51aac40f33ebe8f61466b7894161564716c25e0c57367dc16ec5 I0325 11:01:02.462454 1 recorder.go:75] Recording events/openshift-console with fingerprint=03649e7b066ff3f677ade6f086caa352ea5f78262434827284a82a1c43055989 I0325 11:01:02.462472 1 recorder.go:75] Recording events/openshift-dns with fingerprint=bcc1e03efaebe817ed85f6f6b1786117f31faef7c1c2059b8f40e5c4c99798ec I0325 11:01:02.462564 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=de405fc58352fbe664a1c09174f1019d5d7bcef4050029a1cc484edc8eefa02d I0325 11:01:02.462591 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=5883c3fdf104135266beadca02fdb80769331e1efb805ab8871aca82df27d5d0 I0325 11:01:02.462643 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=96c207fb20f183310d55c3ee41033b85404d3138c99cd99d71b05970fc5f31ff I0325 11:01:02.462654 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=28e6ed048285e8170f6c17392bb3a1011f3bb051926ade3056127d18794e9476 I0325 11:01:02.462662 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-j8n2x/networking-console-plugin_current.log with fingerprint=0948e47b960ebe9acee4d16212a840fc8462507f9b18857d9990c18cb53e15fa I0325 11:01:02.462669 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-mzqpk/networking-console-plugin_current.log with fingerprint=ec3729d11cb96df6818b125912894114ce8099bc3d2026351e1820a2f8e30730 I0325 11:01:02.462801 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8 with fingerprint=e480c72b2477cc341619ca1421d4667aec871783d90d599df041c57fd1b91613 E0325 11:01:02.462844 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json" was already recorded and had the fingerprint "e480c72b2477cc341619ca1421d4667aec871783d90d599df041c57fd1b91613", overwriting with the record having fingerprint "e480c72b2477cc341619ca1421d4667aec871783d90d599df041c57fd1b91613" W0325 11:01:02.462857 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json" because of the warning: warning: the record with the same fingerprint "e480c72b2477cc341619ca1421d4667aec871783d90d599df041c57fd1b91613" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json" I0325 11:01:02.462869 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-vwrr8/console-operator_previous.log with fingerprint=e5f622c36c69bbd7195cd7c4d5d7346d5b37307fc18357ea63036479ba7f77ec I0325 11:01:02.463046 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-vwrr8/console-operator_current.log with fingerprint=b95ae22dc49796563a6e6766f1cb403690d7b0f14b0105d03f5800acb5687d79 I0325 11:01:02.463061 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-6df455974c-g9djq/console_current.log with fingerprint=5dbb825bfbc142ebd332e4ed6535175bb1f0118cf2e410f0587bd625cdca4c5b I0325 11:01:02.463066 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-dt66l/download-server_current.log with fingerprint=5d0cd453172568c80bb4c0334f44ad112576251debf03e7af881278197238585 I0325 11:01:02.463190 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-knvzq with fingerprint=00903aa7f550d2af78c80633ff0b9bf960011c3e02f489af2b70c65373ec2148 I0325 11:01:02.463218 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-5474746cdf-v457c/registry_current.log with fingerprint=dcaedece627f303a4e3e0ce1527a230e9a25bbdcb4777026041d8a8afee7bd9b I0325 11:01:02.463225 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-6kl8n/node-ca_current.log with fingerprint=d1213d86e1d897d7fcb4efc0d63325821bdadd7144d34a63c3fa5f3820158820 I0325 11:01:02.463240 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5867d65d4f-9djh8/router_current.log with fingerprint=385a9e93b1739b2de67f1604900ecb7a26b74c2136b1c40a8502a45446d291f1 I0325 11:01:02.463255 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-6c7f47974-2jk55/router_current.log with fingerprint=a3f7f4c4ca1585a07c1f26217edf3427d8a820e31b08bbbe49f26f5761bac8d8 I0325 11:01:02.463314 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-xdq6k with fingerprint=05d80f892b4da64aa0615bda77b061c23d1464c7ae9cfeccf73ebbdf48c89219 I0325 11:01:02.463349 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 5.659366574s to process 21 records I0325 11:01:09.646664 1 tasks_processing.go:74] worker 27 stopped. I0325 11:01:09.646710 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0325 11:01:09.646724 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.833638647s to process 1 records I0325 11:01:10.417471 1 tasks_processing.go:74] worker 25 stopped. I0325 11:01:10.417748 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=1a12195152df53baccddfc9d0b61ab216e3b4b53237b871699c391e051196073 I0325 11:01:10.417768 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.606881418s to process 1 records E0325 11:01:10.417847 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.616s with: function \"machines\" failed with an error, function \"silenced_alerts\" failed with an error, function \"tsdb_status\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"metrics\" failed with an error, function \"active_alerts\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json\"" I0325 11:01:10.418953 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-vwrr8.json" I0325 11:01:10.418967 1 periodic.go:209] Running workloads gatherer I0325 11:01:10.418980 1 tasks_processing.go:45] number of workers: 2 I0325 11:01:10.418986 1 tasks_processing.go:69] worker 1 listening for tasks. I0325 11:01:10.418990 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0325 11:01:10.419054 1 tasks_processing.go:69] worker 0 listening for tasks. I0325 11:01:10.419107 1 tasks_processing.go:71] worker 0 working on workload_info task. I0325 11:01:10.425725 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-29.ec2.internal... I0325 11:01:10.451359 1 tasks_processing.go:74] worker 1 stopped. I0325 11:01:10.451374 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 32.356556ms to process 0 records I0325 11:01:13.880534 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 3s I0325 11:01:13.897296 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 20s for image data I0325 11:01:13.907737 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (11ms) I0325 11:01:13.917988 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0325 11:01:13.927538 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (10ms) I0325 11:01:13.937134 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (10ms) I0325 11:01:13.946186 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (9ms) I0325 11:01:13.956739 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (11ms) I0325 11:01:13.966913 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (10ms) I0325 11:01:13.976333 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (9ms) I0325 11:01:13.987001 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (11ms) I0325 11:01:13.997642 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (11ms) I0325 11:01:14.007760 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (10ms) I0325 11:01:14.108507 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (101ms) I0325 11:01:14.208501 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0325 11:01:14.307676 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (99ms) I0325 11:01:14.407529 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0325 11:01:14.508537 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (101ms) I0325 11:01:14.608722 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0325 11:01:14.709855 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (101ms) I0325 11:01:14.809106 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (99ms) I0325 11:01:14.908285 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (99ms) I0325 11:01:15.008416 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0325 11:01:15.112646 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (104ms) I0325 11:01:15.209195 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (97ms) I0325 11:01:15.309532 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (100ms) I0325 11:01:15.409168 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (100ms) I0325 11:01:15.508833 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0325 11:01:15.607979 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (99ms) I0325 11:01:15.708831 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0325 11:01:15.821791 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (113ms) I0325 11:01:15.913978 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (92ms) I0325 11:01:16.013757 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0325 11:01:16.108968 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (95ms) I0325 11:01:16.208628 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0325 11:01:16.308460 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0325 11:01:16.408224 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (100ms) I0325 11:01:16.509892 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (102ms) I0325 11:01:16.509932 1 tasks_processing.go:74] worker 0 stopped. I0325 11:01:16.510297 1 recorder.go:75] Recording config/workload_info with fingerprint=8eacb1a933704069907aa66725f4d80a49c38bd0cd08a61e62a588dd2a29990a I0325 11:01:16.510319 1 gather.go:177] gatherer "workloads" function "workload_info" took 6.090810911s to process 1 records I0325 11:01:16.510350 1 periodic.go:256] Periodic gather workloads completed in 6.091s I0325 11:01:16.510363 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0325 11:01:16.510369 1 periodic.go:209] Running conditional gatherer I0325 11:01:16.517604 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0325 11:01:16.525277 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.12:35969->172.30.0.10:53: read: connection refused I0325 11:01:16.525554 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0325 11:01:16.529744 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.12:39418->172.30.0.10:53: read: connection refused I0325 11:01:16.529836 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0325 11:01:16.538409 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0325 11:01:16.538483 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538512 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538536 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538559 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538582 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538605 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538627 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538649 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0325 11:01:16.538671 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0325 11:01:16.538706 1 tasks_processing.go:45] number of workers: 3 I0325 11:01:16.538736 1 tasks_processing.go:69] worker 2 listening for tasks. I0325 11:01:16.538759 1 tasks_processing.go:71] worker 2 working on remote_configuration task. I0325 11:01:16.538854 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0325 11:01:16.538886 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.463µs to process 1 records I0325 11:01:16.538914 1 tasks_processing.go:69] worker 0 listening for tasks. I0325 11:01:16.538936 1 tasks_processing.go:69] worker 1 listening for tasks. I0325 11:01:16.538961 1 tasks_processing.go:74] worker 1 stopped. I0325 11:01:16.538983 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0325 11:01:16.539007 1 tasks_processing.go:74] worker 2 stopped. I0325 11:01:16.539084 1 tasks_processing.go:71] worker 0 working on rapid_container_logs task. I0325 11:01:16.539130 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0325 11:01:16.539236 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 660ns to process 1 records I0325 11:01:16.539569 1 tasks_processing.go:74] worker 0 stopped. I0325 11:01:16.539588 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 392.155µs to process 0 records I0325 11:01:16.539616 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.12:35969->172.30.0.10:53: read: connection refused I0325 11:01:16.539639 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0325 11:01:16.569141 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=ff8cda387ea20fb21056e1b43ddd7db62044294b426556fc40d2dea55313d9e7 I0325 11:01:16.569268 1 diskrecorder.go:70] Writing 119 records to /var/lib/insights-operator/insights-2026-03-25-110116.tar.gz I0325 11:01:16.577630 1 diskrecorder.go:51] Wrote 119 records to disk in 8ms I0325 11:01:16.577665 1 periodic.go:278] Gathering cluster info every 2h0m0s I0325 11:01:16.577681 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0325 11:02:26.801717 1 diskrecorder.go:223] Found files to send: insights-2026-03-25-110116.tar.gz I0325 11:02:26.801749 1 insightsuploader.go:150] Checking archives to upload periodically every 15m51.519779245s I0325 11:02:26.801760 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0325 11:02:26.814622 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0325 11:02:27.096753 1 requests.go:87] Successfully reported id=2026-03-25T11:02:26Z x-rh-insights-request-id=77d7ead66dfb41638d984e45aa998f95, wrote=62287 I0325 11:02:27.096849 1 insightsuploader.go:187] Uploaded report successfully in 295.087264ms I0325 11:02:27.096866 1 controller.go:128] Initializing last reported time to 2026-03-25T11:02:26Z I0325 11:02:27.096942 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0325 11:02:27.096957 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0325 11:02:27.096966 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0325 11:02:27.104502 1 controller.go:512] The operator is healthy I0325 11:02:56.824370 1 controller.go:512] The operator is healthy I0325 11:03:27.192242 1 insightsreport.go:137] Pulling report from smart-proxy I0325 11:03:27.192270 1 insightsreport.go:149] Retrieving report I0325 11:03:27.202754 1 requests.go:110] Retrieving report for cluster: 04077b84-4229-48f6-8321-d40a95220cf8 I0325 11:03:27.202767 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/04077b84-4229-48f6-8321-d40a95220cf8/reports I0325 11:03:27.205576 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/04077b84-4229-48f6-8321-d40a95220cf8/reports I0325 11:03:27.516397 1 insightsreport.go:184] Report retrieved I0325 11:03:27.545745 1 insightsreport.go:239] Report retrieved correctly I0325 11:04:56.822183 1 controller.go:512] The operator is healthy I0325 11:05:56.792445 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:05:56.799198 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:05:56.799221 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:05:56.804409 1 secretconfigobserver.go:119] support secret does not exist I0325 11:06:56.821939 1 controller.go:512] The operator is healthy I0325 11:08:56.821069 1 controller.go:512] The operator is healthy I0325 11:10:56.790524 1 configmapobserver.go:84] configmaps "insights-config" not found I0325 11:10:56.804600 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:10:56.810411 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:10:56.810426 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:10:56.815389 1 secretconfigobserver.go:119] support secret does not exist I0325 11:10:56.820455 1 controller.go:512] The operator is healthy I0325 11:12:56.826854 1 controller.go:512] The operator is healthy I0325 11:14:56.825401 1 controller.go:512] The operator is healthy I0325 11:15:56.815892 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:15:56.824895 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:15:56.824916 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:15:56.830291 1 secretconfigobserver.go:119] support secret does not exist I0325 11:16:56.827207 1 controller.go:512] The operator is healthy I0325 11:18:23.617367 1 insightsuploader.go:146] Nothing to report since 2026-03-25T11:02:26Z I0325 11:18:56.824985 1 controller.go:512] The operator is healthy I0325 11:20:56.786934 1 configmapobserver.go:84] configmaps "insights-config" not found I0325 11:20:56.821643 1 controller.go:512] The operator is healthy I0325 11:20:56.830450 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:20:56.836292 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:20:56.836309 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:20:56.841616 1 secretconfigobserver.go:119] support secret does not exist I0325 11:22:56.821413 1 controller.go:512] The operator is healthy I0325 11:24:56.820223 1 controller.go:512] The operator is healthy I0325 11:25:56.842525 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:25:56.849281 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:25:56.849305 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:25:56.854371 1 secretconfigobserver.go:119] support secret does not exist I0325 11:26:56.821385 1 controller.go:512] The operator is healthy I0325 11:28:56.821305 1 controller.go:512] The operator is healthy I0325 11:30:56.787945 1 configmapobserver.go:84] configmaps "insights-config" not found I0325 11:30:56.821649 1 controller.go:512] The operator is healthy I0325 11:30:56.854587 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:30:56.861256 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:30:56.861281 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:30:56.866912 1 secretconfigobserver.go:119] support secret does not exist I0325 11:32:56.821312 1 controller.go:512] The operator is healthy I0325 11:34:51.528463 1 insightsuploader.go:146] Nothing to report since 2026-03-25T11:02:26Z I0325 11:34:56.824163 1 controller.go:512] The operator is healthy I0325 11:35:56.867769 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0325 11:35:56.878326 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0325 11:35:56.878354 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0325 11:35:56.883377 1 secretconfigobserver.go:119] support secret does not exist I0325 11:36:56.823508 1 controller.go:512] The operator is healthy