I0324 09:03:55.583686 1 cmd.go:253] Using service-serving-cert provided certificates I0324 09:03:55.584160 1 observer_polling.go:159] Starting file observer I0324 09:03:55.923561 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0324 09:03:55.923804 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0324 09:03:55.924278 1 secure_serving.go:57] Forcing use of http/1.1 only W0324 09:03:55.924296 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0324 09:03:55.924302 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0324 09:03:55.924306 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0324 09:03:55.924309 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0324 09:03:55.924311 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0324 09:03:55.924314 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0324 09:03:55.924362 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0324 09:03:55.929465 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0324 09:03:55.929484 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"9ea6d7dd-856b-49f6-8c59-aeb5ad249036", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0324 09:03:55.930051 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0324 09:03:55.930068 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0324 09:03:55.930093 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0324 09:03:55.930088 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0324 09:03:55.930105 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0324 09:03:55.930111 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0324 09:03:55.930293 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0324 09:03:55.930513 1 secure_serving.go:213] Serving securely on [::]:8443 I0324 09:03:55.930537 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0324 09:03:55.938324 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0324 09:03:55.938351 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0324 09:03:55.938433 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0324 09:03:55.943015 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:03:55.943031 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0324 09:03:55.946720 1 secretconfigobserver.go:119] support secret does not exist I0324 09:03:55.950733 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:03:55.954634 1 secretconfigobserver.go:119] support secret does not exist I0324 09:03:55.957249 1 recorder.go:161] Pruning old reports every 8h22m11s, max age is 288h0m0s I0324 09:03:55.961506 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0324 09:03:55.961520 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0324 09:03:55.961526 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0324 09:03:55.961530 1 insightsreport.go:296] Starting report retriever I0324 09:03:55.961541 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0324 09:03:55.961552 1 periodic.go:209] Running clusterconfig gatherer I0324 09:03:55.961623 1 tasks_processing.go:45] number of workers: 32 I0324 09:03:55.961656 1 tasks_processing.go:69] worker 31 listening for tasks. I0324 09:03:55.961663 1 tasks_processing.go:71] worker 31 working on nodes task. I0324 09:03:55.961686 1 tasks_processing.go:69] worker 15 listening for tasks. I0324 09:03:55.961693 1 tasks_processing.go:69] worker 7 listening for tasks. I0324 09:03:55.961699 1 tasks_processing.go:71] worker 7 working on storage_cluster task. I0324 09:03:55.961700 1 tasks_processing.go:69] worker 8 listening for tasks. I0324 09:03:55.961709 1 tasks_processing.go:69] worker 9 listening for tasks. I0324 09:03:55.961714 1 tasks_processing.go:69] worker 5 listening for tasks. I0324 09:03:55.961716 1 tasks_processing.go:69] worker 10 listening for tasks. I0324 09:03:55.961723 1 tasks_processing.go:69] worker 11 listening for tasks. I0324 09:03:55.961728 1 tasks_processing.go:69] worker 4 listening for tasks. I0324 09:03:55.961733 1 tasks_processing.go:69] worker 12 listening for tasks. I0324 09:03:55.961737 1 tasks_processing.go:69] worker 19 listening for tasks. I0324 09:03:55.961735 1 tasks_processing.go:69] worker 23 listening for tasks. I0324 09:03:55.961741 1 tasks_processing.go:69] worker 14 listening for tasks. I0324 09:03:55.961744 1 tasks_processing.go:69] worker 16 listening for tasks. I0324 09:03:55.961719 1 tasks_processing.go:69] worker 3 listening for tasks. I0324 09:03:55.961748 1 tasks_processing.go:69] worker 20 listening for tasks. I0324 09:03:55.961751 1 tasks_processing.go:69] worker 27 listening for tasks. I0324 09:03:55.961752 1 tasks_processing.go:69] worker 13 listening for tasks. I0324 09:03:55.961756 1 tasks_processing.go:69] worker 0 listening for tasks. I0324 09:03:55.961762 1 tasks_processing.go:69] worker 24 listening for tasks. I0324 09:03:55.961761 1 tasks_processing.go:69] worker 22 listening for tasks. I0324 09:03:55.961766 1 tasks_processing.go:69] worker 25 listening for tasks. I0324 09:03:55.961767 1 tasks_processing.go:69] worker 18 listening for tasks. I0324 09:03:55.961755 1 tasks_processing.go:69] worker 21 listening for tasks. I0324 09:03:55.961770 1 tasks_processing.go:69] worker 26 listening for tasks. I0324 09:03:55.961777 1 tasks_processing.go:69] worker 2 listening for tasks. I0324 09:03:55.961776 1 tasks_processing.go:69] worker 29 listening for tasks. I0324 09:03:55.961781 1 tasks_processing.go:69] worker 30 listening for tasks. I0324 09:03:55.961783 1 tasks_processing.go:69] worker 6 listening for tasks. I0324 09:03:55.961760 1 tasks_processing.go:69] worker 17 listening for tasks. I0324 09:03:55.961770 1 tasks_processing.go:69] worker 1 listening for tasks. I0324 09:03:55.961694 1 tasks_processing.go:71] worker 15 working on config_maps task. I0324 09:03:55.961835 1 tasks_processing.go:71] worker 1 working on node_logs task. I0324 09:03:55.961901 1 tasks_processing.go:71] worker 3 working on openstack_dataplanedeployments task. I0324 09:03:55.961895 1 tasks_processing.go:71] worker 18 working on version task. I0324 09:03:55.961777 1 tasks_processing.go:69] worker 28 listening for tasks. I0324 09:03:55.961932 1 tasks_processing.go:71] worker 28 working on image task. I0324 09:03:55.961863 1 tasks_processing.go:71] worker 8 working on openshift_logging task. I0324 09:03:55.961965 1 tasks_processing.go:71] worker 29 working on monitoring_persistent_volumes task. I0324 09:03:55.962098 1 tasks_processing.go:71] worker 20 working on validating_webhook_configurations task. I0324 09:03:55.962140 1 tasks_processing.go:71] worker 17 working on pdbs task. I0324 09:03:55.962183 1 tasks_processing.go:71] worker 2 working on install_plans task. I0324 09:03:55.961858 1 tasks_processing.go:71] worker 27 working on machine_sets task. I0324 09:03:55.961867 1 tasks_processing.go:71] worker 9 working on openstack_dataplanenodesets task. I0324 09:03:55.961866 1 tasks_processing.go:71] worker 21 working on active_alerts task. I0324 09:03:55.961871 1 tasks_processing.go:71] worker 5 working on cluster_apiserver task. I0324 09:03:55.961871 1 tasks_processing.go:71] worker 13 working on metrics task. I0324 09:03:55.961874 1 tasks_processing.go:71] worker 10 working on sap_datahubs task. I0324 09:03:55.961878 1 tasks_processing.go:71] worker 0 working on openshift_machine_api_events task. I0324 09:03:55.961881 1 tasks_processing.go:71] worker 4 working on mutating_webhook_configurations task. I0324 09:03:55.961883 1 tasks_processing.go:71] worker 24 working on oauths task. I0324 09:03:55.961884 1 tasks_processing.go:71] worker 12 working on sap_config task. I0324 09:03:55.961887 1 tasks_processing.go:71] worker 22 working on pod_network_connectivity_checks task. I0324 09:03:55.961888 1 tasks_processing.go:71] worker 19 working on operators_pods_and_events task. I0324 09:03:55.961891 1 tasks_processing.go:71] worker 25 working on openstack_version task. I0324 09:03:55.961891 1 tasks_processing.go:71] worker 23 working on machine_configs task. I0324 09:03:55.961895 1 tasks_processing.go:71] worker 14 working on clusterroles task. I0324 09:03:55.961898 1 tasks_processing.go:71] worker 16 working on feature_gates task. I0324 09:03:55.962115 1 tasks_processing.go:71] worker 30 working on machine_config_pools task. I0324 09:03:55.962120 1 tasks_processing.go:71] worker 6 working on ceph_cluster task. I0324 09:03:55.962130 1 tasks_processing.go:71] worker 26 working on crds task. I0324 09:03:55.961878 1 tasks_processing.go:71] worker 11 working on container_runtime_configs task. I0324 09:03:55.965042 1 tasks_processing.go:71] worker 8 working on operators task. I0324 09:03:55.965068 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 3.092217ms to process 0 records I0324 09:03:55.965087 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 3.141929ms to process 0 records I0324 09:03:55.965092 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 3.358064ms to process 0 records I0324 09:03:55.965098 1 tasks_processing.go:71] worker 7 working on qemu_kubevirt_launcher_logs task. I0324 09:03:55.965237 1 tasks_processing.go:71] worker 3 working on networks task. E0324 09:03:55.967299 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.23.27:9094: connect: connection refused I0324 09:03:55.967324 1 tasks_processing.go:71] worker 21 working on certificate_signing_requests task. E0324 09:03:55.967317 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.144.37:9091: connect: connection refused E0324 09:03:55.967337 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.23.27:9094: connect: connection refused I0324 09:03:55.967358 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 4.939586ms to process 0 records E0324 09:03:55.967375 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.144.37:9091: connect: connection refused I0324 09:03:55.967392 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 4.762607ms to process 0 records I0324 09:03:55.967405 1 tasks_processing.go:71] worker 13 working on openstack_controlplanes task. I0324 09:03:55.968333 1 tasks_processing.go:71] worker 10 working on olm_operators task. I0324 09:03:55.968351 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 5.664734ms to process 0 records I0324 09:03:55.968367 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 6.010666ms to process 0 records I0324 09:03:55.968375 1 tasks_processing.go:71] worker 27 working on image_registries task. I0324 09:03:55.968675 1 tasks_processing.go:71] worker 28 working on overlapping_namespace_uids task. I0324 09:03:55.968796 1 recorder.go:75] Recording config/image with fingerprint=fa059c133f253b99fce44549dce1acedf42de3ee826720e5a6c70c79c1220713 I0324 09:03:55.968811 1 gather.go:177] gatherer "clusterconfig" function "image" took 6.727806ms to process 1 records I0324 09:03:55.968865 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0324 09:03:55.968884 1 controller.go:326] The initial operator extension status is healthy I0324 09:03:55.968898 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0324 09:03:55.968907 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0324 09:03:55.968912 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0324 09:03:55.968957 1 controller.go:489] The operator is still being initialized I0324 09:03:55.968968 1 controller.go:512] The operator is healthy I0324 09:03:55.969050 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0324 09:03:55.969050 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0324 09:03:55.969157 1 operator.go:288] started I0324 09:03:55.969185 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0324 09:03:55.971151 1 tasks_processing.go:71] worker 9 working on authentication task. I0324 09:03:55.971172 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 8.781873ms to process 0 records I0324 09:03:55.971374 1 tasks_processing.go:71] worker 31 working on container_images task. I0324 09:03:55.972384 1 recorder.go:75] Recording config/node/ip-10-0-0-191.ec2.internal with fingerprint=85976e61ecf3231170b17a5160f196fd8c5556898b184005c2a5cacfdf9b72e1 I0324 09:03:55.972632 1 recorder.go:75] Recording config/node/ip-10-0-1-92.ec2.internal with fingerprint=6a53aea8d8a02262208cc2010e28cf3fc5a08bc0e46d9d2a8fff0a3d9ad59adb I0324 09:03:55.972789 1 recorder.go:75] Recording config/node/ip-10-0-2-34.ec2.internal with fingerprint=c74a47817556518d9fcdb9420d97eebf576df24f4e2382eff279848abd2d6f7d I0324 09:03:55.972823 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 9.703869ms to process 3 records I0324 09:03:55.976842 1 tasks_processing.go:71] worker 1 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0324 09:03:55.976959 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 14.931922ms to process 0 records I0324 09:03:55.977100 1 tasks_processing.go:71] worker 17 working on sap_pods task. I0324 09:03:55.977165 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=cdcea7ebd71683852e77bb2d7a302578546d630ec5a8411c47de9de134a0ae09 I0324 09:03:55.977218 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0324 09:03:55.977270 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0324 09:03:55.977297 1 controller.go:212] Source scaController *sca.Controller is not ready I0324 09:03:55.977321 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0324 09:03:55.977343 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0324 09:03:55.977381 1 controller.go:489] The operator is still being initialized I0324 09:03:55.977408 1 controller.go:512] The operator is healthy I0324 09:03:55.977222 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=50732b72f887eb07a550cd69f2d7f95c97f876e47f988914b407329c89d4592f I0324 09:03:55.977503 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=e5321c0dcedc39a820f0410fd143d756f7604db2195bc1f063d00256e93a086a I0324 09:03:55.977534 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=fe2d3343dbec6f6342bc8cc8d9a264cdfab064a128dc1a271f89334a53960e40 I0324 09:03:55.977558 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=c5fa63d6fb4577488f33d810695bc80677b1245291af31d05421e9e22c8afb10 I0324 09:03:55.977584 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=a6a4d21ae1c6bbc87b7cfdceaeee67ba30d33028d6185608239a514450c8e2d8 I0324 09:03:55.977630 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=3fccf747da356f7a594bddf9a1c03c2702c73a9f4aab4ba3b5f2ca2284797f7c I0324 09:03:55.977652 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=63f24b28c0036cbe1c2d9ee92e8fbf3d774fb1fb8c5ab29edf8bd253437b1e22 I0324 09:03:55.977677 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=2a7c8d6744e5e2f06d6c0eea9e8098612cabbefe6564de993ee2d6c589d6bd15 I0324 09:03:55.977690 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 14.769319ms to process 9 records I0324 09:03:55.977705 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 13.760399ms to process 0 records I0324 09:03:55.977717 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 14.17755ms to process 0 records E0324 09:03:55.977724 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0324 09:03:55.977732 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 13.726222ms to process 0 records I0324 09:03:55.977742 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 14.987506ms to process 0 records I0324 09:03:55.977750 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 13.565543ms to process 0 records I0324 09:03:55.977755 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 12.986156ms to process 0 records I0324 09:03:55.977760 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 13.160068ms to process 0 records I0324 09:03:55.977769 1 tasks_processing.go:71] worker 22 working on image_pruners task. I0324 09:03:55.977797 1 tasks_processing.go:71] worker 0 working on dvo_metrics task. I0324 09:03:55.977888 1 tasks_processing.go:71] worker 11 working on machine_healthchecks task. I0324 09:03:55.977940 1 tasks_processing.go:71] worker 24 working on ingress task. I0324 09:03:55.977981 1 tasks_processing.go:71] worker 29 working on machines task. I0324 09:03:55.978065 1 recorder.go:75] Recording config/oauth with fingerprint=8fa06515064729c5c4da6cdb0156c6f842e7cde93b11b8032410334ea4c3e2f7 I0324 09:03:55.978098 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 14.467029ms to process 1 records I0324 09:03:55.977984 1 tasks_processing.go:71] worker 12 working on tsdb_status task. I0324 09:03:55.977968 1 tasks_processing.go:71] worker 25 working on machine_autoscalers task. I0324 09:03:55.978210 1 tasks_processing.go:71] worker 5 working on ingress_certificates task. I0324 09:03:55.977999 1 tasks_processing.go:71] worker 6 working on aggregated_monitoring_cr_names task. I0324 09:03:55.978236 1 recorder.go:75] Recording config/apiserver with fingerprint=f92b92d57929eb8ed5f2c60c4280b7d5928f082d92c092c6d841a8e4cfcbaf03 I0324 09:03:55.978253 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 15.075749ms to process 1 records I0324 09:03:55.979641 1 tasks_processing.go:71] worker 13 working on silenced_alerts task. I0324 09:03:55.979651 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 12.221635ms to process 0 records E0324 09:03:55.980092 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.144.37:9091: connect: connection refused I0324 09:03:55.980107 1 tasks_processing.go:71] worker 12 working on nodenetworkstates task. E0324 09:03:55.980125 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.144.37:9091: connect: connection refused I0324 09:03:55.980146 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 1.970485ms to process 0 records E0324 09:03:55.981555 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.23.27:9094: connect: connection refused I0324 09:03:55.981572 1 tasks_processing.go:71] worker 13 working on service_accounts task. E0324 09:03:55.981585 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.23.27:9094: connect: connection refused I0324 09:03:55.981624 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.915315ms to process 0 records I0324 09:03:55.988218 1 tasks_processing.go:71] worker 4 working on jaegers task. I0324 09:03:55.988424 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=c4fd9cd4e1777560623f13b43e91969f314ce481ee8038a3e0a8168e29c45094 I0324 09:03:55.988458 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=95f4c861778fbc6653bfd20967473d0a9cc028517227582725ac7923ca1109d8 I0324 09:03:55.988480 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=85802167a9f92c46b5b2b14c70a919a79069abb1e28f3526ff59f97d6d1d666f I0324 09:03:55.988489 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 25.368898ms to process 3 records I0324 09:03:55.989002 1 tasks_processing.go:71] worker 28 working on schedulers task. I0324 09:03:55.989020 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0324 09:03:55.989031 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 20.300492ms to process 1 records I0324 09:03:55.990106 1 tasks_processing.go:71] worker 30 working on lokistack task. I0324 09:03:55.990118 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 26.165869ms to process 0 records I0324 09:03:55.990364 1 tasks_processing.go:71] worker 9 working on cost_management_metrics_configs task. I0324 09:03:55.990817 1 recorder.go:75] Recording config/authentication with fingerprint=1ba1ecc9046d6064abb176214787c0ba01ee2bf2eb9684f782857ed6286d73e5 I0324 09:03:55.990889 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 19.188536ms to process 1 records I0324 09:03:55.990922 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 13.416695ms to process 0 records I0324 09:03:55.991037 1 tasks_processing.go:71] worker 17 working on storage_classes task. I0324 09:03:55.991696 1 recorder.go:75] Recording config/network with fingerprint=c584b34e4bbb0933bd9b51cb36605eb6300dfca8039ff0c5445766d5e5c1cf63 I0324 09:03:55.991773 1 gather.go:177] gatherer "clusterconfig" function "networks" took 26.02037ms to process 1 records I0324 09:03:55.991934 1 tasks_processing.go:71] worker 3 working on proxies task. I0324 09:03:55.992071 1 tasks_processing.go:71] worker 16 working on nodenetworkconfigurationpolicies task. I0324 09:03:55.992491 1 recorder.go:75] Recording config/featuregate with fingerprint=07942ed45eb3005cc2952dc7c8862a7441162a1f4a1b22ac1da142f8418b3fc7 I0324 09:03:55.992535 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 28.201141ms to process 1 records I0324 09:03:55.992705 1 tasks_processing.go:71] worker 27 working on infrastructures task. I0324 09:03:55.993639 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=a4a217736e0e7fd7a335425a0e41692a043dbec443ebbd2e37ca8b71112a0929 I0324 09:03:55.993697 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 24.160171ms to process 1 records I0324 09:03:55.995284 1 tasks_processing.go:71] worker 12 working on support_secret task. I0324 09:03:55.995341 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 15.168238ms to process 0 records I0324 09:03:55.995478 1 tasks_processing.go:74] worker 24 stopped. I0324 09:03:55.995562 1 recorder.go:75] Recording config/ingress with fingerprint=08acffdfb29b7f6df108274a4980d831ebd04b9495ec205f1d9b438717a229f2 I0324 09:03:55.995631 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 17.43028ms to process 1 records I0324 09:03:55.995717 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=6cd9329c4bde13dacddc1fee4d23a46914a086d9bc4655d921cdd29134977227 I0324 09:03:55.995730 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 17.673017ms to process 1 records I0324 09:03:55.995746 1 tasks_processing.go:74] worker 22 stopped. I0324 09:03:55.995823 1 tasks_processing.go:74] worker 18 stopped. I0324 09:03:55.995940 1 gather_logs.go:145] no pods in namespace were found I0324 09:03:55.995993 1 recorder.go:75] Recording config/version with fingerprint=cee9bd37d8a70ce7d9b2d4148c92b2e2987780fe4200154a170328be2ac574ac I0324 09:03:55.996010 1 recorder.go:75] Recording config/id with fingerprint=99b16b0b2698eea7271d1f48584d27e752260961ed37ec222ac6b7d32a781d0e I0324 09:03:55.996016 1 gather.go:177] gatherer "clusterconfig" function "version" took 33.611746ms to process 2 records I0324 09:03:55.996026 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 17.66749ms to process 0 records E0324 09:03:55.996034 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0324 09:03:55.996041 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 18.014401ms to process 0 records I0324 09:03:55.996052 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 28.614974ms to process 0 records I0324 09:03:55.996060 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 30.849694ms to process 0 records I0324 09:03:55.996060 1 tasks_processing.go:74] worker 25 stopped. E0324 09:03:55.996068 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0324 09:03:55.996071 1 tasks_processing.go:74] worker 21 stopped. I0324 09:03:55.996075 1 gather.go:177] gatherer "clusterconfig" function "machines" took 18.038211ms to process 0 records I0324 09:03:55.996077 1 tasks_processing.go:74] worker 11 stopped. I0324 09:03:55.996084 1 tasks_processing.go:74] worker 7 stopped. I0324 09:03:55.996084 1 tasks_processing.go:74] worker 29 stopped. I0324 09:03:55.998090 1 tasks_processing.go:74] worker 14 stopped. I0324 09:03:55.998422 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=aab4e1a5caa07a02eb48d328e0034ffb9a31a5f28ff96b7833c55a4247d26e5d I0324 09:03:55.998773 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=0213a6108bc6245c5b9b8a3be7f09d10f3134c5d80de745864b22f4ac6abfaee I0324 09:03:55.998809 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 34.341411ms to process 2 records I0324 09:03:55.998832 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 8.040532ms to process 0 records I0324 09:03:55.998855 1 tasks_processing.go:74] worker 30 stopped. I0324 09:03:55.998960 1 tasks_processing.go:74] worker 4 stopped. I0324 09:03:55.999003 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 10.724495ms to process 0 records I0324 09:03:56.000146 1 tasks_processing.go:74] worker 28 stopped. I0324 09:03:56.000263 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e5f3ac47935406cb458bdf41ab19f1e359d29a5441b620360f2efeda1ece017a I0324 09:03:56.000283 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 11.127057ms to process 1 records W0324 09:03:56.002430 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0324 09:03:56.006165 1 tasks_processing.go:74] worker 9 stopped. I0324 09:03:56.006182 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 15.78231ms to process 0 records I0324 09:03:56.006242 1 tasks_processing.go:74] worker 16 stopped. I0324 09:03:56.006258 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 14.117481ms to process 0 records I0324 09:03:56.013869 1 tasks_processing.go:74] worker 12 stopped. E0324 09:03:56.013886 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0324 09:03:56.013895 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 18.532367ms to process 0 records I0324 09:03:56.013976 1 tasks_processing.go:74] worker 3 stopped. I0324 09:03:56.014052 1 recorder.go:75] Recording config/proxy with fingerprint=02d5ec477e628400fac7f6ab7c0b0eae48e9d662e0ffeaf3532c211d88583b64 I0324 09:03:56.014069 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 22.005868ms to process 1 records I0324 09:03:56.014459 1 tasks_processing.go:74] worker 17 stopped. I0324 09:03:56.014568 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=909570ca69f807f89b5150f583fba9ddd32d3be86f0c0a0ee8a51223f9461846 I0324 09:03:56.014591 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=87e3257892c435788dd1c7875ec170f5df9e7a7b9824f492a89d8a345860ce11 I0324 09:03:56.014618 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 23.391608ms to process 2 records I0324 09:03:56.014640 1 tasks_processing.go:74] worker 27 stopped. I0324 09:03:56.015358 1 recorder.go:75] Recording config/infrastructure with fingerprint=6d9fc1ceceec2b82643b2729a03880d89efdd1ec5a282312d4039675532ae7d1 I0324 09:03:56.015376 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 21.878933ms to process 1 records I0324 09:03:56.015464 1 tasks_processing.go:74] worker 26 stopped. I0324 09:03:56.016002 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=c6c1393749941b83a33ea484a47079fffc5c0f70592c53472c6ad4740cf37afd I0324 09:03:56.016285 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=2648054233673d9bc6b63a05f2d7926c35c0683ec046ec739073e0c0108769c4 I0324 09:03:56.016303 1 gather.go:177] gatherer "clusterconfig" function "crds" took 50.670122ms to process 2 records I0324 09:03:56.016352 1 recorder.go:75] Recording config/olm_operators with fingerprint=e8c1f10f15c98c7ad9062d1357e170cd8c9da1afb5a7960774dd8e45cc8dda01 I0324 09:03:56.016365 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 46.677761ms to process 1 records I0324 09:03:56.016375 1 tasks_processing.go:74] worker 10 stopped. I0324 09:03:56.017052 1 tasks_processing.go:74] worker 20 stopped. I0324 09:03:56.017709 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=99711eda7425502b32ce93afa313754e98891c2a90cced1f436c39d717c4e00d I0324 09:03:56.017852 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=ecdb9f47cc466ab6eda64bec8844286b0461fff9a47462be30a55b0dbf8fb212 I0324 09:03:56.018002 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=f513afc594e0f00173a4b3bf678b638b6dc72d853a7096f64d237230253e6178 I0324 09:03:56.018206 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=6e9342bfe39bda72262555fa84f6340b9d00a08a679f8819695dc2559b3bae72 I0324 09:03:56.018281 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=cd3debfcb0ab647982f7c4152ff3685942143feac4e33cdcab8dd04804f9869f I0324 09:03:56.018365 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=ade012175b6f1345c34bdaad922ba5302dd0d775c39fa2b20d2df6725a98bdbc I0324 09:03:56.018437 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=c647868c5f6b7b5f7092d973aeefecfc8903bde5c0d9659775c14f12ae4d2a2d I0324 09:03:56.018508 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=66161ebd749679e332d6b6b7810efbf9ff21a7f907fab1c5ae896a6227808ee5 I0324 09:03:56.018579 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=e5fff5ef82226e25a53f7926c48761c256cc3a7c3b3860e8f404b38f76b18628 I0324 09:03:56.018675 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=c2481473650213811f8a07c37b11788d01f48c8c287cf7bb087bf5903a44942f I0324 09:03:56.018752 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=a1365f7dc5e17d3db40e5f6bf1575b30bfa0e825e37a906ca1ed797bd7df8349 I0324 09:03:56.018823 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=639bf3f1d704e3250cca215defc05f00fc8ce9571aad1dd0132370b27772263c I0324 09:03:56.018893 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=f76f849677043cab02efc243c97c2aee95b9de339d8b57f867609012ca04425c I0324 09:03:56.018925 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 54.89996ms to process 13 records I0324 09:03:56.019312 1 tasks_processing.go:74] worker 6 stopped. I0324 09:03:56.019373 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 41.068594ms to process 0 records I0324 09:03:56.021815 1 tasks_processing.go:74] worker 23 stopped. I0324 09:03:56.021890 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0324 09:03:56.021925 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 58.300049ms to process 1 records I0324 09:03:56.022192 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0324 09:03:56.025943 1 tasks_processing.go:74] worker 31 stopped. I0324 09:03:56.028551 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-jm2wr with fingerprint=f37dd2a9e7cac283bf457ba13f7a315236d60161beec4d7ca514004ed6b0de32 I0324 09:03:56.028802 1 recorder.go:75] Recording config/pod/openshift-monitoring/cluster-monitoring-operator-698978dbbf-f6fg7 with fingerprint=75273b236f0c9c5d9060485703a5bfb8fd3fa05aed7ad11a6a39c826a526d83f I0324 09:03:56.029376 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-z674x with fingerprint=c75f65cb67418b80ad476e4c27fd8d27eca093563e3d294e952da404a6ff156f I0324 09:03:56.029453 1 recorder.go:75] Recording config/running_containers with fingerprint=51ca5c1b02b295995ea22140d72d0ad77c50f19500d06be9ddac836e5eff32ab I0324 09:03:56.029463 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 54.541493ms to process 4 records I0324 09:03:56.029473 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 52.530975ms to process 0 records I0324 09:03:56.029485 1 tasks_processing.go:74] worker 1 stopped. I0324 09:03:56.030720 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0324 09:03:56.030734 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0324 09:03:56.030744 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0324 09:03:56.032464 1 tasks_processing.go:74] worker 15 stopped. E0324 09:03:56.032479 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0324 09:03:56.032487 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0324 09:03:56.032492 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0324 09:03:56.032503 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0324 09:03:56.032536 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0324 09:03:56.032561 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0324 09:03:56.032570 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0324 09:03:56.032583 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0324 09:03:56.032654 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0324 09:03:56.032664 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0324 09:03:56.032671 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 70.656168ms to process 7 records I0324 09:03:56.039072 1 base_controller.go:82] Caches are synced for ConfigController I0324 09:03:56.039086 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0324 09:03:56.051525 1 configmapobserver.go:84] configmaps "insights-config" not found I0324 09:03:56.069584 1 base_controller.go:82] Caches are synced for LoggingSyncer I0324 09:03:56.069618 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0324 09:03:56.075031 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0324 09:03:56.079923 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0324 09:03:56.089940 1 cluster_transfer.go:102] no available accepted cluster transfer I0324 09:03:56.089954 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0324 09:03:56.124170 1 tasks_processing.go:74] worker 5 stopped. E0324 09:03:56.124186 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0324 09:03:56.124193 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2p801npmg0omcqigltaint51nceap4pa-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2p801npmg0omcqigltaint51nceap4pa-primary-cert-bundle-secret" not found I0324 09:03:56.124271 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=d4914855911966f173de054287b9ce94132cdfd6b928e991c39265b05d488b0c I0324 09:03:56.124288 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 145.936448ms to process 1 records I0324 09:03:56.156258 1 sca.go:266] etc-pki-entitlement secret successfully created I0324 09:03:56.156275 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0324 09:03:56.408337 1 gather_cluster_operator_pods_and_events.go:121] Found 13 pods with 17 containers I0324 09:03:56.408353 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1480342 bytes I0324 09:03:56.409287 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-8dx87 pod in namespace openshift-dns (previous: false). I0324 09:03:56.614968 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-8dx87 pod in namespace openshift-dns (previous: false). I0324 09:03:56.810067 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-czspk pod in namespace openshift-dns (previous: false). W0324 09:03:57.001256 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0324 09:03:57.012085 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-czspk pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-czspk\" is waiting to start: ContainerCreating" I0324 09:03:57.012102 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-czspk\" is waiting to start: ContainerCreating" I0324 09:03:57.012109 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-czspk pod in namespace openshift-dns (previous: false). I0324 09:03:57.210483 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-czspk pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-czspk\" is waiting to start: ContainerCreating" I0324 09:03:57.210499 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-czspk\" is waiting to start: ContainerCreating" I0324 09:03:57.210530 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-f7xkr pod in namespace openshift-dns (previous: false). I0324 09:03:57.411408 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-f7xkr pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-f7xkr\" is waiting to start: ContainerCreating" I0324 09:03:57.411425 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-f7xkr\" is waiting to start: ContainerCreating" I0324 09:03:57.411433 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-f7xkr pod in namespace openshift-dns (previous: false). I0324 09:03:57.609655 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-f7xkr pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-f7xkr\" is waiting to start: ContainerCreating" I0324 09:03:57.609671 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-f7xkr\" is waiting to start: ContainerCreating" I0324 09:03:57.609680 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-ppgfc pod in namespace openshift-dns (previous: false). I0324 09:03:57.815578 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0324 09:03:57.815610 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-pwqq8 pod in namespace openshift-dns (previous: false). W0324 09:03:58.000016 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0324 09:03:58.009098 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0324 09:03:58.009115 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-sswrx pod in namespace openshift-dns (previous: false). I0324 09:03:58.209537 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0324 09:03:58.209556 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-659d68dfb9-td5cb pod in namespace openshift-ingress (previous: false). I0324 09:03:58.411669 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-d97595c9b-bfm8f pod in namespace openshift-ingress (previous: false). I0324 09:03:58.429452 1 tasks_processing.go:74] worker 8 stopped. I0324 09:03:58.429577 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=efde84b313e2a97ced7e919e7b932fc9cee933b2f9f30460ae22326f275b33a1 I0324 09:03:58.429660 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0324 09:03:58.429701 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=9cd10e8c447df2ca8aa2bed9f4eb523f5b8637ef2c5524b2e784e301f4646746 I0324 09:03:58.429717 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0324 09:03:58.429754 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=812deb4fab40aeb57eada1ee2181953f2d11bfca5fc93c955c71fa11bc670577 I0324 09:03:58.429779 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0324 09:03:58.429813 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=4453fdcdf53e0263ae3ba7d5e28bcd39f7de77d6e03a295e20e321cc3fe66dd8 I0324 09:03:58.429857 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=217813e5be022feffc2284deb255afe79af5aac2b3cef8289ca8522233e770f7 I0324 09:03:58.429893 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=0d519917d03d71ea107d48bdf883a55b562b6b2eab68186237cd466ac9a79545 I0324 09:03:58.429933 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=473af53bfc46a2a39fd534cf089712cbf07d8d6233d83cdbc7c375cadb0bf64d I0324 09:03:58.430204 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=5174bf48d4490fbe6a62857866091266e870380429b83ddb61721f98eee90bbe I0324 09:03:58.430235 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=88838a5b14df8755781108681f18135d62b42a3e9e59428025fbb2361444a920 I0324 09:03:58.430250 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0324 09:03:58.430276 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=b2c39ae21a04b44a58ed624e338f144f8c413c27c9f253ded7d9ed617cf92283 I0324 09:03:58.430291 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0324 09:03:58.430315 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=6ea0a9b4b3e4990e6611f8337264731056ead90aefd8adef509b96f1d199cd26 I0324 09:03:58.430329 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0324 09:03:58.430357 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=5b9069206c9ea1c40f311dbf3dc59fa54c3d0fdc044c96755a7fed4e50d5f20d I0324 09:03:58.430371 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0324 09:03:58.430402 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=65204087f3a59eec0303a2b629f7842882e3c222901f6484763fdb2620380fee I0324 09:03:58.430512 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=2a416f77372fa350d2589ae14af06c5aed6a23bdcab796daf91904ca1bf41f6d I0324 09:03:58.430526 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0324 09:03:58.430537 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0324 09:03:58.430568 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0324 09:03:58.430623 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=db9b60c62691b554eb1892b6861d243346116ad325bb721849746c8a563db3e6 I0324 09:03:58.430660 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=108fa024440930fe3713bdb41fd24ff0d1dcc3c601bbe3a8374c811a17dcca8a I0324 09:03:58.430675 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0324 09:03:58.430700 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=fcc81cc5c770bae5cae9b5ec63e05054689569845b8021c8e686d007849a289c I0324 09:03:58.430715 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0324 09:03:58.430741 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=e590cf542ce1c751b40766de9bad28f03d1b63c19a340459a131cc620d9459aa I0324 09:03:58.430755 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0324 09:03:58.430768 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=2122d26cfb73c00c46fb4b220228df63e40ed22b2fa7f482847d9a7a3e0d3fcb I0324 09:03:58.430781 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=0b1e59747d946aceb2600c28643d15d94bdc2b8b87afe3738ae5db546c2c0784 I0324 09:03:58.430795 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=4edd0f5497528b2528f9968b5255d3e2696fda6365f599b5ea215b852246c6d8 I0324 09:03:58.430816 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=c4120e0c4acc7852a177eff6807e0aeb6304f15f1ee5f5b342a1e45919da5150 I0324 09:03:58.430826 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0324 09:03:58.430850 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=7d171bd2a6d2ceef92bd96605939abcbca5b5c8b1dc776a628de0dd90dfd331b I0324 09:03:58.430867 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0324 09:03:58.430874 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0324 09:03:58.430881 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.4643866s to process 39 records I0324 09:03:58.610750 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-d97595c9b-nvrwz pod in namespace openshift-ingress (previous: false). I0324 09:03:58.810634 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-kc97f pod in namespace openshift-ingress-canary (previous: false). W0324 09:03:59.000232 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0324 09:03:59.014562 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-plfzb pod in namespace openshift-ingress-canary (previous: false). I0324 09:03:59.214354 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-plfzb pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-plfzb\" is waiting to start: ContainerCreating" I0324 09:03:59.214380 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-plfzb\" is waiting to start: ContainerCreating" I0324 09:03:59.214419 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-rwq9d pod in namespace openshift-ingress-canary (previous: false). I0324 09:03:59.410402 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-rwq9d pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rwq9d\" is waiting to start: ContainerCreating" I0324 09:03:59.410420 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rwq9d\" is waiting to start: ContainerCreating" I0324 09:03:59.410432 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-samples-operator container cluster-samples-operator-8696bc5b94-h8vk9 pod in namespace openshift-cluster-samples-operator (previous: false). I0324 09:03:59.616948 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-samples-operator-watch container cluster-samples-operator-8696bc5b94-h8vk9 pod in namespace openshift-cluster-samples-operator (previous: false). I0324 09:03:59.809979 1 tasks_processing.go:74] worker 19 stopped. I0324 09:03:59.810113 1 recorder.go:75] Recording events/openshift-dns with fingerprint=375ef82058a1dad93b232d591694d7c94acc91377067507d48cf338be69d2999 I0324 09:03:59.810156 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=550beec7b99be3703ee0feadd92202d492c713a98d0f62f00033c210f8929235 I0324 09:03:59.810233 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=c90ac904b44b5b54a7ee89be38297b595cdd0e2279df2a7e4cfe3ae4b2f9f21e I0324 09:03:59.810259 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=5878c455f76d89547b54969afc96f366ef7060c17601ddd5e93751ed3285c54b I0324 09:03:59.810295 1 recorder.go:75] Recording events/openshift-cluster-samples-operator with fingerprint=ceb336765c8307da9523a2e2b9883e4913bbb99e797cb1d9b2704762f92f3756 I0324 09:03:59.810308 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-8dx87/dns_current.log with fingerprint=5cf8e0c01ec82b7a09031a64e979aff412130d2f1261d492d4f309b6fd7220ea I0324 09:03:59.810315 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-8dx87/kube-rbac-proxy_current.log with fingerprint=30caad83984f42c360aefa5f2a30b3f351586b1ba56e3f852451117290eb947a I0324 09:03:59.810461 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-czspk with fingerprint=10980214b9ebb5cab864f0732452963b09b6f2b50f3a354905cc02358a3f6cae I0324 09:03:59.810548 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-f7xkr with fingerprint=c39332b2e4eaf2a8ed2860bdd2a61464d2a5078ed8ca25ccac905b665622adf6 I0324 09:03:59.810580 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-659d68dfb9-td5cb/router_current.log with fingerprint=2c7689765d12860ed347a4484a2684489dc03787dfbc6d73cca759b1ce139011 I0324 09:03:59.810639 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-d97595c9b-bfm8f/router_current.log with fingerprint=d53246b4fdd69f647cb293e6cec157b59a6c597c933c4655145cdd4a7c8bc9e6 I0324 09:03:59.810709 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-d97595c9b-nvrwz/router_current.log with fingerprint=a8eb3601af1ee8435cc327549879052063816b059deb3b8f46aa7f3ad78ec7e5 I0324 09:03:59.810716 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-kc97f/serve-healthcheck-canary_current.log with fingerprint=2e72d559ae4d6493af321ae7c09e52c949de3171677748dd2df8d0331d2720c5 I0324 09:03:59.810787 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-plfzb with fingerprint=18a2529fbd4844411e65227ba2af3a86e0cf4f5322440afc3ce7ff5ed8e5f7cd I0324 09:03:59.810845 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-rwq9d with fingerprint=d2b92668d5d87ea3ffa8bc4e0abab35ea454436891987d5f1c006f689a09fa0e I0324 09:03:59.810910 1 recorder.go:75] Recording config/pod/openshift-cluster-samples-operator/logs/cluster-samples-operator-8696bc5b94-h8vk9/cluster-samples-operator_current.log with fingerprint=51dfe4dbaaab22dd59607137528c9e45a8080f7d0bbaefea1a03684a88adcf29 I0324 09:03:59.810920 1 recorder.go:75] Recording config/pod/openshift-cluster-samples-operator/logs/cluster-samples-operator-8696bc5b94-h8vk9/cluster-samples-operator-watch_current.log with fingerprint=e9afa0567f113510cad106002b2db9d9a8c7c645da0b9214310e681250941721 I0324 09:03:59.810929 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 3.846600411s to process 17 records W0324 09:04:00.000190 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0324 09:04:00.998857 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0324 09:04:00.998881 1 tasks_processing.go:74] worker 0 stopped. E0324 09:04:00.998900 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0324 09:04:00.998920 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0324 09:04:00.998931 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.021067186s to process 1 records I0324 09:04:08.783370 1 tasks_processing.go:74] worker 2 stopped. I0324 09:04:08.783407 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0324 09:04:08.783418 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.821132733s to process 1 records I0324 09:04:09.586955 1 tasks_processing.go:74] worker 13 stopped. I0324 09:04:09.587325 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=4aff5860ec073ddfdf1bec96303c96ef7753a0d476f8315de0a2062bde3a2fd5 I0324 09:04:09.587343 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.605363847s to process 1 records E0324 09:04:09.587405 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.625s with: function \"active_alerts\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"tsdb_status\" failed with an error, function \"silenced_alerts\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0324 09:04:09.588518 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "active_alerts" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "tsdb_status" failed with an error, function "silenced_alerts" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0324 09:04:09.588534 1 periodic.go:209] Running workloads gatherer I0324 09:04:09.588555 1 tasks_processing.go:45] number of workers: 2 I0324 09:04:09.588569 1 tasks_processing.go:69] worker 1 listening for tasks. I0324 09:04:09.588575 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0324 09:04:09.588578 1 tasks_processing.go:69] worker 0 listening for tasks. I0324 09:04:09.588623 1 tasks_processing.go:71] worker 0 working on workload_info task. I0324 09:04:09.594424 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-34.ec2.internal... I0324 09:04:09.594432 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-191.ec2.internal... I0324 09:04:09.594442 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-92.ec2.internal... I0324 09:04:09.613194 1 tasks_processing.go:74] worker 1 stopped. I0324 09:04:09.613209 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 24.606142ms to process 0 records I0324 09:04:14.257638 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0324 09:04:14.286494 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0324 09:04:14.294184 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0324 09:04:14.302505 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (8ms) I0324 09:04:14.310590 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0324 09:04:14.318314 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (8ms) I0324 09:04:14.325904 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (8ms) I0324 09:04:14.333665 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (8ms) I0324 09:04:14.341787 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (8ms) I0324 09:04:14.349046 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (7ms) I0324 09:04:14.356323 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (7ms) I0324 09:04:14.363779 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (7ms) I0324 09:04:14.393368 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (30ms) I0324 09:04:14.493909 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (101ms) I0324 09:04:14.594214 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0324 09:04:14.693545 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (99ms) I0324 09:04:14.793988 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (100ms) I0324 09:04:14.894590 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (101ms) I0324 09:04:14.993713 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (99ms) I0324 09:04:15.095872 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (102ms) I0324 09:04:15.194065 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (98ms) I0324 09:04:15.293128 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (99ms) I0324 09:04:15.393428 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0324 09:04:15.494940 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (101ms) I0324 09:04:15.594653 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0324 09:04:15.694361 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0324 09:04:15.794990 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (101ms) I0324 09:04:15.894551 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0324 09:04:15.993327 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (99ms) I0324 09:04:16.093921 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (101ms) I0324 09:04:16.194373 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0324 09:04:16.294011 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0324 09:04:16.394558 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (101ms) I0324 09:04:16.494396 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (100ms) I0324 09:04:16.595668 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (101ms) I0324 09:04:16.695119 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (99ms) I0324 09:04:16.793681 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (99ms) I0324 09:04:16.894346 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (101ms) I0324 09:04:16.994479 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0324 09:04:17.094588 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0324 09:04:17.194877 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0324 09:04:17.293871 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (99ms) I0324 09:04:17.394525 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0324 09:04:17.494592 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0324 09:04:17.595814 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (101ms) I0324 09:04:17.693628 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (98ms) I0324 09:04:17.794149 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (101ms) I0324 09:04:17.893805 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0324 09:04:17.993819 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0324 09:04:18.094672 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (101ms) I0324 09:04:18.194257 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (100ms) I0324 09:04:18.293664 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (99ms) I0324 09:04:18.293696 1 tasks_processing.go:74] worker 0 stopped. I0324 09:04:18.294012 1 recorder.go:75] Recording config/workload_info with fingerprint=5f962d6e65ef7bf5f57f95f1d69392b14040edd731813442a3fab5a12ef037ae I0324 09:04:18.294028 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.705059137s to process 1 records I0324 09:04:18.294043 1 periodic.go:256] Periodic gather workloads completed in 8.705s I0324 09:04:18.294052 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0324 09:04:18.294056 1 periodic.go:209] Running conditional gatherer I0324 09:04:18.299439 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0324 09:04:18.463701 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0324 09:04:18.466084 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.144.37:9091: connect: connection refused I0324 09:04:18.466153 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0324 09:04:18.472019 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0324 09:04:18.472034 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472040 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472043 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472046 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472049 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472053 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472056 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472059 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472062 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0324 09:04:18.472064 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0324 09:04:18.472077 1 tasks_processing.go:45] number of workers: 3 I0324 09:04:18.472089 1 tasks_processing.go:69] worker 2 listening for tasks. I0324 09:04:18.472093 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0324 09:04:18.472102 1 tasks_processing.go:69] worker 0 listening for tasks. I0324 09:04:18.472116 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0324 09:04:18.472121 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0324 09:04:18.472120 1 tasks_processing.go:69] worker 1 listening for tasks. I0324 09:04:18.472129 1 tasks_processing.go:74] worker 1 stopped. I0324 09:04:18.472180 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0324 09:04:18.472194 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.074µs to process 1 records I0324 09:04:18.472234 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0324 09:04:18.472242 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 939ns to process 1 records I0324 09:04:18.472248 1 tasks_processing.go:74] worker 0 stopped. I0324 09:04:18.472349 1 tasks_processing.go:74] worker 2 stopped. I0324 09:04:18.472360 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 222.893µs to process 0 records I0324 09:04:18.472374 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0324 09:04:18.472392 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0324 09:04:18.502507 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=0ca2bd8eefb127224283979bed922406c51ac3e6e897cb8d6784000aa27c8ed6 I0324 09:04:18.502649 1 diskrecorder.go:70] Writing 126 records to /var/lib/insights-operator/insights-2026-03-24-090418.tar.gz I0324 09:04:18.510741 1 diskrecorder.go:51] Wrote 126 records to disk in 8ms I0324 09:04:18.510769 1 periodic.go:278] Gathering cluster info every 2h0m0s I0324 09:04:18.510783 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0324 09:05:25.962556 1 diskrecorder.go:223] Found files to send: insights-2026-03-24-090418.tar.gz I0324 09:05:25.962586 1 insightsuploader.go:150] Checking archives to upload periodically every 15m35.524160431s I0324 09:05:25.962610 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0324 09:05:25.971921 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0324 09:05:26.290125 1 requests.go:87] Successfully reported id=2026-03-24T09:05:25Z x-rh-insights-request-id=82275843060b49eeaee0c0210081b35c, wrote=64948 I0324 09:05:26.290244 1 insightsuploader.go:187] Uploaded report successfully in 327.641315ms I0324 09:05:26.290271 1 controller.go:128] Initializing last reported time to 2026-03-24T09:05:25Z I0324 09:05:26.290343 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0324 09:05:26.290353 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0324 09:05:26.290365 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0324 09:05:26.296068 1 controller.go:512] The operator is healthy I0324 09:05:55.975454 1 controller.go:512] The operator is healthy I0324 09:06:27.613548 1 insightsreport.go:137] Pulling report from smart-proxy I0324 09:06:27.613569 1 insightsreport.go:149] Retrieving report I0324 09:06:27.620440 1 requests.go:110] Retrieving report for cluster: 0aa043f0-2ade-443b-aeac-9af016f58260 I0324 09:06:27.620454 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/0aa043f0-2ade-443b-aeac-9af016f58260/reports I0324 09:06:27.623064 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/0aa043f0-2ade-443b-aeac-9af016f58260/reports I0324 09:06:27.767948 1 insightsreport.go:184] Report retrieved I0324 09:06:27.794968 1 insightsreport.go:239] Report retrieved correctly I0324 09:07:55.976030 1 controller.go:512] The operator is healthy I0324 09:08:55.955568 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0324 09:08:55.962487 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:08:55.962508 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0324 09:08:55.966549 1 secretconfigobserver.go:119] support secret does not exist I0324 09:09:55.975966 1 controller.go:512] The operator is healthy I0324 09:11:55.981427 1 controller.go:512] The operator is healthy I0324 09:13:55.952910 1 configmapobserver.go:84] configmaps "insights-config" not found I0324 09:13:55.958727 1 configmapobserver.go:84] configmaps "insights-config" not found I0324 09:13:55.966814 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0324 09:13:55.971350 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:13:55.971366 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0324 09:13:55.974696 1 controller.go:512] The operator is healthy I0324 09:13:55.975099 1 secretconfigobserver.go:119] support secret does not exist I0324 09:15:55.975946 1 controller.go:512] The operator is healthy I0324 09:17:55.984161 1 controller.go:512] The operator is healthy I0324 09:18:55.975504 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0324 09:18:55.981227 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:18:55.981244 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0324 09:18:55.985186 1 secretconfigobserver.go:119] support secret does not exist I0324 09:19:55.976117 1 controller.go:512] The operator is healthy I0324 09:21:06.815150 1 insightsuploader.go:146] Nothing to report since 2026-03-24T09:05:25Z I0324 09:21:55.976640 1 controller.go:512] The operator is healthy I0324 09:23:55.955166 1 configmapobserver.go:84] configmaps "insights-config" not found I0324 09:23:55.959411 1 configmapobserver.go:84] configmaps "insights-config" not found I0324 09:23:55.974815 1 controller.go:512] The operator is healthy I0324 09:23:55.985753 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0324 09:23:55.990244 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:23:55.990264 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0324 09:23:55.994210 1 secretconfigobserver.go:119] support secret does not exist I0324 09:25:55.975896 1 controller.go:512] The operator is healthy I0324 09:27:55.979031 1 controller.go:512] The operator is healthy I0324 09:28:55.994557 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0324 09:28:55.999738 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:28:55.999756 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0324 09:28:56.003458 1 secretconfigobserver.go:119] support secret does not exist I0324 09:29:55.975723 1 controller.go:512] The operator is healthy I0324 09:31:55.976176 1 controller.go:512] The operator is healthy I0324 09:33:55.955963 1 configmapobserver.go:84] configmaps "insights-config" not found I0324 09:33:55.975475 1 controller.go:512] The operator is healthy I0324 09:33:56.004466 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0324 09:33:56.009289 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0324 09:33:56.009309 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0324 09:33:56.013746 1 secretconfigobserver.go:119] support secret does not exist I0324 09:35:55.977020 1 controller.go:512] The operator is healthy I0324 09:36:18.569512 1 insightsuploader.go:146] Nothing to report since 2026-03-24T09:05:25Z