I0425 23:10:03.838046 1 cmd.go:253] Using service-serving-cert provided certificates I0425 23:10:03.838512 1 observer_polling.go:159] Starting file observer I0425 23:10:04.238135 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0425 23:10:04.238343 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0425 23:10:04.238861 1 secure_serving.go:57] Forcing use of http/1.1 only W0425 23:10:04.238883 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0425 23:10:04.238889 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0425 23:10:04.238892 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0425 23:10:04.238896 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0425 23:10:04.238898 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0425 23:10:04.238900 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0425 23:10:04.238990 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0425 23:10:04.242233 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0425 23:10:04.242256 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"40a21106-635b-4b81-bfa5-07e06978c8c2", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0425 23:10:04.243838 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0425 23:10:04.243845 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0425 23:10:04.243948 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0425 23:10:04.243915 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0425 23:10:04.244021 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0425 23:10:04.244136 1 secure_serving.go:213] Serving securely on [::]:8443 I0425 23:10:04.244139 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0425 23:10:04.244259 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0425 23:10:04.244485 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0425 23:10:04.248347 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0425 23:10:04.248371 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0425 23:10:04.248399 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0425 23:10:04.253522 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 23:10:04.253539 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0425 23:10:04.257359 1 secretconfigobserver.go:119] support secret does not exist I0425 23:10:04.261370 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 23:10:04.268226 1 secretconfigobserver.go:119] support secret does not exist I0425 23:10:04.272948 1 recorder.go:161] Pruning old reports every 6h41m55s, max age is 288h0m0s I0425 23:10:04.277126 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0425 23:10:04.277137 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0425 23:10:04.277141 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0425 23:10:04.277146 1 insightsreport.go:296] Starting report retriever I0425 23:10:04.277169 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0425 23:10:04.277179 1 periodic.go:209] Running clusterconfig gatherer I0425 23:10:04.277218 1 tasks_processing.go:45] number of workers: 64 I0425 23:10:04.277244 1 tasks_processing.go:69] worker 6 listening for tasks. I0425 23:10:04.277252 1 tasks_processing.go:69] worker 21 listening for tasks. I0425 23:10:04.277255 1 tasks_processing.go:71] worker 6 working on validating_webhook_configurations task. I0425 23:10:04.277258 1 tasks_processing.go:71] worker 21 working on openstack_version task. I0425 23:10:04.277258 1 tasks_processing.go:69] worker 1 listening for tasks. I0425 23:10:04.277268 1 tasks_processing.go:69] worker 2 listening for tasks. I0425 23:10:04.277274 1 tasks_processing.go:69] worker 5 listening for tasks. I0425 23:10:04.277278 1 tasks_processing.go:69] worker 4 listening for tasks. I0425 23:10:04.277277 1 tasks_processing.go:69] worker 42 listening for tasks. I0425 23:10:04.277288 1 tasks_processing.go:69] worker 3 listening for tasks. I0425 23:10:04.277290 1 tasks_processing.go:69] worker 8 listening for tasks. I0425 23:10:04.277280 1 tasks_processing.go:69] worker 14 listening for tasks. I0425 23:10:04.277285 1 tasks_processing.go:69] worker 7 listening for tasks. I0425 23:10:04.277296 1 tasks_processing.go:69] worker 9 listening for tasks. I0425 23:10:04.277298 1 tasks_processing.go:69] worker 22 listening for tasks. I0425 23:10:04.277301 1 tasks_processing.go:69] worker 10 listening for tasks. I0425 23:10:04.277302 1 tasks_processing.go:69] worker 18 listening for tasks. I0425 23:10:04.277307 1 tasks_processing.go:69] worker 11 listening for tasks. I0425 23:10:04.277307 1 tasks_processing.go:69] worker 15 listening for tasks. I0425 23:10:04.277309 1 tasks_processing.go:69] worker 23 listening for tasks. I0425 23:10:04.277313 1 tasks_processing.go:69] worker 12 listening for tasks. I0425 23:10:04.277315 1 tasks_processing.go:69] worker 17 listening for tasks. I0425 23:10:04.277316 1 tasks_processing.go:69] worker 24 listening for tasks. I0425 23:10:04.277318 1 tasks_processing.go:69] worker 13 listening for tasks. I0425 23:10:04.277321 1 tasks_processing.go:69] worker 43 listening for tasks. I0425 23:10:04.277310 1 tasks_processing.go:69] worker 16 listening for tasks. I0425 23:10:04.277327 1 tasks_processing.go:69] worker 44 listening for tasks. I0425 23:10:04.277325 1 tasks_processing.go:69] worker 45 listening for tasks. I0425 23:10:04.277332 1 tasks_processing.go:69] worker 54 listening for tasks. I0425 23:10:04.277333 1 tasks_processing.go:69] worker 53 listening for tasks. I0425 23:10:04.277340 1 tasks_processing.go:69] worker 55 listening for tasks. I0425 23:10:04.277342 1 tasks_processing.go:69] worker 57 listening for tasks. I0425 23:10:04.277317 1 tasks_processing.go:69] worker 47 listening for tasks. I0425 23:10:04.277343 1 tasks_processing.go:69] worker 27 listening for tasks. I0425 23:10:04.277325 1 tasks_processing.go:69] worker 58 listening for tasks. I0425 23:10:04.277349 1 tasks_processing.go:69] worker 52 listening for tasks. I0425 23:10:04.277350 1 tasks_processing.go:69] worker 48 listening for tasks. I0425 23:10:04.277352 1 tasks_processing.go:69] worker 50 listening for tasks. I0425 23:10:04.277360 1 tasks_processing.go:69] worker 29 listening for tasks. I0425 23:10:04.277360 1 tasks_processing.go:69] worker 40 listening for tasks. I0425 23:10:04.277357 1 tasks_processing.go:69] worker 51 listening for tasks. I0425 23:10:04.277361 1 tasks_processing.go:69] worker 36 listening for tasks. I0425 23:10:04.277368 1 tasks_processing.go:69] worker 37 listening for tasks. I0425 23:10:04.277370 1 tasks_processing.go:69] worker 31 listening for tasks. I0425 23:10:04.277368 1 tasks_processing.go:69] worker 19 listening for tasks. I0425 23:10:04.277371 1 tasks_processing.go:69] worker 32 listening for tasks. I0425 23:10:04.277376 1 tasks_processing.go:69] worker 62 listening for tasks. I0425 23:10:04.277378 1 tasks_processing.go:69] worker 63 listening for tasks. I0425 23:10:04.277383 1 tasks_processing.go:69] worker 60 listening for tasks. I0425 23:10:04.277379 1 tasks_processing.go:69] worker 59 listening for tasks. I0425 23:10:04.277326 1 tasks_processing.go:69] worker 25 listening for tasks. I0425 23:10:04.277334 1 tasks_processing.go:69] worker 56 listening for tasks. I0425 23:10:04.277332 1 tasks_processing.go:69] worker 26 listening for tasks. I0425 23:10:04.277391 1 tasks_processing.go:69] worker 33 listening for tasks. I0425 23:10:04.277330 1 tasks_processing.go:69] worker 46 listening for tasks. I0425 23:10:04.277338 1 tasks_processing.go:69] worker 34 listening for tasks. I0425 23:10:04.277352 1 tasks_processing.go:69] worker 35 listening for tasks. I0425 23:10:04.277355 1 tasks_processing.go:69] worker 28 listening for tasks. I0425 23:10:04.277355 1 tasks_processing.go:69] worker 38 listening for tasks. I0425 23:10:04.277360 1 tasks_processing.go:69] worker 39 listening for tasks. I0425 23:10:04.277362 1 tasks_processing.go:69] worker 20 listening for tasks. I0425 23:10:04.277254 1 tasks_processing.go:69] worker 0 listening for tasks. I0425 23:10:04.277366 1 tasks_processing.go:69] worker 30 listening for tasks. I0425 23:10:04.277372 1 tasks_processing.go:69] worker 61 listening for tasks. I0425 23:10:04.277368 1 tasks_processing.go:69] worker 41 listening for tasks. I0425 23:10:04.277373 1 tasks_processing.go:69] worker 49 listening for tasks. I0425 23:10:04.277449 1 tasks_processing.go:71] worker 2 working on active_alerts task. I0425 23:10:04.277454 1 tasks_processing.go:71] worker 15 working on machines task. I0425 23:10:04.277456 1 tasks_processing.go:71] worker 1 working on storage_classes task. I0425 23:10:04.277465 1 tasks_processing.go:71] worker 23 working on storage_cluster task. I0425 23:10:04.277460 1 tasks_processing.go:71] worker 49 working on clusterroles task. I0425 23:10:04.277513 1 tasks_processing.go:71] worker 4 working on image task. I0425 23:10:04.277523 1 tasks_processing.go:71] worker 37 working on pod_network_connectivity_checks task. I0425 23:10:04.277531 1 tasks_processing.go:71] worker 31 working on authentication task. I0425 23:10:04.277538 1 tasks_processing.go:71] worker 9 working on mutating_webhook_configurations task. I0425 23:10:04.277544 1 tasks_processing.go:71] worker 55 working on ingress_certificates task. I0425 23:10:04.277501 1 tasks_processing.go:71] worker 14 working on openshift_machine_api_events task. I0425 23:10:04.277556 1 tasks_processing.go:71] worker 42 working on ceph_cluster task. I0425 23:10:04.277562 1 tasks_processing.go:71] worker 10 working on node_logs task. I0425 23:10:04.277566 1 tasks_processing.go:71] worker 32 working on monitoring_persistent_volumes task. I0425 23:10:04.277576 1 tasks_processing.go:71] worker 48 working on image_registries task. I0425 23:10:04.277596 1 tasks_processing.go:71] worker 22 working on sap_config task. I0425 23:10:04.277612 1 tasks_processing.go:71] worker 8 working on sap_datahubs task. I0425 23:10:04.277614 1 tasks_processing.go:71] worker 27 working on proxies task. I0425 23:10:04.277635 1 tasks_processing.go:71] worker 20 working on aggregated_monitoring_cr_names task. I0425 23:10:04.277705 1 tasks_processing.go:71] worker 58 working on metrics task. I0425 23:10:04.277561 1 tasks_processing.go:71] worker 19 working on nodes task. I0425 23:10:04.277731 1 tasks_processing.go:71] worker 62 working on silenced_alerts task. I0425 23:10:04.277741 1 tasks_processing.go:71] worker 18 working on machine_autoscalers task. I0425 23:10:04.277750 1 tasks_processing.go:71] worker 63 working on olm_operators task. I0425 23:10:04.277751 1 tasks_processing.go:71] worker 52 working on config_maps task. I0425 23:10:04.277509 1 tasks_processing.go:71] worker 5 working on nodenetworkstates task. I0425 23:10:04.277608 1 tasks_processing.go:71] worker 3 working on operators task. I0425 23:10:04.277765 1 tasks_processing.go:71] worker 12 working on crds task. I0425 23:10:04.277769 1 tasks_processing.go:71] worker 30 working on openstack_dataplanenodesets task. I0425 23:10:04.277796 1 tasks_processing.go:71] worker 40 working on nodenetworkconfigurationpolicies task. I0425 23:10:04.277803 1 tasks_processing.go:71] worker 51 working on service_accounts task. I0425 23:10:04.277808 1 tasks_processing.go:71] worker 34 working on openstack_dataplanedeployments task. I0425 23:10:04.277804 1 tasks_processing.go:71] worker 41 working on cost_management_metrics_configs task. I0425 23:10:04.277811 1 tasks_processing.go:71] worker 28 working on support_secret task. I0425 23:10:04.277813 1 tasks_processing.go:71] worker 0 working on ingress task. I0425 23:10:04.277819 1 tasks_processing.go:71] worker 59 working on schedulers task. I0425 23:10:04.277822 1 tasks_processing.go:71] worker 29 working on image_pruners task. I0425 23:10:04.277826 1 tasks_processing.go:71] worker 26 working on container_runtime_configs task. I0425 23:10:04.277836 1 tasks_processing.go:71] worker 38 working on oauths task. I0425 23:10:04.277552 1 tasks_processing.go:71] worker 60 working on machine_healthchecks task. I0425 23:10:04.277848 1 tasks_processing.go:71] worker 36 working on machine_configs task. I0425 23:10:04.277856 1 tasks_processing.go:71] worker 16 working on qemu_kubevirt_launcher_logs task. I0425 23:10:04.277526 1 tasks_processing.go:71] worker 33 working on version task. I0425 23:10:04.277856 1 tasks_processing.go:71] worker 11 working on feature_gates task. I0425 23:10:04.277862 1 tasks_processing.go:71] worker 45 working on install_plans task. I0425 23:10:04.277865 1 tasks_processing.go:71] worker 44 working on machine_config_pools task. I0425 23:10:04.277868 1 tasks_processing.go:71] worker 57 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0425 23:10:04.278640 1 tasks_processing.go:71] worker 47 working on container_images task. I0425 23:10:04.278696 1 tasks_processing.go:71] worker 54 working on machine_sets task. I0425 23:10:04.278799 1 tasks_processing.go:71] worker 43 working on overlapping_namespace_uids task. I0425 23:10:04.278903 1 tasks_processing.go:71] worker 53 working on dvo_metrics task. I0425 23:10:04.278664 1 tasks_processing.go:71] worker 24 working on tsdb_status task. I0425 23:10:04.277801 1 tasks_processing.go:71] worker 35 working on openstack_controlplanes task. I0425 23:10:04.277534 1 tasks_processing.go:71] worker 7 working on cluster_apiserver task. I0425 23:10:04.277806 1 tasks_processing.go:71] worker 46 working on openshift_logging task. I0425 23:10:04.278807 1 tasks_processing.go:71] worker 13 working on certificate_signing_requests task. I0425 23:10:04.277800 1 tasks_processing.go:71] worker 61 working on sap_pods task. I0425 23:10:04.277819 1 tasks_processing.go:71] worker 50 working on operators_pods_and_events task. I0425 23:10:04.277822 1 tasks_processing.go:71] worker 56 working on pdbs task. I0425 23:10:04.278679 1 tasks_processing.go:71] worker 17 working on jaegers task. I0425 23:10:04.277816 1 tasks_processing.go:71] worker 25 working on infrastructures task. I0425 23:10:04.281431 1 tasks_processing.go:71] worker 21 working on lokistack task. I0425 23:10:04.281470 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 4.164043ms to process 0 records I0425 23:10:04.282189 1 tasks_processing.go:74] worker 37 stopped. E0425 23:10:04.282207 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0425 23:10:04.282222 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 4.650831ms to process 0 records I0425 23:10:04.282635 1 tasks_processing.go:71] worker 39 working on networks task. I0425 23:10:04.283068 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0425 23:10:04.283090 1 controller.go:326] The initial operator extension status is healthy I0425 23:10:04.283100 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0425 23:10:04.283110 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0425 23:10:04.283114 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0425 23:10:04.283133 1 controller.go:489] The operator is still being initialized I0425 23:10:04.283144 1 controller.go:512] The operator is healthy I0425 23:10:04.283243 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0425 23:10:04.283292 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0425 23:10:04.283388 1 operator.go:288] started I0425 23:10:04.283416 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0425 23:10:04.284554 1 tasks_processing.go:74] worker 8 stopped. I0425 23:10:04.284569 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 6.932894ms to process 0 records I0425 23:10:04.284579 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 6.811938ms to process 0 records I0425 23:10:04.284584 1 tasks_processing.go:74] worker 18 stopped. I0425 23:10:04.284644 1 tasks_processing.go:74] worker 35 stopped. I0425 23:10:04.284659 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 5.640212ms to process 0 records E0425 23:10:04.284667 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0425 23:10:04.284673 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 6.808159ms to process 0 records I0425 23:10:04.284680 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 7.058339ms to process 0 records I0425 23:10:04.284685 1 tasks_processing.go:74] worker 22 stopped. I0425 23:10:04.284687 1 tasks_processing.go:74] worker 60 stopped. E0425 23:10:04.288226 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:40799->172.30.0.10:53: read: connection refused I0425 23:10:04.288248 1 tasks_processing.go:74] worker 24 stopped. E0425 23:10:04.288258 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:40799->172.30.0.10:53: read: connection refused I0425 23:10:04.288272 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 9.283451ms to process 0 records E0425 23:10:04.288305 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:40799->172.30.0.10:53: read: connection refused I0425 23:10:04.288328 1 tasks_processing.go:74] worker 58 stopped. E0425 23:10:04.288336 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:40799->172.30.0.10:53: read: connection refused I0425 23:10:04.288349 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 10.605481ms to process 0 records E0425 23:10:04.288458 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:43281->172.30.0.10:53: read: connection refused I0425 23:10:04.288470 1 tasks_processing.go:74] worker 2 stopped. E0425 23:10:04.288476 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:43281->172.30.0.10:53: read: connection refused I0425 23:10:04.288484 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 11.012583ms to process 0 records E0425 23:10:04.288512 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:43281->172.30.0.10:53: read: connection refused I0425 23:10:04.288527 1 tasks_processing.go:74] worker 62 stopped. E0425 23:10:04.288540 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:43281->172.30.0.10:53: read: connection refused I0425 23:10:04.288553 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 10.781038ms to process 0 records I0425 23:10:04.289134 1 tasks_processing.go:74] worker 6 stopped. I0425 23:10:04.289349 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=af55ea152d27c0ed699a3591e58208dd3eb7ec7977c612e67d432f15a695b053 I0425 23:10:04.289451 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=98025a127c689edd5d0b1d71b62823335c033b47cabf5fb0b3245667a09e2646 I0425 23:10:04.289576 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=28c9a6586d5a2dacdfa023fcdf1fd0e730de86372c3bd1432aaa098252cf5a33 I0425 23:10:04.289606 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=b6d0b2ec130ad4a0abe369346aa993c2003d38cb4d6017946a5c57524c0cea86 I0425 23:10:04.289629 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=9886b11629d63a44e9c6be27ff24f9b87b297b6d2f1e90bcd556952f8ada9a8d I0425 23:10:04.289660 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=76d3d79acda8d809c11eaca98fb1b43f996da8330b92fba62158ea580ff9d36c I0425 23:10:04.289684 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=05dda391a71956c37e57e63fbdac309943765d3873cf7ea54fec17eb728f95d2 I0425 23:10:04.289708 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=4d887d3a8aad575e4192a162bb22ac02383cc8d6854117b7532529fa83b7b92e I0425 23:10:04.289737 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=a638c09283cb26d26f936cc12e88c318ab88887aa501742c8b31a17d814edae3 I0425 23:10:04.289773 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=92fd883d66d18c0e92b806158dada87989bdcc97404b0eadeca668420e32c997 I0425 23:10:04.289801 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=a7bb6fec5d49e2bcaa68d8aaffc9bb57e6e5c790b6169899e80ca4f1188ff047 I0425 23:10:04.289825 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=17b47cb7836da8744e3ff1aee45fe565d8f27f57e2ec0edaa94a45415adfca4b I0425 23:10:04.289854 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=9f9c822014733498d48fe1079f2bfb0c7ae765cd1b88fe2402e9091715392609 I0425 23:10:04.289864 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 11.86553ms to process 13 records I0425 23:10:04.291786 1 tasks_processing.go:74] worker 41 stopped. I0425 23:10:04.291800 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 13.966275ms to process 0 records I0425 23:10:04.291814 1 tasks_processing.go:74] worker 54 stopped. I0425 23:10:04.291823 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 13.059253ms to process 0 records I0425 23:10:04.291840 1 tasks_processing.go:74] worker 17 stopped. I0425 23:10:04.291848 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 11.866051ms to process 0 records I0425 23:10:04.291915 1 tasks_processing.go:74] worker 27 stopped. I0425 23:10:04.291986 1 recorder.go:75] Recording config/proxy with fingerprint=86a400628c6e1dfd97fdce9e10356bc40436539f0b76e94e6f541e24e521d924 I0425 23:10:04.292003 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 14.285114ms to process 1 records I0425 23:10:04.292010 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 12.709486ms to process 0 records I0425 23:10:04.292015 1 tasks_processing.go:74] worker 46 stopped. I0425 23:10:04.292059 1 tasks_processing.go:74] worker 42 stopped. I0425 23:10:04.292078 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 14.490798ms to process 0 records I0425 23:10:04.292169 1 tasks_processing.go:74] worker 38 stopped. I0425 23:10:04.292464 1 recorder.go:75] Recording config/oauth with fingerprint=1b4e7582f20927196651cd276d2a8d091e2fcb0f04dcb7fdb3569cb59638ed30 I0425 23:10:04.292513 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 14.249218ms to process 1 records I0425 23:10:04.292606 1 tasks_processing.go:74] worker 11 stopped. I0425 23:10:04.292673 1 recorder.go:75] Recording config/featuregate with fingerprint=2473729a618db1e166dd449661b76a5d7ec7d375fcacad0c22484c4f2737c57f I0425 23:10:04.292686 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 14.351171ms to process 1 records I0425 23:10:04.294646 1 tasks_processing.go:74] worker 30 stopped. I0425 23:10:04.294656 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 16.868537ms to process 0 records I0425 23:10:04.294679 1 tasks_processing.go:74] worker 15 stopped. E0425 23:10:04.294689 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0425 23:10:04.294696 1 gather.go:177] gatherer "clusterconfig" function "machines" took 17.216332ms to process 0 records I0425 23:10:04.303861 1 tasks_processing.go:74] worker 61 stopped. I0425 23:10:04.303875 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 24.358396ms to process 0 records I0425 23:10:04.303881 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 26.096313ms to process 0 records I0425 23:10:04.303886 1 tasks_processing.go:74] worker 5 stopped. I0425 23:10:04.304040 1 tasks_processing.go:74] worker 26 stopped. I0425 23:10:04.304047 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 26.205582ms to process 0 records I0425 23:10:04.304058 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 26.225544ms to process 0 records I0425 23:10:04.304064 1 tasks_processing.go:74] worker 34 stopped. I0425 23:10:04.304136 1 tasks_processing.go:74] worker 32 stopped. I0425 23:10:04.304145 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 26.565308ms to process 0 records I0425 23:10:04.304273 1 tasks_processing.go:74] worker 40 stopped. I0425 23:10:04.304282 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 26.464444ms to process 0 records I0425 23:10:04.304439 1 tasks_processing.go:74] worker 28 stopped. E0425 23:10:04.304449 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0425 23:10:04.304457 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 26.616805ms to process 0 records I0425 23:10:04.304497 1 tasks_processing.go:74] worker 7 stopped. I0425 23:10:04.304629 1 recorder.go:75] Recording config/apiserver with fingerprint=0d31c6b00f4026e9c20a2538b4541d246d0d0f13ea52ac02e2359f641ad220b4 I0425 23:10:04.304641 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 25.424313ms to process 1 records I0425 23:10:04.304828 1 tasks_processing.go:74] worker 29 stopped. I0425 23:10:04.305013 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=8cead6bd51c8a0a05ad0af58cb0be9d7300319a2a751779a17c7de134f43d2b4 I0425 23:10:04.305034 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 26.6981ms to process 1 records I0425 23:10:04.305041 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 27.0043ms to process 0 records I0425 23:10:04.305048 1 tasks_processing.go:74] worker 14 stopped. I0425 23:10:04.305078 1 tasks_processing.go:74] worker 21 stopped. I0425 23:10:04.305093 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 23.628543ms to process 0 records I0425 23:10:04.305132 1 tasks_processing.go:74] worker 25 stopped. I0425 23:10:04.306580 1 recorder.go:75] Recording config/infrastructure with fingerprint=0a90e862207207d21afbffa5d20fe06753af65e4abc6cc437095d8c33534d938 I0425 23:10:04.306598 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 24.77521ms to process 1 records I0425 23:10:04.306678 1 tasks_processing.go:74] worker 31 stopped. I0425 23:10:04.306751 1 recorder.go:75] Recording config/authentication with fingerprint=ecfdf7b53c0e47cba4b8e5b7837379ba9b41c29bff65a5c9686e2752ff0595f6 I0425 23:10:04.306761 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 27.590212ms to process 1 records I0425 23:10:04.306825 1 recorder.go:75] Recording config/image with fingerprint=d84dc6823bd1568f9ed6def76c7f881bd17d5d83afc31fcff4b13d3da5962c2d I0425 23:10:04.306835 1 gather.go:177] gatherer "clusterconfig" function "image" took 27.925333ms to process 1 records I0425 23:10:04.306847 1 tasks_processing.go:74] worker 4 stopped. I0425 23:10:04.306916 1 tasks_processing.go:74] worker 0 stopped. I0425 23:10:04.306947 1 recorder.go:75] Recording config/ingress with fingerprint=1b686c97d9fb4dca520b032315b3c057529ba1e163d450929a518a0098ec179b I0425 23:10:04.306959 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 27.737454ms to process 1 records I0425 23:10:04.311287 1 tasks_processing.go:74] worker 9 stopped. I0425 23:10:04.311373 1 gather_logs.go:145] no pods in namespace were found I0425 23:10:04.311397 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=bf5e09846669e56ec9455927dca32c754e799802a647899619dae47af7c397e3 I0425 23:10:04.311423 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=a3ca5a82596d7b602d281e8773ce059fa3c859a023349574f9dd332a1d3fce39 I0425 23:10:04.311448 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=e6388131811f38e57e82ec0a7850e17af1e3666426fb911b1e1021ee0066576b I0425 23:10:04.311456 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 33.741754ms to process 3 records I0425 23:10:04.311462 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 33.877156ms to process 0 records I0425 23:10:04.311466 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 33.524674ms to process 0 records I0425 23:10:04.311479 1 tasks_processing.go:74] worker 23 stopped. I0425 23:10:04.311482 1 tasks_processing.go:74] worker 16 stopped. I0425 23:10:04.311536 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=9f4ef39e7e7d50182bf8dc612cab3f41680c00d2b9fe3c95afbdeb95a012bcd3 I0425 23:10:04.311554 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=3f53f3e22c0fb9b1bbe0618e7a6926ecd391bdc769f899d945378d72fc641f1d I0425 23:10:04.311564 1 tasks_processing.go:74] worker 56 stopped. I0425 23:10:04.311569 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=c1463f0653342edc7bca8f3bd6a452e85c4d9947b030cca34f3f51e967c30340 I0425 23:10:04.311592 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=9c5a809a302f641210d59443b6d70af4a579206dd637c0f084b280666f33640d I0425 23:10:04.311616 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=abac0ad1cd78b02d64d885516064c36d3f24fd47d65255c323cfbce06d627107 I0425 23:10:04.311631 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=2aa88c9b230fdae56f68e3a153c4e3f8c041e5311bd3ee9826192e34d1152a6d I0425 23:10:04.311644 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=038e0e9727b67a8139ed6123305da6bf853efd003365e5cb49433868166069b9 I0425 23:10:04.311663 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=18c239efee5128bdb2f80dba751d8dbbf3457dc037b747404db9f3bd8222404d I0425 23:10:04.311681 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=97bf89d05773a3e030ccc072b854eb1629fa976f7a892ef35f00fae479ad49a4 I0425 23:10:04.311697 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=b613a0d517c3115133102b93faead2af1b17356c8024228528edb32c513cbf0b I0425 23:10:04.311712 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=5b6833c85dd1be96cd85b841be6fdb8a2d6a4d8963822e8b2f0162c5554ce92a I0425 23:10:04.311719 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 31.490709ms to process 11 records I0425 23:10:04.311915 1 tasks_processing.go:74] worker 10 stopped. I0425 23:10:04.311930 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 34.340764ms to process 0 records I0425 23:10:04.312030 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=b5e0be1063a88fc6793324049c54bc997f208ce3a4ab632b7f827f7ea5049db5 I0425 23:10:04.312055 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=f6f9a2eb46699866d82b3db4fc73a7db428c1a2dff7e275b26c4a12e009de50e I0425 23:10:04.312064 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 34.448615ms to process 2 records I0425 23:10:04.312073 1 tasks_processing.go:74] worker 1 stopped. I0425 23:10:04.312095 1 tasks_processing.go:74] worker 48 stopped. I0425 23:10:04.312454 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=22958085614abd66281531c08be218d1c4b31e3ba35696028baee5ddd084b209 I0425 23:10:04.312469 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 34.496418ms to process 1 records I0425 23:10:04.312654 1 tasks_processing.go:74] worker 12 stopped. I0425 23:10:04.313286 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=872a3791f1a44a692941c1732860e19c71171692fe4682bcb335f4aef6cf07ff I0425 23:10:04.314059 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=c29d2a8bc50aa084f6cc6b52c0a8c701df97da3bd3b6fba8a634022ccc754430 I0425 23:10:04.314177 1 gather.go:177] gatherer "clusterconfig" function "crds" took 34.680481ms to process 2 records I0425 23:10:04.314221 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 34.711731ms to process 0 records I0425 23:10:04.314305 1 tasks_processing.go:74] worker 44 stopped. I0425 23:10:04.314418 1 tasks_processing.go:74] worker 39 stopped. I0425 23:10:04.314520 1 recorder.go:75] Recording config/network with fingerprint=5f0a945742efa3502771e6150c03f6ad89a648538aa99864bc2979f15bb47664 I0425 23:10:04.314533 1 gather.go:177] gatherer "clusterconfig" function "networks" took 30.142064ms to process 1 records I0425 23:10:04.314578 1 tasks_processing.go:74] worker 59 stopped. I0425 23:10:04.314614 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=b0d51d0f34644d9d7d2d462d84e7b4bf2c148c27a174ce951e3d0f52ace7776b I0425 23:10:04.314635 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 35.823688ms to process 1 records I0425 23:10:04.316050 1 tasks_processing.go:74] worker 19 stopped. I0425 23:10:04.316369 1 recorder.go:75] Recording config/node/ip-10-0-1-89.ec2.internal with fingerprint=b6acfd4d09e82c83731791caf60fce639abe2448a62f5b5b3152250e2885895a I0425 23:10:04.316436 1 recorder.go:75] Recording config/node/ip-10-0-2-210.ec2.internal with fingerprint=3509c3bf3a0b056c095b67ae00838e951eeec4d4ac8f655c0e774bada370b4b8 I0425 23:10:04.316447 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 38.30918ms to process 2 records I0425 23:10:04.316602 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0425 23:10:04.316621 1 controller.go:212] Source scaController *sca.Controller is not ready I0425 23:10:04.316627 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0425 23:10:04.316632 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0425 23:10:04.316636 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0425 23:10:04.316658 1 controller.go:489] The operator is still being initialized I0425 23:10:04.316667 1 controller.go:512] The operator is healthy I0425 23:10:04.316862 1 tasks_processing.go:74] worker 43 stopped. I0425 23:10:04.316893 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0425 23:10:04.316905 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 37.982388ms to process 1 records I0425 23:10:04.317060 1 tasks_processing.go:74] worker 13 stopped. I0425 23:10:04.317076 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 37.587886ms to process 0 records W0425 23:10:04.317783 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0425 23:10:04.318987 1 tasks_processing.go:74] worker 20 stopped. I0425 23:10:04.319009 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 41.339288ms to process 0 records I0425 23:10:04.322015 1 tasks_processing.go:74] worker 49 stopped. I0425 23:10:04.322204 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=a0be0374b5dd3d5fabedc2c45ede6b1d66b4fd9aa8417b7325b2209306367585 I0425 23:10:04.322289 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=3f98e05a980dc65b806d44dc571af64129f73f0f1f3503e031f03161852d075b I0425 23:10:04.322297 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 44.48952ms to process 2 records E0425 23:10:04.322595 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27fed508f2-a6c4-4ff5-b23a-0aa59d7606cf%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.17:45278->172.30.0.10:53: read: connection refused I0425 23:10:04.322606 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27fed508f2-a6c4-4ff5-b23a-0aa59d7606cf%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.17:45278->172.30.0.10:53: read: connection refused I0425 23:10:04.328618 1 tasks_processing.go:74] worker 63 stopped. I0425 23:10:04.328686 1 recorder.go:75] Recording config/olm_operators with fingerprint=78be2f8666b4f6be8a833cc026e5f07a191f57ebff6636b2b2985ee008b0240f I0425 23:10:04.328703 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 50.853994ms to process 1 records I0425 23:10:04.334110 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0425 23:10:04.334257 1 tasks_processing.go:74] worker 47 stopped. I0425 23:10:04.335624 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-5g2rj with fingerprint=c6fd71f01ce6e3abdc37e34089e2dc21d3002e0414eb10ee823a72f1c23e2016 I0425 23:10:04.335702 1 recorder.go:75] Recording config/running_containers with fingerprint=0107692249e80e5906b6ba0d68aefc78a9a487eab314d0cf2f07bc3908b42c65 I0425 23:10:04.335712 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 55.551788ms to process 2 records I0425 23:10:04.336930 1 tasks_processing.go:74] worker 52 stopped. E0425 23:10:04.336949 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0425 23:10:04.336960 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0425 23:10:04.336964 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0425 23:10:04.336974 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0425 23:10:04.337001 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0425 23:10:04.337008 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0425 23:10:04.337012 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0425 23:10:04.337020 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0425 23:10:04.337058 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0425 23:10:04.337080 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0425 23:10:04.337085 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 59.160882ms to process 7 records W0425 23:10:04.337622 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.17:56038->172.30.0.10:53: read: connection refused I0425 23:10:04.337637 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.17:56038->172.30.0.10:53: read: connection refused I0425 23:10:04.338783 1 tasks_processing.go:74] worker 57 stopped. I0425 23:10:04.338799 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 60.241046ms to process 0 records I0425 23:10:04.342034 1 tasks_processing.go:74] worker 55 stopped. E0425 23:10:04.342048 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0425 23:10:04.342055 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ptfd094rc7r1k8tmu19s34b5k055b6m-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ptfd094rc7r1k8tmu19s34b5k055b6m-primary-cert-bundle-secret" not found I0425 23:10:04.342117 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=e1b840d14f3d3763ba48eefadc48dfdb664a6be1062d11bbb7f2655f108ac9b1 I0425 23:10:04.342130 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 64.474233ms to process 1 records I0425 23:10:04.344073 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0425 23:10:04.344737 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0425 23:10:04.344747 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0425 23:10:04.348865 1 base_controller.go:82] Caches are synced for ConfigController I0425 23:10:04.348879 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0425 23:10:04.352409 1 configmapobserver.go:84] configmaps "insights-config" not found I0425 23:10:04.383470 1 base_controller.go:82] Caches are synced for LoggingSyncer I0425 23:10:04.383485 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0425 23:10:04.396079 1 tasks_processing.go:74] worker 33 stopped. I0425 23:10:04.396613 1 recorder.go:75] Recording config/version with fingerprint=7c407ac9e0fab747bcd4b74ccf4024ae5f6d202e4de4d0d8903ea4920ad68cfd I0425 23:10:04.396658 1 recorder.go:75] Recording config/id with fingerprint=cc548eaa0c257376c045f28ac60776f3869c229851c2e3d576947acbb44aaa1a I0425 23:10:04.396676 1 gather.go:177] gatherer "clusterconfig" function "version" took 118.212922ms to process 2 records I0425 23:10:04.408016 1 tasks_processing.go:74] worker 36 stopped. I0425 23:10:04.408041 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0425 23:10:04.408050 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 130.150613ms to process 1 records I0425 23:10:04.800218 1 gather_cluster_operator_pods_and_events.go:121] Found 14 pods with 16 containers I0425 23:10:04.800233 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1572864 bytes I0425 23:10:04.801031 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-9j9g5 pod in namespace openshift-dns (previous: false). I0425 23:10:05.006183 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-9j9g5 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-9j9g5\" is waiting to start: ContainerCreating" I0425 23:10:05.006202 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-9j9g5\" is waiting to start: ContainerCreating" I0425 23:10:05.006210 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-9j9g5 pod in namespace openshift-dns (previous: false). I0425 23:10:05.206403 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-9j9g5 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-9j9g5\" is waiting to start: ContainerCreating" I0425 23:10:05.206425 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-9j9g5\" is waiting to start: ContainerCreating" I0425 23:10:05.206457 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-hr96k pod in namespace openshift-dns (previous: false). W0425 23:10:05.316978 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0425 23:10:05.407848 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-hr96k pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-hr96k\" is waiting to start: ContainerCreating" I0425 23:10:05.407870 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-hr96k\" is waiting to start: ContainerCreating" I0425 23:10:05.407878 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-hr96k pod in namespace openshift-dns (previous: false). I0425 23:10:05.605964 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-hr96k pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-hr96k\" is waiting to start: ContainerCreating" I0425 23:10:05.605985 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-hr96k\" is waiting to start: ContainerCreating" I0425 23:10:05.606002 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-mngwq pod in namespace openshift-dns (previous: false). I0425 23:10:05.806294 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0425 23:10:05.806311 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-npztr pod in namespace openshift-dns (previous: false). I0425 23:10:06.008903 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0425 23:10:06.008922 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6485787969-lrmgf pod in namespace openshift-image-registry (previous: false). I0425 23:10:06.140432 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0425 23:10:06.200342 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0425 23:10:06.200359 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6485787969-spkvj pod in namespace openshift-image-registry (previous: false). W0425 23:10:06.317019 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0425 23:10:06.405914 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-777f7d7b64-zmjdc pod in namespace openshift-image-registry (previous: false). I0425 23:10:06.606477 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-777f7d7b64-zmjdc pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-777f7d7b64-zmjdc\" is waiting to start: ContainerCreating" I0425 23:10:06.606493 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-777f7d7b64-zmjdc\" is waiting to start: ContainerCreating" I0425 23:10:06.606507 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9qqh2 pod in namespace openshift-image-registry (previous: false). I0425 23:10:06.742331 1 tasks_processing.go:74] worker 3 stopped. I0425 23:10:06.742463 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=b43b94872a5aea3acc689da711ab026b7a669c00da03dbcfa8a309567564d832 I0425 23:10:06.742525 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0425 23:10:06.742560 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=e25068ac2123850678cc459b7f1e5d2d5376a0246b9a0e21cd1b511b130fa2a2 I0425 23:10:06.742577 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0425 23:10:06.742612 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=381f3806c350360563f0a3b1ebe0c96dc3aca107a340a0013ce236a3a75ecf68 I0425 23:10:06.742642 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0425 23:10:06.742679 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=6b4026b5aa80019011b28a88243d47f20ca0bdf100b474c4093b35b0f7663973 I0425 23:10:06.742719 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=bdc05a2d1d85753b96fccc5d4e3e3084f6be708d15e5b9042c9214c656bbec82 I0425 23:10:06.742754 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=330f8369e6b49656f1074f9ac357f5b825bc2c5b7ec4ddf52d9f2f4c9c7218b3 I0425 23:10:06.742795 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=7a268bc3a0f959c5efd2957415db1dce40ccb5bdbdbc2508a119e3db7ebcc6a0 I0425 23:10:06.743058 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e5108f32043d8381ab739405709a248b8718d50f184de385971aa6970ec0a084 I0425 23:10:06.743090 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=c3606eeb15225dc95c44273feb06e90e66d3c5b6e4c9e8c24c5e872d79513f29 I0425 23:10:06.743104 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0425 23:10:06.743134 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=403333e38d75e560b0afdfe0726088c296ddb2138539bc6d2b00cc5a2ee78139 I0425 23:10:06.743168 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0425 23:10:06.743196 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=d0922ab52f9ae6a4ec08ac9bc0bceaa1de29070a77e3564f9fa2273addddbb23 I0425 23:10:06.743211 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0425 23:10:06.743240 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=9baef41a5b232115e2c6ae217ca359f2544b7bb6169037c09690ab121efa9b16 I0425 23:10:06.743256 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0425 23:10:06.743285 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=bcb68afe8631b544ed9dee8326ebdc584ee1a37637c9ff15e694647d0b0e3075 I0425 23:10:06.743397 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=782bc628004057682f8d7cad126d8dd2106ab87fb61d9b6e2c8a65102ac8773f I0425 23:10:06.743411 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0425 23:10:06.743425 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0425 23:10:06.743466 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0425 23:10:06.743499 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=ce463825355f67f1b02bf6d4305a87976bdfe3704e0b23f488699c3619ae75f5 I0425 23:10:06.743533 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=808993e512c5c4406581cafc4643b59749001cfbe111d9ed00427a66e94ab8cc I0425 23:10:06.743548 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0425 23:10:06.743572 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=81415a4a07ffe9077dc5edd80ecf0876fa15669e2b38406795311f1a9c683bb0 I0425 23:10:06.743587 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0425 23:10:06.743612 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=f5a03250f64dadbf93296e40fe7aa991ea454e1bfc04f734e4f774dd65f998f3 I0425 23:10:06.743637 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=821444fcc0fe9b750618ea5353005af95efecf876275bf54784d6979f6f05cf5 I0425 23:10:06.743662 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=ca8742a45688e8a329c0ae58d85fce9730515a1fd58e88bf19a85d226368a3a0 I0425 23:10:06.743687 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=1e23cd649737727b35fe2feb7072e4b2319bb33476f9f38e6b0d6539f6b2c629 I0425 23:10:06.743719 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=76efa4a2062250a78a0354a4fb609468b494d0bbd28b29b6f3a5fed009179509 I0425 23:10:06.743734 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0425 23:10:06.743766 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=91c6842d22c5ad1de7aec430ecff0ff898d45ff79274d97e2fab029b98a4571c I0425 23:10:06.743792 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0425 23:10:06.743805 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0425 23:10:06.743817 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.464553793s to process 38 records I0425 23:10:06.806529 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-gwk7n pod in namespace openshift-image-registry (previous: false). I0425 23:10:07.007270 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5f5b65f667-c4sxm pod in namespace openshift-ingress (previous: false). I0425 23:10:07.206938 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5f5b65f667-ft4xk pod in namespace openshift-ingress (previous: false). W0425 23:10:07.317369 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0425 23:10:07.407697 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6b54945f69-l9bgg pod in namespace openshift-ingress (previous: false). I0425 23:10:07.616628 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-j76bm pod in namespace openshift-ingress-canary (previous: false). I0425 23:10:07.825864 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-j76bm pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-j76bm\" is waiting to start: ContainerCreating" I0425 23:10:07.825884 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-j76bm\" is waiting to start: ContainerCreating" I0425 23:10:07.825920 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-n9ldj pod in namespace openshift-ingress-canary (previous: false). I0425 23:10:08.010682 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-n9ldj pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-n9ldj\" is waiting to start: ContainerCreating" I0425 23:10:08.010703 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-n9ldj\" is waiting to start: ContainerCreating" I0425 23:10:08.010721 1 tasks_processing.go:74] worker 50 stopped. I0425 23:10:08.010832 1 recorder.go:75] Recording events/openshift-dns with fingerprint=8bdb2375be0a454ecae046c2c40ab3fd56f5343e5447eafd198eb5b02edfbfe3 I0425 23:10:08.010961 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=b1a2c51d44091e1cf13751b598721bb4db1f8efe41b31afd55f25a2fe993c14c I0425 23:10:08.010991 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=f755eb93a0002d4fa398d526bc9b9152ac6905aeef1525f6f7db799da354716c I0425 23:10:08.011070 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=e64c500adb7c94ea3145a96d555d8585b82650a74897c1023d8ae58cec6cf701 I0425 23:10:08.011086 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=06de4b4ee801178c650144cbccf7244b8b4ac91721c88d4b00b28d3f91b5b84c I0425 23:10:08.011270 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-9j9g5 with fingerprint=b0d2e34618162949c9e27b9900b9e6ed210c901b5f9c9f7037a5ba198644fe7d I0425 23:10:08.011370 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-hr96k with fingerprint=cc5c41ddc0497c7a1bccbd08385e6924951dcedf3b5ef20b184d1b3a9bb43641 I0425 23:10:08.011408 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-6485787969-spkvj/registry_current.log with fingerprint=3184c3159039bcbd5f7d163c77912328b8a97f6a57c312ca80ad57c999c0bede I0425 23:10:08.011542 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-777f7d7b64-zmjdc with fingerprint=e511d181e518307e85724373a3915d62838f844d248cee7746a1264a9cd3794c I0425 23:10:08.011553 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-9qqh2/node-ca_current.log with fingerprint=3b20450e9fb20d267a885651aa3f3df4d52db4c16b28b2539915697045d8d9b1 I0425 23:10:08.011558 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-gwk7n/node-ca_current.log with fingerprint=3aabd72e13616dd1c11d08e2291ab621b682b9a3ab26e6207d82076d6b7be3aa I0425 23:10:08.011585 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5f5b65f667-c4sxm/router_current.log with fingerprint=95bece75caddc0d76512708d3e9906d611dedd1591282d1436a5f71279291910 I0425 23:10:08.011614 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5f5b65f667-ft4xk/router_current.log with fingerprint=ff3f21526cc7186e2eb8e691bc50cf6a7235f35c08c88a1580338da235107c4e I0425 23:10:08.011671 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-6b54945f69-l9bgg/router_current.log with fingerprint=a226a70b265bb9c48affe5e53ae8b344cbf3f1f205b76db530a4008d0d007bdf I0425 23:10:08.011737 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-j76bm with fingerprint=0f918fe049801cb7a22ec01396c3eb554e370b49985851e47dc71c9e4972d656 I0425 23:10:08.011792 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-n9ldj with fingerprint=a6768387274852c130865c3893b50e6e3664f2514316db1161c3ede5fe9b2f5f I0425 23:10:08.011802 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 3.731080232s to process 16 records W0425 23:10:08.315988 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0425 23:10:09.312289 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded E0425 23:10:09.312326 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0425 23:10:09.312341 1 tasks_processing.go:74] worker 53 stopped. I0425 23:10:09.312350 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0425 23:10:09.312367 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.033359445s to process 1 records I0425 23:10:17.197298 1 tasks_processing.go:74] worker 45 stopped. I0425 23:10:17.197338 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0425 23:10:17.197352 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.919416545s to process 1 records I0425 23:10:17.885223 1 tasks_processing.go:74] worker 51 stopped. I0425 23:10:17.885518 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=f5738032f28aea743b389898fadee779c5cd56b79a04dcc121fb2c80f0e9fc94 I0425 23:10:17.885536 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.607400471s to process 1 records E0425 23:10:17.885629 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.608s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0425 23:10:17.886741 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0425 23:10:17.886758 1 periodic.go:209] Running workloads gatherer I0425 23:10:17.886775 1 tasks_processing.go:45] number of workers: 2 I0425 23:10:17.886786 1 tasks_processing.go:69] worker 1 listening for tasks. I0425 23:10:17.886791 1 tasks_processing.go:71] worker 1 working on workload_info task. I0425 23:10:17.886800 1 tasks_processing.go:69] worker 0 listening for tasks. I0425 23:10:17.886822 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0425 23:10:17.892456 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-210.ec2.internal... I0425 23:10:17.892463 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-89.ec2.internal... I0425 23:10:17.909987 1 tasks_processing.go:74] worker 0 stopped. I0425 23:10:17.910003 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 23.143888ms to process 0 records I0425 23:10:23.973762 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 6s I0425 23:10:23.996294 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 23s for image data I0425 23:10:24.005418 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0425 23:10:24.015315 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0425 23:10:24.024332 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (9ms) I0425 23:10:24.033317 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (9ms) I0425 23:10:24.042207 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (9ms) I0425 23:10:24.050626 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (8ms) I0425 23:10:24.058755 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (8ms) I0425 23:10:24.066814 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (8ms) I0425 23:10:24.074809 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (8ms) I0425 23:10:24.083010 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (8ms) I0425 23:10:24.105235 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (22ms) I0425 23:10:24.205277 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0425 23:10:24.305791 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0425 23:10:24.405264 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (99ms) I0425 23:10:24.508958 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (104ms) I0425 23:10:24.605809 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (97ms) I0425 23:10:24.706197 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0425 23:10:24.804861 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0425 23:10:24.907921 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (103ms) I0425 23:10:25.006486 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (99ms) I0425 23:10:25.112868 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (106ms) I0425 23:10:25.205308 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (92ms) I0425 23:10:25.305634 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (100ms) I0425 23:10:25.405337 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (100ms) I0425 23:10:25.505773 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0425 23:10:25.605661 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0425 23:10:25.705453 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0425 23:10:25.805950 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (100ms) I0425 23:10:25.906786 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (101ms) I0425 23:10:26.005978 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (99ms) I0425 23:10:26.106059 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0425 23:10:26.212421 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (106ms) I0425 23:10:26.306888 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (94ms) I0425 23:10:26.406981 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0425 23:10:26.505370 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (98ms) I0425 23:10:26.604869 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0425 23:10:26.705138 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (100ms) I0425 23:10:26.805396 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (100ms) I0425 23:10:26.905030 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0425 23:10:27.005944 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (101ms) I0425 23:10:27.105686 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (100ms) I0425 23:10:27.205260 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0425 23:10:27.306267 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (101ms) I0425 23:10:27.404942 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (99ms) I0425 23:10:27.506274 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (101ms) I0425 23:10:27.605189 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (99ms) I0425 23:10:27.704682 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (99ms) I0425 23:10:27.806963 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (102ms) I0425 23:10:27.806993 1 tasks_processing.go:74] worker 1 stopped. I0425 23:10:27.807322 1 recorder.go:75] Recording config/workload_info with fingerprint=c3c43477acef8627094637d9b37885de913d0fa0fed3d8e71e7369ee95ddd647 I0425 23:10:27.807340 1 gather.go:177] gatherer "workloads" function "workload_info" took 9.920195508s to process 1 records I0425 23:10:27.807355 1 periodic.go:256] Periodic gather workloads completed in 9.92s I0425 23:10:27.807362 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0425 23:10:27.807367 1 periodic.go:209] Running conditional gatherer I0425 23:10:27.814136 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0425 23:10:27.820868 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.17:40840->172.30.0.10:53: read: connection refused I0425 23:10:27.821180 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0425 23:10:27.824493 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.17:49865->172.30.0.10:53: read: connection refused I0425 23:10:27.824565 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0425 23:10:27.830324 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0425 23:10:27.830340 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830346 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830350 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830353 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830357 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830362 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830365 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830368 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0425 23:10:27.830371 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0425 23:10:27.830387 1 tasks_processing.go:45] number of workers: 3 I0425 23:10:27.830400 1 tasks_processing.go:69] worker 2 listening for tasks. I0425 23:10:27.830404 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0425 23:10:27.830412 1 tasks_processing.go:69] worker 0 listening for tasks. I0425 23:10:27.830427 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0425 23:10:27.830426 1 tasks_processing.go:69] worker 1 listening for tasks. I0425 23:10:27.830433 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0425 23:10:27.830437 1 tasks_processing.go:74] worker 1 stopped. I0425 23:10:27.830492 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0425 23:10:27.830505 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 907ns to process 1 records I0425 23:10:27.830537 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0425 23:10:27.830547 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.128µs to process 1 records I0425 23:10:27.830556 1 tasks_processing.go:74] worker 0 stopped. I0425 23:10:27.830716 1 tasks_processing.go:74] worker 2 stopped. I0425 23:10:27.830738 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 274.045µs to process 0 records I0425 23:10:27.830773 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.17:40840->172.30.0.10:53: read: connection refused I0425 23:10:27.830794 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0425 23:10:27.860722 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=21f91c0d8f40d5ec7a69673fe980606118f428d96dbbf0ec7d63dad69bb11927 I0425 23:10:27.860843 1 diskrecorder.go:70] Writing 123 records to /var/lib/insights-operator/insights-2026-04-25-231027.tar.gz I0425 23:10:27.868597 1 diskrecorder.go:51] Wrote 123 records to disk in 7ms I0425 23:10:27.868627 1 periodic.go:278] Gathering cluster info every 2h0m0s I0425 23:10:27.868640 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0425 23:11:34.277423 1 diskrecorder.go:223] Found files to send: insights-2026-04-25-231027.tar.gz I0425 23:11:34.277454 1 insightsuploader.go:150] Checking archives to upload periodically every 16m0.462402666s I0425 23:11:34.277464 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0425 23:11:34.288587 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0425 23:11:34.532286 1 requests.go:87] Successfully reported id=2026-04-25T23:11:34Z x-rh-insights-request-id=f1a245962b0247de932f97d409df878f, wrote=61217 I0425 23:11:34.532340 1 insightsuploader.go:187] Uploaded report successfully in 254.869651ms I0425 23:11:34.532357 1 controller.go:128] Initializing last reported time to 2026-04-25T23:11:34Z I0425 23:11:34.532372 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0425 23:11:34.532386 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0425 23:11:34.532394 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0425 23:11:34.545061 1 controller.go:512] The operator is healthy I0425 23:12:04.289712 1 controller.go:512] The operator is healthy I0425 23:12:39.554909 1 insightsreport.go:137] Pulling report from smart-proxy I0425 23:12:39.554936 1 insightsreport.go:149] Retrieving report I0425 23:12:39.561857 1 requests.go:110] Retrieving report for cluster: fed508f2-a6c4-4ff5-b23a-0aa59d7606cf I0425 23:12:39.561870 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/fed508f2-a6c4-4ff5-b23a-0aa59d7606cf/reports I0425 23:12:39.566581 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/fed508f2-a6c4-4ff5-b23a-0aa59d7606cf/reports I0425 23:12:39.785537 1 insightsreport.go:184] Report retrieved I0425 23:12:39.812267 1 insightsreport.go:239] Report retrieved correctly I0425 23:14:04.289268 1 controller.go:512] The operator is healthy I0425 23:15:04.268459 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0425 23:15:04.273923 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 23:15:04.273944 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0425 23:15:04.278309 1 secretconfigobserver.go:119] support secret does not exist I0425 23:16:04.289564 1 controller.go:512] The operator is healthy I0425 23:18:04.289654 1 controller.go:512] The operator is healthy I0425 23:20:04.267073 1 configmapobserver.go:84] configmaps "insights-config" not found I0425 23:20:04.279197 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0425 23:20:04.283789 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 23:20:04.283808 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0425 23:20:04.287610 1 secretconfigobserver.go:119] support secret does not exist I0425 23:20:04.288361 1 controller.go:512] The operator is healthy I0425 23:22:04.293415 1 controller.go:512] The operator is healthy