I0704 18:16:14.309616 1 cmd.go:253] Using service-serving-cert provided certificates I0704 18:16:14.310099 1 observer_polling.go:159] Starting file observer I0704 18:16:14.786298 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0704 18:16:14.786508 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0704 18:16:14.787062 1 secure_serving.go:57] Forcing use of http/1.1 only W0704 18:16:14.787085 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0704 18:16:14.787092 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0704 18:16:14.787097 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0704 18:16:14.787102 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0704 18:16:14.787106 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0704 18:16:14.787112 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0704 18:16:14.787183 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0704 18:16:14.790760 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0704 18:16:14.790788 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"ab1a42d4-ed64-4d4e-8ba1-e9da502519c2", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0704 18:16:14.792075 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0704 18:16:14.792101 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0704 18:16:14.792096 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0704 18:16:14.792106 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0704 18:16:14.792122 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 18:16:14.792112 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 18:16:14.792402 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0704 18:16:14.792705 1 secure_serving.go:213] Serving securely on [::]:8443 I0704 18:16:14.792759 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0704 18:16:14.795197 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0704 18:16:14.795224 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 18:16:14.795328 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0704 18:16:14.799936 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:16:14.799956 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 18:16:14.803862 1 secretconfigobserver.go:119] support secret does not exist I0704 18:16:14.808343 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:16:14.811936 1 secretconfigobserver.go:119] support secret does not exist I0704 18:16:14.814637 1 recorder.go:161] Pruning old reports every 5h13m58s, max age is 288h0m0s I0704 18:16:14.818953 1 periodic.go:209] Running clusterconfig gatherer I0704 18:16:14.818970 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0704 18:16:14.818980 1 insightsreport.go:296] Starting report retriever I0704 18:16:14.818987 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0704 18:16:14.819017 1 tasks_processing.go:45] number of workers: 32 I0704 18:16:14.819046 1 tasks_processing.go:69] worker 31 listening for tasks. I0704 18:16:14.819055 1 tasks_processing.go:71] worker 31 working on machines task. I0704 18:16:14.819069 1 tasks_processing.go:69] worker 22 listening for tasks. I0704 18:16:14.819069 1 tasks_processing.go:69] worker 14 listening for tasks. I0704 18:16:14.819081 1 tasks_processing.go:69] worker 26 listening for tasks. I0704 18:16:14.819084 1 tasks_processing.go:69] worker 15 listening for tasks. I0704 18:16:14.819089 1 tasks_processing.go:69] worker 7 listening for tasks. I0704 18:16:14.819091 1 tasks_processing.go:69] worker 16 listening for tasks. I0704 18:16:14.819095 1 tasks_processing.go:69] worker 23 listening for tasks. I0704 18:16:14.819098 1 tasks_processing.go:69] worker 8 listening for tasks. I0704 18:16:14.819099 1 tasks_processing.go:69] worker 17 listening for tasks. I0704 18:16:14.819105 1 tasks_processing.go:69] worker 18 listening for tasks. I0704 18:16:14.819109 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 18:16:14.819110 1 tasks_processing.go:69] worker 25 listening for tasks. I0704 18:16:14.819113 1 tasks_processing.go:69] worker 9 listening for tasks. I0704 18:16:14.819117 1 tasks_processing.go:69] worker 21 listening for tasks. I0704 18:16:14.819100 1 tasks_processing.go:69] worker 6 listening for tasks. I0704 18:16:14.819104 1 tasks_processing.go:69] worker 24 listening for tasks. I0704 18:16:14.819125 1 tasks_processing.go:69] worker 27 listening for tasks. I0704 18:16:14.818953 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0704 18:16:14.819127 1 tasks_processing.go:69] worker 20 listening for tasks. I0704 18:16:14.819140 1 tasks_processing.go:69] worker 12 listening for tasks. I0704 18:16:14.819127 1 tasks_processing.go:69] worker 10 listening for tasks. I0704 18:16:14.819145 1 tasks_processing.go:69] worker 5 listening for tasks. I0704 18:16:14.819148 1 tasks_processing.go:69] worker 29 listening for tasks. I0704 18:16:14.819133 1 tasks_processing.go:69] worker 4 listening for tasks. I0704 18:16:14.819153 1 tasks_processing.go:69] worker 30 listening for tasks. I0704 18:16:14.819133 1 tasks_processing.go:69] worker 11 listening for tasks. I0704 18:16:14.819159 1 tasks_processing.go:71] worker 22 working on sap_config task. I0704 18:16:14.819160 1 tasks_processing.go:71] worker 14 working on nodenetworkstates task. I0704 18:16:14.819167 1 tasks_processing.go:71] worker 9 working on support_secret task. I0704 18:16:14.819171 1 tasks_processing.go:71] worker 10 working on mutating_webhook_configurations task. I0704 18:16:14.819177 1 tasks_processing.go:71] worker 12 working on feature_gates task. I0704 18:16:14.819137 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 18:16:14.819385 1 tasks_processing.go:71] worker 6 working on pdbs task. I0704 18:16:14.819390 1 tasks_processing.go:71] worker 2 working on infrastructures task. I0704 18:16:14.819434 1 tasks_processing.go:71] worker 23 working on monitoring_persistent_volumes task. I0704 18:16:14.819537 1 tasks_processing.go:71] worker 24 working on machine_configs task. I0704 18:16:14.819136 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0704 18:16:14.819573 1 tasks_processing.go:71] worker 8 working on aggregated_monitoring_cr_names task. I0704 18:16:14.819121 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 18:16:14.819602 1 tasks_processing.go:71] worker 1 working on cost_management_metrics_configs task. I0704 18:16:14.819653 1 tasks_processing.go:71] worker 5 working on machine_config_pools task. I0704 18:16:14.819758 1 tasks_processing.go:71] worker 17 working on qemu_kubevirt_launcher_logs task. I0704 18:16:14.819810 1 tasks_processing.go:71] worker 20 working on container_runtime_configs task. I0704 18:16:14.819140 1 tasks_processing.go:69] worker 3 listening for tasks. I0704 18:16:14.819149 1 tasks_processing.go:69] worker 19 listening for tasks. I0704 18:16:14.819150 1 tasks_processing.go:69] worker 13 listening for tasks. I0704 18:16:14.819380 1 tasks_processing.go:71] worker 21 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0704 18:16:14.819847 1 tasks_processing.go:71] worker 26 working on ingress task. I0704 18:16:14.819805 1 tasks_processing.go:71] worker 27 working on install_plans task. I0704 18:16:14.819913 1 tasks_processing.go:71] worker 29 working on nodenetworkconfigurationpolicies task. I0704 18:16:14.819983 1 tasks_processing.go:71] worker 7 working on dvo_metrics task. I0704 18:16:14.820058 1 tasks_processing.go:71] worker 16 working on oauths task. I0704 18:16:14.820176 1 tasks_processing.go:71] worker 4 working on overlapping_namespace_uids task. I0704 18:16:14.820197 1 tasks_processing.go:71] worker 18 working on openshift_machine_api_events task. I0704 18:16:14.820246 1 tasks_processing.go:71] worker 30 working on machine_healthchecks task. I0704 18:16:14.820383 1 tasks_processing.go:71] worker 13 working on crds task. I0704 18:16:14.820447 1 tasks_processing.go:71] worker 3 working on certificate_signing_requests task. I0704 18:16:14.820500 1 tasks_processing.go:71] worker 15 working on silenced_alerts task. I0704 18:16:14.820544 1 tasks_processing.go:71] worker 19 working on version task. I0704 18:16:14.820649 1 tasks_processing.go:71] worker 25 working on operators_pods_and_events task. I0704 18:16:14.819167 1 tasks_processing.go:71] worker 11 working on machine_sets task. I0704 18:16:14.820664 1 tasks_processing.go:71] worker 0 working on storage_classes task. I0704 18:16:14.820750 1 tasks_processing.go:69] worker 28 listening for tasks. I0704 18:16:14.820764 1 tasks_processing.go:71] worker 28 working on storage_cluster task. I0704 18:16:14.821974 1 tasks_processing.go:71] worker 14 working on openshift_logging task. I0704 18:16:14.821999 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 2.799399ms to process 0 records I0704 18:16:14.823136 1 tasks_processing.go:71] worker 22 working on ceph_cluster task. I0704 18:16:14.823344 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 3.950923ms to process 0 records I0704 18:16:14.823719 1 tasks_processing.go:71] worker 1 working on machine_autoscalers task. I0704 18:16:14.824080 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 4.107735ms to process 0 records I0704 18:16:14.826037 1 tasks_processing.go:71] worker 29 working on authentication task. I0704 18:16:14.826062 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 6.073101ms to process 0 records I0704 18:16:14.826170 1 tasks_processing.go:71] worker 31 working on openstack_version task. E0704 18:16:14.826183 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0704 18:16:14.826198 1 gather.go:177] gatherer "clusterconfig" function "machines" took 7.102793ms to process 0 records I0704 18:16:14.832476 1 tasks_processing.go:71] worker 30 working on image task. E0704 18:16:14.832487 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0704 18:16:14.832649 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 12.177113ms to process 0 records I0704 18:16:14.832659 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 11.803594ms to process 0 records I0704 18:16:14.832665 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 11.781656ms to process 0 records I0704 18:16:14.832670 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 12.428599ms to process 0 records I0704 18:16:14.832677 1 tasks_processing.go:71] worker 18 working on openstack_dataplanedeployments task. I0704 18:16:14.832869 1 tasks_processing.go:71] worker 11 working on active_alerts task. I0704 18:16:14.832925 1 tasks_processing.go:71] worker 9 working on container_images task. E0704 18:16:14.832937 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0704 18:16:14.833390 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 13.748945ms to process 0 records I0704 18:16:14.833710 1 recorder.go:75] Recording config/ingress with fingerprint=4966ecbc5be945a172b7bf97269243c4e13585e36b6d54f26a5c47fe97b36bc4 I0704 18:16:14.833758 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 13.305797ms to process 1 records I0704 18:16:14.833817 1 tasks_processing.go:71] worker 2 working on nodes task. I0704 18:16:14.832939 1 tasks_processing.go:71] worker 28 working on operators task. I0704 18:16:14.834973 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0704 18:16:14.835049 1 controller.go:326] The initial operator extension status is healthy I0704 18:16:14.835084 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 18:16:14.835113 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0704 18:16:14.835140 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0704 18:16:14.835203 1 controller.go:489] The operator is still being initialized I0704 18:16:14.835232 1 controller.go:512] The operator is healthy W0704 18:16:14.835985 1 operator.go:288] started I0704 18:16:14.836154 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0704 18:16:14.836837 1 recorder.go:75] Recording config/infrastructure with fingerprint=57282429e7cbd290602cd1d1ba6e0a40e670d1b066ea4b0046dbe8c76009097a I0704 18:16:14.836872 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0704 18:16:14.837016 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0704 18:16:14.836909 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 13.78181ms to process 1 records I0704 18:16:14.837195 1 tasks_processing.go:71] worker 26 working on openstack_controlplanes task. I0704 18:16:14.837721 1 tasks_processing.go:71] worker 16 working on cluster_apiserver task. I0704 18:16:14.837817 1 recorder.go:75] Recording config/oauth with fingerprint=21ef8a5f557f5467a5daaba6b636a3578be3feb5702468de5d33f9c1f8d69160 I0704 18:16:14.837882 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 13.172516ms to process 1 records I0704 18:16:14.838159 1 recorder.go:75] Recording config/featuregate with fingerprint=a263834283590a306805dbe214b2ffaa6bbfa8c2f038d9a684088163092c23f1 I0704 18:16:14.838234 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 14.465873ms to process 1 records I0704 18:16:14.838268 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 9.87714ms to process 0 records I0704 18:16:14.838304 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 7.493158ms to process 0 records I0704 18:16:14.838338 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 11.019804ms to process 0 records I0704 18:16:14.838519 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=8fca1e3c9cdad1c7ba2a1337ea824423dab0ccbbb8b0b8c5c6573d7be00c49ca I0704 18:16:14.838657 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=107074f2314c36307e750c5b37c8cca22c56f4707337067786984edf9317d032 I0704 18:16:14.838717 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 13.591304ms to process 2 records I0704 18:16:14.838777 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 12.523018ms to process 0 records I0704 18:16:14.838812 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 14.747985ms to process 0 records I0704 18:16:14.839113 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 19.397096ms to process 0 records I0704 18:16:14.839674 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 19.175019ms to process 0 records I0704 18:16:14.839938 1 tasks_processing.go:71] worker 3 working on proxies task. I0704 18:16:14.839971 1 tasks_processing.go:71] worker 12 working on openstack_dataplanenodesets task. I0704 18:16:14.840109 1 tasks_processing.go:71] worker 20 working on sap_datahubs task. I0704 18:16:14.840122 1 tasks_processing.go:71] worker 1 working on metrics task. I0704 18:16:14.840132 1 tasks_processing.go:71] worker 0 working on tsdb_status task. I0704 18:16:14.840386 1 tasks_processing.go:71] worker 14 working on image_registries task. I0704 18:16:14.840495 1 tasks_processing.go:71] worker 22 working on jaegers task. I0704 18:16:14.840559 1 tasks_processing.go:71] worker 5 working on image_pruners task. I0704 18:16:14.841456 1 tasks_processing.go:71] worker 31 working on config_maps task. I0704 18:16:14.842969 1 tasks_processing.go:71] worker 23 working on lokistack task. I0704 18:16:14.843169 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 23.519641ms to process 0 records I0704 18:16:14.843261 1 gather_logs.go:145] no pods in namespace were found I0704 18:16:14.843380 1 tasks_processing.go:71] worker 10 working on service_accounts task. I0704 18:16:14.843483 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=c1664d6400997576eeda39b19f8d67b603e7eb908cf41821fe6694535108788c I0704 18:16:14.843531 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=782888244acb9e79ef630bfd091c1269b5e65a952661872e13205e7f1786142c I0704 18:16:14.843630 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=782970e4a79d516b354afd2ef112e76e0d1d94b1bb7125c7bff94089e7bd0c78 I0704 18:16:14.843648 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 23.819167ms to process 3 records I0704 18:16:14.843662 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 10.324318ms to process 0 records I0704 18:16:14.843671 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 23.455969ms to process 0 records I0704 18:16:14.843680 1 tasks_processing.go:71] worker 17 working on schedulers task. I0704 18:16:14.843872 1 tasks_processing.go:71] worker 18 working on sap_pods task. E0704 18:16:14.844163 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.81.232:9091: connect: connection refused I0704 18:16:14.844181 1 tasks_processing.go:71] worker 0 working on olm_operators task. E0704 18:16:14.844205 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.81.232:9091: connect: connection refused E0704 18:16:14.844208 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.81.232:9091: connect: connection refused I0704 18:16:14.844233 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.038153ms to process 0 records E0704 18:16:14.844247 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.81.232:9091: connect: connection refused I0704 18:16:14.844270 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 4.096573ms to process 0 records I0704 18:16:14.844421 1 recorder.go:75] Recording config/image with fingerprint=c22937246384c8bcdfff9cfa803189d9e1622a1273bd12b88a0cf4fcd5d75fd7 I0704 18:16:14.844440 1 gather.go:177] gatherer "clusterconfig" function "image" took 11.809078ms to process 1 records I0704 18:16:14.844475 1 tasks_processing.go:71] worker 30 working on validating_webhook_configurations task. I0704 18:16:14.844519 1 tasks_processing.go:71] worker 6 working on ingress_certificates task. I0704 18:16:14.844547 1 tasks_processing.go:71] worker 1 working on pod_network_connectivity_checks task. I0704 18:16:14.844859 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=2c10ba0a9880a56a0a1571309fc997add3d1e50eb5068375b4e3bcc1c6a80d26 I0704 18:16:14.844887 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=3fd8a3b46e1412ced3d6d00ca08234a2d49ff7aed657b6db57a36bf87432b836 I0704 18:16:14.844909 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=124d09f053e17f0c0c0f2637de38d69803171a1ec5563922edb37b0b830dfd94 I0704 18:16:14.844930 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=fa7f3fa84ac6a8ef4f24cfb209aa698da3a42e1c3098368c490f990961473945 I0704 18:16:14.844955 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=6860fe03c771764a2faaaeae9386ee651e49d1d4e13e783f7a4ca19688f6e904 I0704 18:16:14.845002 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=e1bc4045bc05de34b8ac94a8a2f4fca3b39e1aec73be2fb52802cf64ecf6675f I0704 18:16:14.845030 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=b37b5494d164c119f1277aae42025262d7f33d22a767bbb2cfae0ddae8af5acc I0704 18:16:14.845055 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=41c90bd3c209f5280a565730efc3781292620b8707c8746992f808f36cd83aaf I0704 18:16:14.845075 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=2b5b997d2c0d0c418b0143ab1844f019c91dfba9fbeb696461d20b3349558b98 I0704 18:16:14.845103 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=ee4c800388c7779ff5698aa5e5b8eee7647f7ea78e72d55d2242cd8884912120 I0704 18:16:14.845132 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=be9a86c98c82151a06a180e7511064d22f33a6a5df96d03be726ffd29fde239c I0704 18:16:14.845143 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 25.123255ms to process 11 records I0704 18:16:14.845223 1 tasks_processing.go:71] worker 29 working on clusterroles task. I0704 18:16:14.845361 1 recorder.go:75] Recording config/authentication with fingerprint=a2c471a9b23275c73c594ae78e2a3ed7f54fa89ec809a18badfd7126db9c4695 I0704 18:16:14.845374 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 18.901994ms to process 1 records I0704 18:16:14.848385 1 tasks_processing.go:71] worker 26 working on node_logs task. I0704 18:16:14.848774 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 11.12846ms to process 0 records I0704 18:16:14.850790 1 tasks_processing.go:71] worker 25 working on networks task. I0704 18:16:14.850879 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 30.084788ms to process 0 records I0704 18:16:14.851475 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 11.114533ms to process 0 records I0704 18:16:14.851542 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0704 18:16:14.851595 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 31.421419ms to process 1 records I0704 18:16:14.851519 1 tasks_processing.go:74] worker 20 stopped. I0704 18:16:14.851574 1 tasks_processing.go:74] worker 7 stopped. I0704 18:16:14.853069 1 tasks_processing.go:74] worker 12 stopped. I0704 18:16:14.853090 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 13.075501ms to process 0 records I0704 18:16:14.853100 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 12.021383ms to process 0 records I0704 18:16:14.853109 1 tasks_processing.go:74] worker 22 stopped. I0704 18:16:14.859704 1 tasks_processing.go:74] worker 23 stopped. I0704 18:16:14.859750 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 16.67383ms to process 0 records I0704 18:16:14.860647 1 tasks_processing.go:74] worker 18 stopped. I0704 18:16:14.860795 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 16.757181ms to process 0 records I0704 18:16:14.861060 1 tasks_processing.go:74] worker 19 stopped. I0704 18:16:14.861983 1 recorder.go:75] Recording config/version with fingerprint=c84af13891abce6bb8ecea57dcff0d9dd6e9a8f7e34ebf2d8c7d338057542f7d I0704 18:16:14.862059 1 recorder.go:75] Recording config/id with fingerprint=a7867e71b65c466072972171bf1ad8d4c4fb062600832135be6b3d3dee27a968 I0704 18:16:14.862098 1 gather.go:177] gatherer "clusterconfig" function "version" took 40.498035ms to process 2 records I0704 18:16:14.862235 1 recorder.go:75] Recording config/proxy with fingerprint=8afc9f73909b5d3c32d034c68f01205c85609f91f2db0f14a529c31219bd31be I0704 18:16:14.862271 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 21.330516ms to process 1 records I0704 18:16:14.862337 1 tasks_processing.go:74] worker 3 stopped. I0704 18:16:14.862630 1 tasks_processing.go:74] worker 5 stopped. I0704 18:16:14.862859 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=8caf1cf46da7795536ab97ddd550bf5912526e5585f168858b874a2afd620d2c I0704 18:16:14.862917 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 20.286926ms to process 1 records I0704 18:16:14.863450 1 tasks_processing.go:74] worker 13 stopped. I0704 18:16:14.865825 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=8ae66d4ae73028aa71213ef7e7e704fe69489cfb21985f3006e3399c60ab4e6c I0704 18:16:14.866593 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=536e197f8e175828fd3cde2d7da8e0aa3f4fb4edd50810ac89b258f2963612fe I0704 18:16:14.866663 1 gather.go:177] gatherer "clusterconfig" function "crds" took 41.929188ms to process 2 records I0704 18:16:14.866875 1 tasks_processing.go:74] worker 16 stopped. I0704 18:16:14.867143 1 recorder.go:75] Recording config/apiserver with fingerprint=d6f6a02297bbe11995326a5e2bc41f0ac4564a6ee81982a764c9749f84f061bd I0704 18:16:14.867193 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 26.015279ms to process 1 records I0704 18:16:14.867251 1 tasks_processing.go:74] worker 1 stopped. E0704 18:16:14.867252 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0704 18:16:14.867333 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 19.864913ms to process 0 records I0704 18:16:14.867534 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e3f06220bceeab695f868f497fb1b48911ba7f4ee04d2b1b946479e84b999c53 I0704 18:16:14.867581 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 22.018073ms to process 1 records I0704 18:16:14.867630 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0704 18:16:14.867671 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 47.050524ms to process 1 records I0704 18:16:14.867707 1 tasks_processing.go:74] worker 4 stopped. I0704 18:16:14.867708 1 tasks_processing.go:74] worker 17 stopped. I0704 18:16:14.868226 1 tasks_processing.go:74] worker 26 stopped. I0704 18:16:14.868275 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 19.780965ms to process 0 records I0704 18:16:14.868592 1 tasks_processing.go:74] worker 2 stopped. I0704 18:16:14.868766 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0704 18:16:14.868822 1 controller.go:212] Source scaController *sca.Controller is not ready I0704 18:16:14.868854 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0704 18:16:14.868882 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 18:16:14.868908 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0704 18:16:14.869111 1 controller.go:489] The operator is still being initialized I0704 18:16:14.869149 1 controller.go:512] The operator is healthy I0704 18:16:14.870612 1 recorder.go:75] Recording config/node/ip-10-0-0-39.ec2.internal with fingerprint=f5b2ede17bff325d37d0a8259ac3eafc25d7ec00d4ab428f87592ed680b70b7e I0704 18:16:14.870815 1 recorder.go:75] Recording config/node/ip-10-0-1-178.ec2.internal with fingerprint=2368f35c86bf5ec49693630d4bb33e164215a500728792bdd93d9f9624c76d88 I0704 18:16:14.870983 1 recorder.go:75] Recording config/node/ip-10-0-2-42.ec2.internal with fingerprint=3f0bdd682cd1add70bf3d6d481aa61f00e28fb78c36fb9740949de00c001f458 I0704 18:16:14.871022 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 34.412871ms to process 3 records I0704 18:16:14.871257 1 recorder.go:75] Recording config/network with fingerprint=d40fbedec8ac1a192be8c06835da3280cfdb456acb3eb4e4931144c9bfac137b I0704 18:16:14.871298 1 gather.go:177] gatherer "clusterconfig" function "networks" took 17.815902ms to process 1 records I0704 18:16:14.871332 1 tasks_processing.go:74] worker 25 stopped. I0704 18:16:14.871487 1 tasks_processing.go:74] worker 14 stopped. I0704 18:16:14.872151 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=eb2a20be28cc87ca9b6144019791665fe06674759891d5614a650440f21a521e I0704 18:16:14.872171 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 31.041305ms to process 1 records I0704 18:16:14.872325 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=ce8024219c5ff0bc1529281c15e815a975b204b5fd9f55bfae1c45f09a7810a9 I0704 18:16:14.872566 1 tasks_processing.go:74] worker 30 stopped. I0704 18:16:14.872581 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=5605c743b6dae1b32b6e32f4ec22ecd0a25c65af817506c070b1cccfc86f9dfa I0704 18:16:14.872927 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=69edebc306e4b40e86922834a45d5b86a9be3e730ad1a812288fdb59d98a772c I0704 18:16:14.872989 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=bcb9865c75cd151ba6c3beafb134d3263efaf2908f26b7fb3b3f2dfb5ef3e2d9 I0704 18:16:14.873033 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=c389bfe51a128f3d77e82c7cccf81a9c16d930c048deaa3001fb32a99ed9d206 I0704 18:16:14.873077 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=361df9cccf677804c2bbbe94f226a2b6a9617679e07e9c04a24edcb91b10900b I0704 18:16:14.873145 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=730f89ece56b00da1953ae4d29248ca135af11e20590e3ba9da8bf4355efa07a I0704 18:16:14.873203 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=da5b9ded72d53269ed9bf97e104462d9f764cbdd6f925a6a14ef68548bf6cd0b I0704 18:16:14.873256 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=6e8d1619fe092849c3ea9d6f43efc236a3847f54ce7aa1a670de604314173f7d I0704 18:16:14.873305 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=55e3b81d6f5ebe5dd602e2fe36e9ca5dc78d049eaf2b0229f288b8f191f1389b I0704 18:16:14.873344 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=1f609fb7527b74c7978250630f3fac1385ad8863401cb44bdc9933ef986016f5 I0704 18:16:14.873394 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=3b9fcc3c8f5fc2ef8b3ea116fa548ab29c4d1595fbc0075e368f4fb45ae531d9 I0704 18:16:14.873442 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=a79ec5c1463bcd4a1b8664a54e2594151992e16a33ab995281b36cbf5a90c9d2 I0704 18:16:14.873453 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 27.658212ms to process 13 records I0704 18:16:14.873465 1 tasks_processing.go:74] worker 8 stopped. I0704 18:16:14.873477 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 53.881762ms to process 0 records I0704 18:16:14.878192 1 tasks_processing.go:74] worker 24 stopped. I0704 18:16:14.878274 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0704 18:16:14.878324 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 58.640087ms to process 1 records I0704 18:16:14.881425 1 tasks_processing.go:74] worker 29 stopped. I0704 18:16:14.881768 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=bca7206600096f85aa181472ed9ff4a221643fc7f9b98cdfd838cc2a298faa9a I0704 18:16:14.881978 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=756bec5831846b253898b13ea0ce6c621808f64d2847c7f4b3a1aa403e5d368f I0704 18:16:14.882025 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 36.189252ms to process 2 records I0704 18:16:14.884630 1 tasks_processing.go:74] worker 0 stopped. I0704 18:16:14.884806 1 recorder.go:75] Recording config/olm_operators with fingerprint=270c689b336ad974b07f090ee86c042cc6e9da14093cfcb66afb1aff7f82179a I0704 18:16:14.884860 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 40.425793ms to process 1 records I0704 18:16:14.888127 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0704 18:16:14.889294 1 tasks_processing.go:74] worker 9 stopped. I0704 18:16:14.890947 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-cqt8b with fingerprint=6f2ac5e634327680dd722a6307ef61f001e0050dfa5eb02169e65e3704453299 I0704 18:16:14.891037 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-0-39.ec2.internal with fingerprint=53797ca89b2de4a7ecf60d61827e5ffa161131c6f4b42f3868a495841e261297 I0704 18:16:14.891283 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-96tb4 with fingerprint=57cb23a86ddbf56f66ed863d9fc100ce9d85d0673c88aee740f459b29cf30901 I0704 18:16:14.891519 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-pdmxz with fingerprint=0424d4a1d8fcebbcccc1847c3d40efe2cbd8a0dcfb03dbf2c3ae0b548f4e0638 I0704 18:16:14.891593 1 recorder.go:75] Recording config/running_containers with fingerprint=34b51089bda10bf75d22e818cb733ef46c5c4a5420a914b4375973a76eb3f329 I0704 18:16:14.891602 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 55.998648ms to process 5 records I0704 18:16:14.892213 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 18:16:14.892227 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 18:16:14.892220 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0704 18:16:14.892384 1 tasks_processing.go:74] worker 21 stopped. I0704 18:16:14.892397 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 72.526869ms to process 0 records I0704 18:16:14.895400 1 base_controller.go:82] Caches are synced for ConfigController I0704 18:16:14.895454 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0704 18:16:14.903624 1 tasks_processing.go:74] worker 31 stopped. E0704 18:16:14.903647 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0704 18:16:14.903661 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0704 18:16:14.903666 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0704 18:16:14.903680 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0704 18:16:14.903724 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0704 18:16:14.903756 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0704 18:16:14.903763 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0704 18:16:14.903772 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0704 18:16:14.903821 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0704 18:16:14.903842 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0704 18:16:14.903851 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 62.135736ms to process 7 records I0704 18:16:14.904790 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:16:14.904840 1 tasks_processing.go:74] worker 6 stopped. E0704 18:16:14.904856 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0704 18:16:14.904864 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rbfm8mkhd6428s0ui2gban8fum4jbk0-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rbfm8mkhd6428s0ui2gban8fum4jbk0-primary-cert-bundle-secret" not found I0704 18:16:14.904918 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=aca2b9eaa176ad1276d351dc4ec8342f4ea5e257c15ec4f067311ab05623da84 I0704 18:16:14.904934 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 60.297755ms to process 1 records I0704 18:16:14.936331 1 base_controller.go:82] Caches are synced for LoggingSyncer I0704 18:16:14.936352 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0704 18:16:14.940997 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0704 18:16:14.943079 1 cluster_transfer.go:102] no available accepted cluster transfer I0704 18:16:14.943094 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0704 18:16:15.100656 1 sca.go:266] etc-pki-entitlement secret successfully created I0704 18:16:15.100677 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0704 18:16:15.859895 1 tasks_processing.go:74] worker 11 stopped. I0704 18:16:15.859969 1 recorder.go:75] Recording config/alerts with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 W0704 18:16:15.859995 1 gather.go:155] issue recording gatherer "clusterconfig" function "active_alerts" result "config/alerts.json" because of the warning: warning: the record with the same fingerprint "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945" was already recorded at path "config/namespaces_with_overlapping_uids.json", recording another one with a different path "config/alerts.json" I0704 18:16:15.860010 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.026873135s to process 1 records I0704 18:16:15.860023 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0704 18:16:15.860034 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.039394821s to process 1 records I0704 18:16:15.860041 1 tasks_processing.go:74] worker 15 stopped. I0704 18:16:17.301015 1 tasks_processing.go:74] worker 28 stopped. I0704 18:16:17.301144 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=552096d2d3fd3b0a9bb5ab1474a92d914bc0b826311993ce969427819c7bccaa I0704 18:16:17.301194 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0704 18:16:17.301226 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=d060998b1be78c70904b5ff0e8c78110b98ee430861509804a3f34f9203b888d I0704 18:16:17.301238 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0704 18:16:17.301261 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=44c71df822ec6533c39b4fcbe9e278afcc7ba36cde2e45c3c8b84a25e411aa22 I0704 18:16:17.301277 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0704 18:16:17.301305 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=475a53091f5468657c8824126eea46531a6b7c889b6d94c841cee8aea8dddb89 I0704 18:16:17.301332 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=50aa308813fabc29195fedafa59f9891c53014776d2333e50f6f59543107ceca I0704 18:16:17.301356 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=355bb94179aba09b7fbb7daea8086d7be06e0c7447bcbf39d53625ce7778063d I0704 18:16:17.301392 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=8b88f1e34b032f34290ec278d30d5704fd6bc1fa5b8e3c4de2d30baeb8f869ca I0704 18:16:17.301669 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=7952e0d091ee1dfe4979fcb0a93781632dab4d5f326154bb2dbf0ab13dff1909 I0704 18:16:17.301696 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=87f0617ab07b732d3247e3f25d38957fde52ff03e709c401e4e1050a116a51b1 I0704 18:16:17.301706 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0704 18:16:17.301721 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=91e9cedd1bf770c37dada749517c1c1addc1b25bb837eaddbae541c836eb1f66 I0704 18:16:17.301747 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0704 18:16:17.301765 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=595fed4c18151c7b82ae3a693caaf348a0e5f3b26013cf92747b21ed159e95a9 I0704 18:16:17.301775 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0704 18:16:17.301795 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=667ee7fcf2d477cd53fcfc75261d75f0af84653714a0eb73903d0b9de253b730 I0704 18:16:17.301805 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0704 18:16:17.301833 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=c76bf0c0fdfde888bf353af66a8364cef1d1e0f80044483ee6331406fe9b075d I0704 18:16:17.301942 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=bd87daeef8ad543412a2056b78b084b4d06775200d216b52cd4de194b8abbf68 I0704 18:16:17.301953 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0704 18:16:17.301961 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0704 18:16:17.301988 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0704 18:16:17.302014 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=c6ebb0efe506e8a2bc0bd9d7b850e6b73bf4474c5b2cbe25506b05eb2a17a985 I0704 18:16:17.302038 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=456ac6a8b845bf5f153459225fbb330685be6477e3f8c09be3537243027f6ae2 I0704 18:16:17.302049 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0704 18:16:17.302068 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=efbbf5b3aac7b28473b30fc6506454fbccfe61f00cbf685af2c7f5753843bf8a I0704 18:16:17.302080 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0704 18:16:17.302099 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=8ebc6a522c3d5eb7fedc0f8d93bf02862920e55843e5922afaac3daee6898328 I0704 18:16:17.302108 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0704 18:16:17.302120 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=12290029e4ff3924275e8d5d41cfe510b6350d0aa1fb520289910bf9b51007f3 I0704 18:16:17.302143 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=c85ed591e7abfae85c66b035eea542331bdd756970312292b2986a4e59608b9f I0704 18:16:17.302159 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=9c7c3e8ea984282d6f72aed733f9d0a3a8802529382ed998053675b80cfd6f9b I0704 18:16:17.302187 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=39b74ce69c7d09f6c6b4f671b8653f757f7e937cfc0d8a1f258b7e09296301b5 I0704 18:16:17.302197 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0704 18:16:17.302233 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=3dace5e290c6093a925e7301f756fc6587b98e51f06bfa0bc99a3d409b2b1091 I0704 18:16:17.302251 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0704 18:16:17.302258 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0704 18:16:17.302266 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.466358164s to process 39 records I0704 18:16:27.873917 1 tasks_processing.go:74] worker 27 stopped. I0704 18:16:27.873959 1 recorder.go:75] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0704 18:16:27.873975 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.054026002s to process 1 records I0704 18:16:28.649534 1 tasks_processing.go:74] worker 10 stopped. I0704 18:16:28.649848 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=ae29e7659fb4cd0f04bcc181913d3f8b69c20a612f3ed6bf2f2479314a8cc58f I0704 18:16:28.649866 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.806132888s to process 1 records E0704 18:16:28.649927 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.83s with: function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error" I0704 18:16:28.651038 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error I0704 18:16:28.651055 1 periodic.go:209] Running workloads gatherer I0704 18:16:28.651076 1 tasks_processing.go:45] number of workers: 2 I0704 18:16:28.651090 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 18:16:28.651097 1 tasks_processing.go:71] worker 1 working on workload_info task. I0704 18:16:28.651103 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 18:16:28.651171 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0704 18:16:28.656029 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-178.ec2.internal... I0704 18:16:28.656042 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-39.ec2.internal... I0704 18:16:28.656052 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-42.ec2.internal... I0704 18:16:28.674507 1 tasks_processing.go:74] worker 0 stopped. I0704 18:16:28.674523 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 23.316257ms to process 0 records I0704 18:16:35.255890 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 7s I0704 18:16:35.282925 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0704 18:16:35.292103 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (11ms) I0704 18:16:35.299907 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0704 18:16:35.307523 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (8ms) I0704 18:16:35.314854 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (7ms) I0704 18:16:35.321789 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (7ms) I0704 18:16:35.328614 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (7ms) I0704 18:16:35.335638 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (7ms) I0704 18:16:35.342425 1 gather_workloads_info.go:387] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (7ms) I0704 18:16:35.349329 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (7ms) I0704 18:16:35.356446 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (7ms) I0704 18:16:35.391937 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (35ms) I0704 18:16:35.490967 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (99ms) I0704 18:16:35.591761 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (101ms) I0704 18:16:35.691254 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0704 18:16:35.791643 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0704 18:16:35.891525 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0704 18:16:35.990972 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (99ms) I0704 18:16:36.091683 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (101ms) I0704 18:16:36.191500 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0704 18:16:36.290598 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (99ms) I0704 18:16:36.390590 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0704 18:16:36.491319 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (101ms) I0704 18:16:36.592177 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (101ms) I0704 18:16:36.691691 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (99ms) I0704 18:16:36.792217 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0704 18:16:36.890992 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (99ms) I0704 18:16:36.991523 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0704 18:16:37.091537 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (100ms) I0704 18:16:37.191812 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0704 18:16:37.291132 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (99ms) I0704 18:16:37.391651 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (100ms) I0704 18:16:37.491321 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (100ms) I0704 18:16:37.591113 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0704 18:16:37.691669 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (101ms) I0704 18:16:37.791294 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0704 18:16:37.890096 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (99ms) I0704 18:16:37.992964 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (103ms) I0704 18:16:38.091135 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (98ms) I0704 18:16:38.191565 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0704 18:16:38.290529 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0704 18:16:38.391060 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (101ms) I0704 18:16:38.491575 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0704 18:16:38.591444 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (100ms) I0704 18:16:38.690565 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (99ms) I0704 18:16:38.792513 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (102ms) I0704 18:16:38.891236 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (99ms) I0704 18:16:38.992026 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (101ms) I0704 18:16:39.091367 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0704 18:16:39.191414 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0704 18:16:39.291200 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0704 18:16:39.291238 1 tasks_processing.go:74] worker 1 stopped. I0704 18:16:39.291537 1 recorder.go:75] Recording config/workload_info with fingerprint=704e7ba9c48c6ad381c704b0503580ea7183bace14f5634aa124cf8e121c391a I0704 18:16:39.291554 1 gather.go:177] gatherer "workloads" function "workload_info" took 10.640131997s to process 1 records I0704 18:16:39.291582 1 periodic.go:256] Periodic gather workloads completed in 10.64s I0704 18:16:39.291592 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0704 18:16:39.291598 1 periodic.go:209] Running conditional gatherer I0704 18:16:39.297855 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0704 18:16:39.544018 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0704 18:16:39.546540 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.81.232:9091: connect: connection refused I0704 18:16:39.546624 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0704 18:16:39.553350 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0704 18:16:39.553367 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553372 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553376 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553379 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553382 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553385 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553408 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553413 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553416 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0704 18:16:39.553420 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0704 18:16:39.553434 1 tasks_processing.go:45] number of workers: 3 I0704 18:16:39.553442 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 18:16:39.553446 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0704 18:16:39.553510 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 18:16:39.553524 1 tasks_processing.go:71] worker 1 working on remote_configuration task. I0704 18:16:39.553532 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 18:16:39.553533 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0704 18:16:39.553535 1 tasks_processing.go:74] worker 0 stopped. I0704 18:16:39.553528 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0704 18:16:39.553545 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.276µs to process 1 records I0704 18:16:39.553578 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0704 18:16:39.553616 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.242µs to process 1 records I0704 18:16:39.553624 1 tasks_processing.go:74] worker 1 stopped. I0704 18:16:39.553812 1 tasks_processing.go:74] worker 2 stopped. I0704 18:16:39.553824 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 264.961µs to process 0 records I0704 18:16:39.553840 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0704 18:16:39.553856 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0704 18:16:39.580656 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=f87e84ec85191a467b3dd69ce42fd068ea734423a8cd7f7c9a2a96f1ec54bdfa I0704 18:16:39.580847 1 diskrecorder.go:70] Writing 114 records to /var/lib/insights-operator/insights-2026-07-04-181639.tar.gz I0704 18:16:39.590051 1 diskrecorder.go:51] Wrote 114 records to disk in 9ms I0704 18:16:39.590106 1 periodic.go:278] Gathering cluster info every 2h0m0s I0704 18:16:39.590129 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0704 18:17:44.820754 1 diskrecorder.go:223] Found files to send: insights-2026-07-04-181639.tar.gz I0704 18:17:44.820785 1 insightsuploader.go:150] Checking archives to upload periodically every 16m19.345070307s I0704 18:17:44.820794 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0704 18:17:44.831093 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0704 18:17:44.994013 1 requests.go:87] Successfully reported id=2026-07-04T18:17:44Z x-rh-insights-request-id=b210b7d8e34f40ed9f8af1ff1b64b39c, wrote=55543 I0704 18:17:44.994132 1 insightsuploader.go:187] Uploaded report successfully in 173.331289ms I0704 18:17:44.994155 1 controller.go:128] Initializing last reported time to 2026-07-04T18:17:44Z I0704 18:17:44.994208 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0704 18:17:44.994224 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0704 18:17:44.994232 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0704 18:17:44.999438 1 controller.go:512] The operator is healthy I0704 18:18:14.843758 1 controller.go:512] The operator is healthy I0704 18:18:50.264034 1 insightsreport.go:137] Pulling report from smart-proxy I0704 18:18:50.264065 1 insightsreport.go:149] Retrieving report I0704 18:18:50.271344 1 requests.go:110] Retrieving report for cluster: feb53f94-da09-459e-9b02-5a19cc5fc927 I0704 18:18:50.271361 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/feb53f94-da09-459e-9b02-5a19cc5fc927/reports I0704 18:18:50.274110 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/feb53f94-da09-459e-9b02-5a19cc5fc927/reports I0704 18:18:50.492371 1 insightsreport.go:184] Report retrieved I0704 18:18:50.533453 1 insightsreport.go:239] Report retrieved correctly I0704 18:20:14.843435 1 controller.go:512] The operator is healthy I0704 18:21:14.812842 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 18:21:14.818647 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:21:14.818666 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 18:21:14.822171 1 secretconfigobserver.go:119] support secret does not exist I0704 18:22:14.843464 1 controller.go:512] The operator is healthy I0704 18:24:14.843144 1 controller.go:512] The operator is healthy I0704 18:26:14.812702 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:26:14.822744 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 18:26:14.826997 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:26:14.827018 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 18:26:14.830770 1 secretconfigobserver.go:119] support secret does not exist I0704 18:26:14.842074 1 controller.go:512] The operator is healthy I0704 18:28:14.842783 1 controller.go:512] The operator is healthy I0704 18:30:14.843422 1 controller.go:512] The operator is healthy I0704 18:31:14.831591 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 18:31:14.835753 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:31:14.835772 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 18:31:14.839329 1 secretconfigobserver.go:119] support secret does not exist I0704 18:32:14.843461 1 controller.go:512] The operator is healthy I0704 18:34:09.339901 1 insightsuploader.go:146] Nothing to report since 2026-07-04T18:17:44Z I0704 18:34:14.845093 1 controller.go:512] The operator is healthy I0704 18:36:14.812955 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:36:14.816815 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 18:36:14.840258 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 18:36:14.842152 1 controller.go:512] The operator is healthy I0704 18:36:14.844203 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 18:36:14.844219 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 18:36:14.847633 1 secretconfigobserver.go:119] support secret does not exist