I0417 18:41:20.349803 1 cmd.go:253] Using service-serving-cert provided certificates I0417 18:41:20.350310 1 observer_polling.go:159] Starting file observer I0417 18:41:20.797134 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0417 18:41:20.797332 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0417 18:41:20.797888 1 secure_serving.go:57] Forcing use of http/1.1 only W0417 18:41:20.797909 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0417 18:41:20.797914 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0417 18:41:20.797918 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0417 18:41:20.797921 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0417 18:41:20.797924 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0417 18:41:20.797927 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0417 18:41:20.798132 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0417 18:41:20.803187 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0417 18:41:20.803255 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"e75b09bf-ee33-4760-a8d8-075997b8af25", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0417 18:41:20.803644 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0417 18:41:20.803657 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0417 18:41:20.803684 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0417 18:41:20.803717 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0417 18:41:20.803714 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0417 18:41:20.803727 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0417 18:41:20.804792 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" I0417 18:41:20.804791 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0417 18:41:20.804776 1 secure_serving.go:213] Serving securely on [::]:8443 W0417 18:41:20.813331 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0417 18:41:20.813363 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0417 18:41:20.813471 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0417 18:41:20.818501 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0417 18:41:20.818519 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0417 18:41:20.823216 1 secretconfigobserver.go:119] support secret does not exist I0417 18:41:20.828006 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0417 18:41:20.832202 1 secretconfigobserver.go:119] support secret does not exist I0417 18:41:20.834090 1 recorder.go:176] Pruning old reports every 7h28m56s, max age is 288h0m0s I0417 18:41:20.838873 1 periodic.go:216] Running clusterconfig gatherer I0417 18:41:20.838943 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0417 18:41:20.838958 1 insightsreport.go:296] Starting report retriever I0417 18:41:20.838966 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0417 18:41:20.838945 1 tasks_processing.go:45] number of workers: 64 I0417 18:41:20.839017 1 tasks_processing.go:69] worker 63 listening for tasks. I0417 18:41:20.839024 1 tasks_processing.go:71] worker 63 working on mutating_webhook_configurations task. I0417 18:41:20.839048 1 tasks_processing.go:69] worker 14 listening for tasks. I0417 18:41:20.839055 1 tasks_processing.go:69] worker 6 listening for tasks. I0417 18:41:20.839065 1 tasks_processing.go:71] worker 6 working on networks task. I0417 18:41:20.839070 1 tasks_processing.go:69] worker 17 listening for tasks. I0417 18:41:20.839070 1 tasks_processing.go:69] worker 8 listening for tasks. I0417 18:41:20.839075 1 tasks_processing.go:69] worker 2 listening for tasks. I0417 18:41:20.839082 1 tasks_processing.go:69] worker 18 listening for tasks. I0417 18:41:20.839057 1 tasks_processing.go:69] worker 15 listening for tasks. I0417 18:41:20.839065 1 tasks_processing.go:69] worker 7 listening for tasks. I0417 18:41:20.839078 1 tasks_processing.go:69] worker 9 listening for tasks. I0417 18:41:20.839089 1 tasks_processing.go:69] worker 0 listening for tasks. I0417 18:41:20.839083 1 tasks_processing.go:69] worker 10 listening for tasks. I0417 18:41:20.839046 1 tasks_processing.go:69] worker 30 listening for tasks. I0417 18:41:20.839094 1 tasks_processing.go:69] worker 46 listening for tasks. I0417 18:41:20.839103 1 tasks_processing.go:69] worker 1 listening for tasks. I0417 18:41:20.838874 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0417 18:41:20.839110 1 tasks_processing.go:69] worker 38 listening for tasks. I0417 18:41:20.839113 1 tasks_processing.go:69] worker 26 listening for tasks. I0417 18:41:20.839117 1 tasks_processing.go:69] worker 31 listening for tasks. I0417 18:41:20.839115 1 tasks_processing.go:69] worker 25 listening for tasks. I0417 18:41:20.839088 1 tasks_processing.go:69] worker 11 listening for tasks. I0417 18:41:20.839123 1 tasks_processing.go:69] worker 22 listening for tasks. I0417 18:41:20.839120 1 tasks_processing.go:69] worker 13 listening for tasks. I0417 18:41:20.839132 1 tasks_processing.go:69] worker 23 listening for tasks. I0417 18:41:20.839099 1 tasks_processing.go:69] worker 20 listening for tasks. I0417 18:41:20.839138 1 tasks_processing.go:69] worker 37 listening for tasks. I0417 18:41:20.839106 1 tasks_processing.go:69] worker 21 listening for tasks. I0417 18:41:20.839141 1 tasks_processing.go:69] worker 24 listening for tasks. I0417 18:41:20.839143 1 tasks_processing.go:69] worker 29 listening for tasks. I0417 18:41:20.839094 1 tasks_processing.go:69] worker 12 listening for tasks. I0417 18:41:20.839146 1 tasks_processing.go:69] worker 45 listening for tasks. I0417 18:41:20.839150 1 tasks_processing.go:69] worker 40 listening for tasks. I0417 18:41:20.839116 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0417 18:41:20.839156 1 tasks_processing.go:69] worker 48 listening for tasks. I0417 18:41:20.839157 1 tasks_processing.go:69] worker 35 listening for tasks. I0417 18:41:20.839159 1 tasks_processing.go:69] worker 5 listening for tasks. I0417 18:41:20.839166 1 tasks_processing.go:69] worker 53 listening for tasks. I0417 18:41:20.839170 1 tasks_processing.go:69] worker 58 listening for tasks. I0417 18:41:20.839095 1 tasks_processing.go:69] worker 19 listening for tasks. I0417 18:41:20.839173 1 tasks_processing.go:69] worker 60 listening for tasks. I0417 18:41:20.839059 1 tasks_processing.go:71] worker 14 working on sap_datahubs task. I0417 18:41:20.839180 1 tasks_processing.go:71] worker 15 working on metrics task. I0417 18:41:20.839180 1 tasks_processing.go:69] worker 55 listening for tasks. I0417 18:41:20.839183 1 tasks_processing.go:69] worker 61 listening for tasks. I0417 18:41:20.839186 1 tasks_processing.go:71] worker 17 working on schedulers task. I0417 18:41:20.839186 1 tasks_processing.go:71] worker 2 working on install_plans task. I0417 18:41:20.839184 1 tasks_processing.go:69] worker 33 listening for tasks. I0417 18:41:20.839194 1 tasks_processing.go:71] worker 61 working on crds task. I0417 18:41:20.839197 1 tasks_processing.go:69] worker 62 listening for tasks. I0417 18:41:20.839192 1 tasks_processing.go:71] worker 1 working on nodes task. I0417 18:41:20.839202 1 tasks_processing.go:71] worker 33 working on image task. I0417 18:41:20.839202 1 tasks_processing.go:71] worker 18 working on operators task. I0417 18:41:20.839210 1 tasks_processing.go:71] worker 25 working on validating_webhook_configurations task. I0417 18:41:20.839212 1 tasks_processing.go:71] worker 30 working on qemu_kubevirt_launcher_logs task. I0417 18:41:20.839065 1 tasks_processing.go:69] worker 16 listening for tasks. I0417 18:41:20.839220 1 tasks_processing.go:71] worker 0 working on storage_cluster task. I0417 18:41:20.839241 1 tasks_processing.go:71] worker 7 working on nodenetworkconfigurationpolicies task. I0417 18:41:20.839264 1 tasks_processing.go:71] worker 20 working on openshift_logging task. I0417 18:41:20.839267 1 tasks_processing.go:71] worker 48 working on machine_healthchecks task. I0417 18:41:20.839266 1 tasks_processing.go:71] worker 24 working on node_features task. I0417 18:41:20.839269 1 tasks_processing.go:71] worker 22 working on authentication task. I0417 18:41:20.839272 1 tasks_processing.go:71] worker 13 working on oauths task. I0417 18:41:20.839276 1 tasks_processing.go:71] worker 16 working on pdbs task. I0417 18:41:20.839128 1 tasks_processing.go:69] worker 32 listening for tasks. I0417 18:41:20.839285 1 tasks_processing.go:71] worker 62 working on image_pruners task. I0417 18:41:20.839292 1 tasks_processing.go:71] worker 32 working on tsdb_status task. I0417 18:41:20.839265 1 tasks_processing.go:71] worker 11 working on machines task. I0417 18:41:20.839127 1 tasks_processing.go:69] worker 43 listening for tasks. I0417 18:41:20.839364 1 tasks_processing.go:71] worker 43 working on config_maps task. I0417 18:41:20.839133 1 tasks_processing.go:69] worker 36 listening for tasks. I0417 18:41:20.839126 1 tasks_processing.go:69] worker 27 listening for tasks. I0417 18:41:20.839131 1 tasks_processing.go:69] worker 42 listening for tasks. I0417 18:41:20.839133 1 tasks_processing.go:69] worker 44 listening for tasks. I0417 18:41:20.839134 1 tasks_processing.go:69] worker 28 listening for tasks. I0417 18:41:20.839143 1 tasks_processing.go:69] worker 39 listening for tasks. I0417 18:41:20.839146 1 tasks_processing.go:69] worker 54 listening for tasks. I0417 18:41:20.839150 1 tasks_processing.go:69] worker 47 listening for tasks. I0417 18:41:20.839154 1 tasks_processing.go:69] worker 50 listening for tasks. I0417 18:41:20.839480 1 tasks_processing.go:71] worker 50 working on container_runtime_configs task. I0417 18:41:20.839486 1 tasks_processing.go:71] worker 28 working on olm_operators task. I0417 18:41:20.839494 1 tasks_processing.go:71] worker 36 working on openshift_machine_api_events task. I0417 18:41:20.839154 1 tasks_processing.go:69] worker 3 listening for tasks. I0417 18:41:20.839155 1 tasks_processing.go:69] worker 51 listening for tasks. I0417 18:41:20.839162 1 tasks_processing.go:69] worker 52 listening for tasks. I0417 18:41:20.839161 1 tasks_processing.go:69] worker 41 listening for tasks. I0417 18:41:20.839162 1 tasks_processing.go:69] worker 4 listening for tasks. I0417 18:41:20.839162 1 tasks_processing.go:69] worker 49 listening for tasks. I0417 18:41:20.839172 1 tasks_processing.go:69] worker 56 listening for tasks. I0417 18:41:20.839173 1 tasks_processing.go:69] worker 34 listening for tasks. I0417 18:41:20.839545 1 tasks_processing.go:71] worker 34 working on sap_pods task. I0417 18:41:20.840218 1 tasks_processing.go:71] worker 56 working on cost_management_metrics_configs task. I0417 18:41:20.839480 1 tasks_processing.go:71] worker 47 working on cluster_apiserver task. I0417 18:41:20.839487 1 tasks_processing.go:71] worker 44 working on openstack_controlplanes task. I0417 18:41:20.840261 1 tasks_processing.go:71] worker 54 working on certificate_signing_requests task. I0417 18:41:20.839179 1 tasks_processing.go:69] worker 57 listening for tasks. I0417 18:41:20.839181 1 tasks_processing.go:71] worker 8 working on overlapping_namespace_uids task. I0417 18:41:20.841917 1 tasks_processing.go:71] worker 57 working on monitoring_persistent_volumes task. I0417 18:41:20.839187 1 tasks_processing.go:71] worker 60 working on machine_config_pools task. I0417 18:41:20.839189 1 tasks_processing.go:71] worker 55 working on service_accounts task. I0417 18:41:20.839192 1 tasks_processing.go:69] worker 59 listening for tasks. I0417 18:41:20.839197 1 tasks_processing.go:71] worker 38 working on openstack_version task. I0417 18:41:20.842045 1 tasks_processing.go:71] worker 59 working on version task. I0417 18:41:20.839198 1 tasks_processing.go:71] worker 46 working on container_images task. I0417 18:41:20.839202 1 tasks_processing.go:71] worker 26 working on operators_pods_and_events task. I0417 18:41:20.839207 1 tasks_processing.go:71] worker 31 working on silenced_alerts task. I0417 18:41:20.839207 1 tasks_processing.go:71] worker 10 working on dvo_metrics task. I0417 18:41:20.839249 1 tasks_processing.go:71] worker 35 working on ingress task. I0417 18:41:20.839254 1 tasks_processing.go:71] worker 12 working on ingress_certificates task. I0417 18:41:20.839255 1 tasks_processing.go:71] worker 53 working on machine_sets task. I0417 18:41:20.839253 1 tasks_processing.go:71] worker 9 working on proxies task. I0417 18:41:20.839258 1 tasks_processing.go:71] worker 45 working on node_logs task. I0417 18:41:20.839258 1 tasks_processing.go:71] worker 58 working on machine_autoscalers task. I0417 18:41:20.839258 1 tasks_processing.go:71] worker 19 working on machine_configs task. I0417 18:41:20.839260 1 tasks_processing.go:71] worker 23 working on lokistack task. I0417 18:41:20.839253 1 tasks_processing.go:71] worker 29 working on sap_config task. I0417 18:41:20.839259 1 tasks_processing.go:71] worker 5 working on ceph_cluster task. I0417 18:41:20.839262 1 tasks_processing.go:71] worker 40 working on feature_gates task. I0417 18:41:20.839265 1 tasks_processing.go:71] worker 37 working on openstack_dataplanedeployments task. I0417 18:41:20.839266 1 tasks_processing.go:71] worker 21 working on active_alerts task. I0417 18:41:20.839533 1 tasks_processing.go:71] worker 39 working on pod_network_connectivity_checks task. I0417 18:41:20.840385 1 tasks_processing.go:71] worker 3 working on support_secret task. I0417 18:41:20.840389 1 tasks_processing.go:71] worker 27 working on image_registries task. I0417 18:41:20.840392 1 tasks_processing.go:71] worker 51 working on aggregated_monitoring_cr_names task. I0417 18:41:20.840499 1 tasks_processing.go:71] worker 52 working on infrastructures task. I0417 18:41:20.840512 1 tasks_processing.go:71] worker 4 working on jaegers task. I0417 18:41:20.840588 1 tasks_processing.go:71] worker 42 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0417 18:41:20.840593 1 tasks_processing.go:71] worker 49 working on storage_classes task. I0417 18:41:20.840642 1 tasks_processing.go:71] worker 41 working on nodenetworkstates task. I0417 18:41:20.844321 1 tasks_processing.go:71] worker 20 working on clusterroles task. I0417 18:41:20.844327 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 5.045747ms to process 0 records I0417 18:41:20.844669 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0417 18:41:20.844708 1 controller.go:368] The initial operator extension status is healthy I0417 18:41:20.844718 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0417 18:41:20.844722 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0417 18:41:20.844725 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0417 18:41:20.844740 1 controller.go:531] The operator is still being initialized I0417 18:41:20.844749 1 controller.go:554] The operator is healthy I0417 18:41:20.844838 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0417 18:41:20.844845 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s W0417 18:41:20.844984 1 operator.go:328] started I0417 18:41:20.845011 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0417 18:41:20.845781 1 tasks_processing.go:71] worker 6 working on openstack_dataplanenodesets task. I0417 18:41:20.845976 1 recorder.go:75] Recording config/network with fingerprint=f7cab76bef68415e5b676d721fc396bd416db5b2c6f74ad06225bc69bb585a5b I0417 18:41:20.845993 1 gather.go:177] gatherer "clusterconfig" function "networks" took 6.703894ms to process 1 records I0417 18:41:20.846004 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 6.460773ms to process 0 records I0417 18:41:20.846011 1 tasks_processing.go:74] worker 50 stopped. E0417 18:41:20.846082 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.153.210:9091: connect: connection refused E0417 18:41:20.846101 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.153.210:9091: connect: connection refused E0417 18:41:20.846103 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.38.253:9094: connect: connection refused I0417 18:41:20.846113 1 tasks_processing.go:74] worker 15 stopped. E0417 18:41:20.846120 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.153.210:9091: connect: connection refused I0417 18:41:20.846139 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.918232ms to process 0 records E0417 18:41:20.846147 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.38.253:9094: connect: connection refused E0417 18:41:20.846151 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.153.210:9091: connect: connection refused I0417 18:41:20.846161 1 tasks_processing.go:74] worker 32 stopped. I0417 18:41:20.846168 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 6.81211ms to process 0 records E0417 18:41:20.846183 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.38.253:9094: connect: connection refused I0417 18:41:20.846193 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 4.007208ms to process 0 records E0417 18:41:20.846200 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.38.253:9094: connect: connection refused I0417 18:41:20.846209 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 3.94886ms to process 0 records I0417 18:41:20.846217 1 tasks_processing.go:74] worker 21 stopped. I0417 18:41:20.846221 1 tasks_processing.go:74] worker 31 stopped. I0417 18:41:20.846482 1 tasks_processing.go:74] worker 7 stopped. I0417 18:41:20.846494 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 7.210528ms to process 0 records E0417 18:41:20.846510 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0417 18:41:20.846522 1 tasks_processing.go:74] worker 24 stopped. I0417 18:41:20.846528 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 7.243078ms to process 0 records I0417 18:41:20.846629 1 tasks_processing.go:74] worker 11 stopped. E0417 18:41:20.846640 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0417 18:41:20.846647 1 gather.go:177] gatherer "clusterconfig" function "machines" took 7.315828ms to process 0 records I0417 18:41:20.846978 1 tasks_processing.go:74] worker 63 stopped. I0417 18:41:20.847167 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=7a33fcdc8855329af737665d2a1af03b6e5f0d93899e391b2f58990809b5d937 I0417 18:41:20.847204 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=59592a77b455fb8d91c098d6adfde6e0c34eef3026395dea638a6c52cfc07f67 I0417 18:41:20.847233 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=2894dad8365d83ed5c046e00f193c2d76fe77b67efd49fd0c79917dfa9e4e7cf I0417 18:41:20.847241 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 7.946257ms to process 3 records I0417 18:41:20.847252 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 6.691297ms to process 0 records I0417 18:41:20.847258 1 tasks_processing.go:74] worker 56 stopped. I0417 18:41:20.847327 1 tasks_processing.go:74] worker 62 stopped. I0417 18:41:20.847437 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=a61238d2a0f5718cd4bac0dbb390b10256e0bb2e6cd97360bc13544629086a45 I0417 18:41:20.847447 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 7.960078ms to process 1 records I0417 18:41:20.847455 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 8.189607ms to process 0 records I0417 18:41:20.847461 1 tasks_processing.go:74] worker 0 stopped. I0417 18:41:20.860577 1 tasks_processing.go:74] worker 34 stopped. I0417 18:41:20.860596 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 20.471467ms to process 0 records I0417 18:41:20.864125 1 tasks_processing.go:74] worker 14 stopped. I0417 18:41:20.864139 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 24.937063ms to process 0 records I0417 18:41:20.864669 1 tasks_processing.go:74] worker 60 stopped. I0417 18:41:20.864703 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 22.731963ms to process 0 records I0417 18:41:20.864758 1 tasks_processing.go:74] worker 53 stopped. I0417 18:41:20.864774 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 22.578255ms to process 0 records E0417 18:41:20.864791 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0417 18:41:20.864811 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 25.490635ms to process 0 records I0417 18:41:20.864826 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 23.518215ms to process 0 records I0417 18:41:20.864832 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 21.952482ms to process 0 records I0417 18:41:20.864837 1 tasks_processing.go:74] worker 41 stopped. I0417 18:41:20.864841 1 tasks_processing.go:74] worker 48 stopped. I0417 18:41:20.864844 1 tasks_processing.go:74] worker 44 stopped. I0417 18:41:20.864868 1 tasks_processing.go:74] worker 13 stopped. I0417 18:41:20.865105 1 recorder.go:75] Recording config/oauth with fingerprint=a6da43fa9d0cf8837ba934f547379cf2a939fccf48d1f4e83e0b7f8eb1029407 I0417 18:41:20.865122 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 25.583781ms to process 1 records I0417 18:41:20.865219 1 tasks_processing.go:74] worker 22 stopped. I0417 18:41:20.865278 1 recorder.go:75] Recording config/authentication with fingerprint=d5cd6555fd2733efe51d78806d46b54c585878868c8375acda8f08d72b93c090 I0417 18:41:20.865286 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 25.628975ms to process 1 records I0417 18:41:20.865407 1 recorder.go:75] Recording config/image with fingerprint=5123899f2e30e46490857fda1ed519e93d28fb7d3ee9a0277f882f503809abdb I0417 18:41:20.865417 1 gather.go:177] gatherer "clusterconfig" function "image" took 25.696669ms to process 1 records I0417 18:41:20.865421 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 22.750168ms to process 0 records I0417 18:41:20.865451 1 tasks_processing.go:74] worker 33 stopped. I0417 18:41:20.865470 1 tasks_processing.go:74] worker 58 stopped. I0417 18:41:20.865482 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=304b2d255b14c922f8b81d6c8299c502a296c98a8cac40bb7e7be6c6ec99de15 I0417 18:41:20.865492 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 25.913654ms to process 1 records I0417 18:41:20.865498 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 25.656655ms to process 0 records I0417 18:41:20.865531 1 tasks_processing.go:74] worker 17 stopped. I0417 18:41:20.865545 1 tasks_processing.go:74] worker 28 stopped. I0417 18:41:20.865623 1 tasks_processing.go:74] worker 35 stopped. I0417 18:41:20.865934 1 recorder.go:75] Recording config/ingress with fingerprint=42eac93580f8ab782e6dd4e78bd2e3842c29a694315eba4e8826f2742cd9d2ba I0417 18:41:20.865958 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 23.326884ms to process 1 records I0417 18:41:20.866048 1 tasks_processing.go:74] worker 47 stopped. I0417 18:41:20.866603 1 recorder.go:75] Recording config/apiserver with fingerprint=b838c87152dce835fe05e3e6b32e1ba210baaee9ae50fe4d9d41b7354add132f I0417 18:41:20.866669 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 24.72537ms to process 1 records I0417 18:41:20.866733 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 23.648823ms to process 0 records I0417 18:41:20.866813 1 tasks_processing.go:74] worker 57 stopped. I0417 18:41:20.866945 1 tasks_processing.go:74] worker 49 stopped. I0417 18:41:20.866954 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=92f4f0dafdd7ce61522ca34f73d1b716464c288a33880eaa61997381ac21b953 I0417 18:41:20.867002 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=bd159552668aa27859e2462858d8a357765c29b1b906f9a7ed90a9d4e2d1b3ba I0417 18:41:20.867014 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 23.517902ms to process 2 records I0417 18:41:20.867666 1 gather_logs.go:145] no pods in namespace were found I0417 18:41:20.867680 1 tasks_processing.go:74] worker 30 stopped. I0417 18:41:20.867702 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 28.458013ms to process 0 records I0417 18:41:20.868292 1 tasks_processing.go:74] worker 9 stopped. I0417 18:41:20.868365 1 recorder.go:75] Recording config/proxy with fingerprint=00ecfbcf865f87bbe1080f6577e560d3d93f702ab6ac92736f89583d6bd9c961 I0417 18:41:20.868379 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 26.111529ms to process 1 records I0417 18:41:20.868385 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 26.134976ms to process 0 records I0417 18:41:20.868390 1 tasks_processing.go:74] worker 23 stopped. I0417 18:41:20.868617 1 tasks_processing.go:74] worker 52 stopped. I0417 18:41:20.869172 1 recorder.go:75] Recording config/infrastructure with fingerprint=4c4a97e679bb44cff7a8a9273c4b2743f909567f629c7e1961297bb6d8bd8c72 I0417 18:41:20.869198 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 26.336988ms to process 1 records I0417 18:41:20.869354 1 tasks_processing.go:74] worker 16 stopped. I0417 18:41:20.869455 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=4ed6cde8d95348ce88542db5705e0fa25cd2b36a470a484493edb68335f24958 I0417 18:41:20.869474 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=86fb48a28f961b3d1ce862971e44f4ab6de5d6b42cb0b118a42a9daa92955a0c I0417 18:41:20.869487 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=b543df654dbd06111a8bcf4439da14f3278569e69d424c9731a546e0b963acd0 I0417 18:41:20.869501 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=01a2ea78e6abcd520ebbc87766fc53f10d498124aa48608b1eda3fdd1ac0fa5f I0417 18:41:20.869521 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=511e0823dea4fd1f6d486e25a542ac6a67f34fd8d426d38d9583694a05288ea2 I0417 18:41:20.869536 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=a22634b4124a8a98f945af327eb8088661856bd957d90ba62d0d6703095b4209 I0417 18:41:20.869549 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=f06b6a598e4b7f3f582c4435820fe8d26ba1bb5a4ddb6b78a9029b948cdb2a49 I0417 18:41:20.869566 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=1aeb32eb99995d613281ab630735c41a42f610155dd0165d28d48758ff29f3f9 I0417 18:41:20.869592 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=984f10b5ff17cffedf9854f47ece6b7a11775eb6539df4e85494e08dae3192cd I0417 18:41:20.869599 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 30.061361ms to process 9 records I0417 18:41:20.871814 1 tasks_processing.go:74] worker 29 stopped. I0417 18:41:20.871831 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 29.616913ms to process 0 records I0417 18:41:20.872070 1 tasks_processing.go:74] worker 25 stopped. I0417 18:41:20.872232 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=66547b5445e8bd9632b81167b013d692b1ea812bfa31e1a9432b01d66ebdb484 I0417 18:41:20.872340 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=ef2efa4a423f312ba9a87e4062e31c31a2760dc3411228ef9fd92c27c939c687 I0417 18:41:20.872474 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=e53687caaecb31289e3072e33950eac2b92620fedb58fb291415a1a35f929850 I0417 18:41:20.872517 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=90562a612c18e9667551e2869aaa58e6da15cebc2a514f1ce2fc8065bf86674a I0417 18:41:20.872554 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=167b3a6440d4727239c434cf1b9ff0d047d0857195f1dcc78c26006b22adcacb I0417 18:41:20.872601 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=9fc48824d016c16374b3953c1a17fe252e10175c4a8eb216121852a256316ce3 I0417 18:41:20.872639 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=6a3f6b7b135538f842f4d136e5368d293d055a4f8913f2a50ee818e11d7b7965 I0417 18:41:20.872678 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=cb2ebd933d6b9b156e4164b5d866b49c191ca02bfe203103ed73377f410ec4d8 I0417 18:41:20.872742 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=461aefa61135fa07149ee65f81643809adb462db14ee90e5f020127413298d09 I0417 18:41:20.872803 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=d680a653018f7cb7faaf9ed9a15cfd45f8a510e84167ec0faf1b7e5c341973ba I0417 18:41:20.872840 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=22336274a4c5e7adfd4f26620b482d089c81fbd5a76fed7927eb7811bb42d083 I0417 18:41:20.872880 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=43b5db9243eca20749117b3c2151ce4467566f14d6b89fa3b7db4732d1428a72 I0417 18:41:20.872918 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=168afbb43035d699ea4ffb3ecd5d92e86bf69ece45ced7e61a19e8cc2ba18911 I0417 18:41:20.872931 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 32.84965ms to process 13 records I0417 18:41:20.872946 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 32.630145ms to process 0 records I0417 18:41:20.872964 1 tasks_processing.go:74] worker 36 stopped. I0417 18:41:20.873045 1 tasks_processing.go:74] worker 1 stopped. I0417 18:41:20.873326 1 recorder.go:75] Recording config/node/ip-10-0-0-216.ec2.internal with fingerprint=a76758ecc08bf929be03b38b7fc72bdd08769f701218166a9a5f219c98289c6c I0417 18:41:20.873418 1 recorder.go:75] Recording config/node/ip-10-0-1-68.ec2.internal with fingerprint=55e6166ce0255adbcf9ea71a7c292e258f6bda9296bdedd6391e3c00c4f9baff I0417 18:41:20.873509 1 recorder.go:75] Recording config/node/ip-10-0-2-177.ec2.internal with fingerprint=233275e751c5cdb21415d2c6b47e75d6b74a2e9576ee87d16a3afdd54373844a I0417 18:41:20.873519 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 33.331227ms to process 3 records I0417 18:41:20.873983 1 tasks_processing.go:74] worker 37 stopped. I0417 18:41:20.874108 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 31.767914ms to process 0 records I0417 18:41:20.874266 1 tasks_processing.go:74] worker 5 stopped. I0417 18:41:20.874321 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 32.064117ms to process 0 records I0417 18:41:20.874379 1 tasks_processing.go:74] worker 38 stopped. I0417 18:41:20.874412 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 32.331919ms to process 0 records I0417 18:41:20.874511 1 tasks_processing.go:74] worker 39 stopped. E0417 18:41:20.874604 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0417 18:41:20.874637 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 32.289869ms to process 0 records W0417 18:41:20.874998 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0417 18:41:20.875112 1 tasks_processing.go:74] worker 6 stopped. I0417 18:41:20.875137 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 29.314483ms to process 0 records E0417 18:41:20.876178 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0417 18:41:20.876324 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 33.821179ms to process 0 records I0417 18:41:20.876323 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0417 18:41:20.876401 1 tasks_processing.go:74] worker 45 stopped. I0417 18:41:20.876379 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 33.912382ms to process 0 records I0417 18:41:20.876421 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 33.787461ms to process 0 records I0417 18:41:20.876430 1 tasks_processing.go:74] worker 4 stopped. I0417 18:41:20.876397 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0417 18:41:20.876502 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0417 18:41:20.876510 1 controller.go:254] Source scaController *sca.Controller is not ready I0417 18:41:20.876517 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0417 18:41:20.876327 1 tasks_processing.go:74] worker 3 stopped. I0417 18:41:20.876789 1 controller.go:531] The operator is still being initialized I0417 18:41:20.876853 1 controller.go:554] The operator is healthy I0417 18:41:20.878748 1 tasks_processing.go:74] worker 19 stopped. E0417 18:41:20.878765 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0417 18:41:20.878801 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0417 18:41:20.878820 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 36.557602ms to process 1 records I0417 18:41:20.878974 1 tasks_processing.go:74] worker 54 stopped. I0417 18:41:20.878996 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 37.260075ms to process 0 records I0417 18:41:20.879193 1 tasks_processing.go:74] worker 40 stopped. I0417 18:41:20.879373 1 recorder.go:75] Recording config/featuregate with fingerprint=9ffb4ef954125996f84aab590b24b84df463c42908c9ea774bbd5ffeb1222a32 I0417 18:41:20.879397 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 36.984678ms to process 1 records I0417 18:41:20.879421 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0417 18:41:20.879434 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 37.418826ms to process 1 records I0417 18:41:20.879445 1 tasks_processing.go:74] worker 8 stopped. I0417 18:41:20.879792 1 tasks_processing.go:74] worker 27 stopped. I0417 18:41:20.880276 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=0bce1ad4e7430718d8c94993646bbbc6c87356a8c488780d8782d81a5e51da32 I0417 18:41:20.880298 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 37.544956ms to process 1 records I0417 18:41:20.880794 1 tasks_processing.go:74] worker 59 stopped. I0417 18:41:20.881032 1 recorder.go:75] Recording config/version with fingerprint=3f99de0e148256fd7ce51a679a0c48c333baa2f133bf67238774844041fd5cac I0417 18:41:20.881048 1 recorder.go:75] Recording config/id with fingerprint=3043f62a22c6cc9490fce4299d1a2789917b1e05b3d14f84a9eaf5bcc6112002 I0417 18:41:20.881055 1 gather.go:177] gatherer "clusterconfig" function "version" took 38.732022ms to process 2 records I0417 18:41:20.897872 1 tasks_processing.go:74] worker 20 stopped. I0417 18:41:20.898093 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=55611738c613337e22d434db5537d0e09a5954146c6bcff48399df519112b9af I0417 18:41:20.898187 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=d1e6840a3191be71c1a85bbe8142fa72aba51e616f22ff68c084e3a03ea3d585 I0417 18:41:20.898196 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 53.530358ms to process 2 records I0417 18:41:20.898312 1 tasks_processing.go:74] worker 61 stopped. I0417 18:41:20.898957 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=61c107934b59e69f9bfa8230e124436f4751674c08af690c8bc4ac0451e46d08 I0417 18:41:20.899245 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=cc2e8b4f3b0226088d0baa1cda677e1b5a99ba430e83ad14dbea27bb0bc2e5fb I0417 18:41:20.899255 1 gather.go:177] gatherer "clusterconfig" function "crds" took 59.101075ms to process 2 records I0417 18:41:20.899336 1 tasks_processing.go:74] worker 46 stopped. I0417 18:41:20.900626 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf with fingerprint=87f34cd89658e4b7fd915a56393d3b142fb3cc08359c9e8150cd28c231df5472 I0417 18:41:20.900740 1 recorder.go:75] Recording config/running_containers with fingerprint=25e2e7331e0f62d07a9f73e6f256c980e26a3bbaca12d928af3d1a40e7a09279 I0417 18:41:20.900752 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 57.166497ms to process 2 records I0417 18:41:20.904142 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0417 18:41:20.904163 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0417 18:41:20.904185 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0417 18:41:20.904471 1 tasks_processing.go:74] worker 42 stopped. I0417 18:41:20.904510 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 62.065814ms to process 0 records I0417 18:41:20.905261 1 tasks_processing.go:74] worker 51 stopped. I0417 18:41:20.905280 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 62.991631ms to process 0 records I0417 18:41:20.910266 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0417 18:41:20.912786 1 tasks_processing.go:74] worker 43 stopped. E0417 18:41:20.912801 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0417 18:41:20.912807 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0417 18:41:20.912810 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0417 18:41:20.912820 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=ad286723f58bdcfc37aeba3ec5b4110c08e1af59cd34d14b4bfaab02d18e4856 I0417 18:41:20.912856 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0417 18:41:20.912864 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0417 18:41:20.912869 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=2ab5ab7b1b10d7fcf1197bb24dea7c90f400e4effc18c7356873209d54fdf84b I0417 18:41:20.912879 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0417 18:41:20.912951 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0417 18:41:20.912960 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0417 18:41:20.912965 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 73.399954ms to process 7 records I0417 18:41:20.913705 1 base_controller.go:82] Caches are synced for ConfigController I0417 18:41:20.913717 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0417 18:41:20.918380 1 configmapobserver.go:84] configmaps "insights-config" not found I0417 18:41:20.922455 1 tasks_processing.go:74] worker 12 stopped. E0417 18:41:20.922470 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0417 18:41:20.922476 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2po2q8b3ubd89mbta5n7l49aiaaqqcse-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2po2q8b3ubd89mbta5n7l49aiaaqqcse-primary-cert-bundle-secret" not found I0417 18:41:20.922544 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=2da06e4854ca1c17c339150b7390b2c8e422519ca0187f139505caeeddd71bab I0417 18:41:20.922556 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 80.284261ms to process 1 records I0417 18:41:20.945186 1 base_controller.go:82] Caches are synced for LoggingSyncer I0417 18:41:20.945202 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0417 18:41:20.950316 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0417 18:41:20.954900 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0417 18:41:20.973455 1 gather_cluster_operator_pods_and_events.go:121] Found 30 pods with 77 containers I0417 18:41:20.973467 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 326828 bytes I0417 18:41:20.973523 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:20.984414 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:20.997658 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:21.075714 1 cluster_transfer.go:102] no available accepted cluster transfer I0417 18:41:21.075737 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0417 18:41:21.093336 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:21.226580 1 sca.go:266] etc-pki-entitlement secret successfully created I0417 18:41:21.226603 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0417 18:41:21.293399 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:21.493068 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:21.692477 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). W0417 18:41:21.874440 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0417 18:41:21.891993 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:22.091965 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:22.291403 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:22.491526 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:22.691772 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). W0417 18:41:22.874651 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0417 18:41:22.892277 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:23.091735 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:23.291605 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-monitoring-operator container cluster-monitoring-operator-557b74df6-l8xzw pod in namespace openshift-monitoring (previous: false). I0417 18:41:23.314396 1 tasks_processing.go:74] worker 18 stopped. I0417 18:41:23.314522 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=d191ff813659dd9e7c0da80853b6cc9382766a85fbe62d5f39e81221741fe7dc I0417 18:41:23.314572 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0417 18:41:23.314606 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=e1b224d716646cecd48e4a8bf8532fe3f03ac014bcf0361259403b4a225ebf32 I0417 18:41:23.314617 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0417 18:41:23.314638 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=63c75b7776b7f79767581c989519a1fa33bd6f5bcb4611011ee3de499e036670 I0417 18:41:23.314652 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0417 18:41:23.314676 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=a6be818c7120dc8506adab58126f3989f1c43f76f0a95127ff31e968cad909aa I0417 18:41:23.314723 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=315b4b9ce8a5f8ca9538e0bd26a66e4fb3f30ae40c0545745abb7af6fc445f0c I0417 18:41:23.314747 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=43e017d04e3bbf64d40a72e21537d46434377e20db1bc4b85a7564dc721eb9ef I0417 18:41:23.314777 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=29e8b84e716d81ce21bf754a8d8dff546b109d97f931354072a5889f6d0704eb I0417 18:41:23.315028 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=96532a463eb907ab947e31cc71721f8e80e0d5bbbf1f9c24123620e39b1a32a6 I0417 18:41:23.315047 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=391353e57f7d59a383da540765ed2b4be03a46c649cc9099d694510439af5fe0 I0417 18:41:23.315057 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0417 18:41:23.315073 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=e227c7cdfa58412258fe097cd7dfd43a38968432bb0d4917f4b02a4f9a8d59b7 I0417 18:41:23.315084 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0417 18:41:23.315101 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=0859005efbd8f9432508071dab33a3d76efe509529c01cb0a70bfea55484ba89 I0417 18:41:23.315111 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0417 18:41:23.315130 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=5b731edc1ea5e02be964239a201a741bec418bfffaf685e3e93a859c0a726090 I0417 18:41:23.315140 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0417 18:41:23.315162 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=751f4117b6f946377091d807b9103eed87487205f2e6d9f9314ca1d7c80eb489 I0417 18:41:23.315267 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=3c4b9fe276089dd9ed6ceccaebc2b0bf8673d0016a1d62c56186c2d6f0a79d24 I0417 18:41:23.315276 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0417 18:41:23.315284 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0417 18:41:23.315304 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0417 18:41:23.315338 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=59c6ac21231c34840bf4d4e6f68f793d8d4322b879d8609e98ebaccdfeab5ec2 I0417 18:41:23.315362 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=643fc1e43177f9c88ead39cb08ddd72a473f93754c88a1ec2503712d1f4cab2e I0417 18:41:23.315371 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0417 18:41:23.315387 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=c3c8b41d86bae4ba2a41218bce475ee4defc9407300e257e1e9bffd857d61ae8 I0417 18:41:23.315395 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0417 18:41:23.315412 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=450e46d0e89694f8ac69723b7cf51e6291c23110a6b184317dba1741cde27c82 I0417 18:41:23.315421 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0417 18:41:23.315434 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=eb4f07eeea268e634062fb6706095d14393d1227088cc0450192eec3479eb556 I0417 18:41:23.315450 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=72014ed0c9dedcc97e2db9fae5950f0cce23b81ec0574773a4117f7b2f00d643 I0417 18:41:23.315464 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=fd01cd12f3081c61854ae7749cd5e78ce11208a1e6f402c1cb7847c22ccd1ed0 I0417 18:41:23.315484 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=50a70a535764ea24d4fbe22cd4c1a2ec4e7b675d65cb4bb2b87888961a17ca1e I0417 18:41:23.315493 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0417 18:41:23.315516 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=1c7ce01ba956e93176851749d3517374a94e5a1db419d2fdb7fbd338c43705da I0417 18:41:23.315534 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0417 18:41:23.315544 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0417 18:41:23.315551 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.475170011s to process 39 records I0417 18:41:23.498501 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-state-metrics container kube-state-metrics-6dfffbb599-pfdhn pod in namespace openshift-monitoring (previous: false). I0417 18:41:23.695254 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container kube-state-metrics-6dfffbb599-pfdhn pod in namespace openshift-monitoring (previous: false). W0417 18:41:23.874367 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0417 18:41:23.892288 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container kube-state-metrics-6dfffbb599-pfdhn pod in namespace openshift-monitoring (previous: false). I0417 18:41:24.097643 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-59d7f65bd7-92jcr pod in namespace openshift-monitoring (previous: false). I0417 18:41:24.292000 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-59d7f65bd7-m76zh pod in namespace openshift-monitoring (previous: false). I0417 18:41:24.493151 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-5cbb48bf6b-9nn8c pod in namespace openshift-monitoring (previous: false). I0417 18:41:24.692821 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-5cbb48bf6b-hxphz pod in namespace openshift-monitoring (previous: false). W0417 18:41:24.874438 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0417 18:41:24.891393 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-dtd9l pod in namespace openshift-monitoring (previous: false). I0417 18:41:25.090850 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0417 18:41:25.090867 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-dtd9l pod in namespace openshift-monitoring (previous: false). I0417 18:41:25.293267 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-dtd9l pod in namespace openshift-monitoring (previous: false). I0417 18:41:25.491952 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-jl8c7 pod in namespace openshift-monitoring (previous: false). I0417 18:41:25.692154 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0417 18:41:25.692173 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-jl8c7 pod in namespace openshift-monitoring (previous: false). W0417 18:41:25.871925 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0417 18:41:25.871954 1 tasks_processing.go:74] worker 10 stopped. E0417 18:41:25.871979 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0417 18:41:25.872004 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0417 18:41:25.872017 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.029834583s to process 1 records I0417 18:41:25.892384 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-jl8c7 pod in namespace openshift-monitoring (previous: false). I0417 18:41:26.091834 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-msqvr pod in namespace openshift-monitoring (previous: false). I0417 18:41:26.290590 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0417 18:41:26.290610 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-msqvr pod in namespace openshift-monitoring (previous: false). I0417 18:41:26.492563 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-msqvr pod in namespace openshift-monitoring (previous: false). I0417 18:41:26.691679 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container openshift-state-metrics-b8587444b-vw876 pod in namespace openshift-monitoring (previous: false). I0417 18:41:26.891371 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container openshift-state-metrics-b8587444b-vw876 pod in namespace openshift-monitoring (previous: false). I0417 18:41:27.092832 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for openshift-state-metrics container openshift-state-metrics-b8587444b-vw876 pod in namespace openshift-monitoring (previous: false). I0417 18:41:27.291471 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:27.492051 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:27.692505 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:27.891170 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:28.092050 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:28.291776 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:28.491928 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0417 18:41:28.692044 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:28.892386 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:29.094155 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:29.293926 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:29.491667 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:29.691784 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:29.891912 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0417 18:41:30.091590 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator container prometheus-operator-76b589d668-chpmc pod in namespace openshift-monitoring (previous: false). I0417 18:41:30.296148 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-operator-76b589d668-chpmc pod in namespace openshift-monitoring (previous: false). I0417 18:41:30.492731 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-59b79769fb-5vngs pod in namespace openshift-monitoring (previous: false). I0417 18:41:30.690884 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-59b79769fb-xbhnn pod in namespace openshift-monitoring (previous: false). I0417 18:41:30.892257 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for telemeter-client container telemeter-client-69c7bc98d9-pqxkw pod in namespace openshift-monitoring (previous: false). I0417 18:41:31.092563 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for reload container telemeter-client-69c7bc98d9-pqxkw pod in namespace openshift-monitoring (previous: false). I0417 18:41:31.291283 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container telemeter-client-69c7bc98d9-pqxkw pod in namespace openshift-monitoring (previous: false). I0417 18:41:31.508254 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-7ccdf587fd-gbqmj pod in namespace openshift-monitoring (previous: false). I0417 18:41:31.692618 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-7ccdf587fd-gbqmj pod in namespace openshift-monitoring (previous: false). I0417 18:41:31.891422 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-7ccdf587fd-gbqmj pod in namespace openshift-monitoring (previous: false). I0417 18:41:32.091741 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-7ccdf587fd-gbqmj pod in namespace openshift-monitoring (previous: false). I0417 18:41:32.291174 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-7ccdf587fd-gbqmj pod in namespace openshift-monitoring (previous: false). I0417 18:41:32.491817 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-7ccdf587fd-gbqmj pod in namespace openshift-monitoring (previous: false). I0417 18:41:32.691658 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-7ccdf587fd-nlhk2 pod in namespace openshift-monitoring (previous: false). I0417 18:41:32.891970 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-7ccdf587fd-nlhk2 pod in namespace openshift-monitoring (previous: false). I0417 18:41:33.091619 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-7ccdf587fd-nlhk2 pod in namespace openshift-monitoring (previous: false). I0417 18:41:33.291011 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-7ccdf587fd-nlhk2 pod in namespace openshift-monitoring (previous: false). I0417 18:41:33.490817 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-7ccdf587fd-nlhk2 pod in namespace openshift-monitoring (previous: false). I0417 18:41:33.683019 1 tasks_processing.go:74] worker 2 stopped. I0417 18:41:33.683058 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0417 18:41:33.683071 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.843811037s to process 1 records I0417 18:41:33.692201 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-7ccdf587fd-nlhk2 pod in namespace openshift-monitoring (previous: false). I0417 18:41:33.891263 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-7f449dfbfd-6825n pod in namespace openshift-network-console (previous: false). I0417 18:41:34.090989 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-7f449dfbfd-pdcx6 pod in namespace openshift-network-console (previous: false). I0417 18:41:34.291017 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-66dbbb6468-44nzf pod in namespace openshift-console-operator (previous: true). I0417 18:41:34.449024 1 tasks_processing.go:74] worker 55 stopped. I0417 18:41:34.449347 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=fcae92fe0b444234f2b5fc2bfacf393cc5f7649b4bb76b1a257137b870b17061 I0417 18:41:34.449369 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.606997841s to process 1 records I0417 18:41:34.491194 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-66dbbb6468-44nzf pod in namespace openshift-console-operator (previous: false). I0417 18:41:34.694301 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-5cf9589c6c-4bts6 pod in namespace openshift-console (previous: false). I0417 18:41:34.886107 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0417 18:41:34.886130 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-5cf9589c6c-brv57 pod in namespace openshift-console (previous: false). I0417 18:41:35.085630 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0417 18:41:35.085649 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-774cbc7cbd-8j2zj pod in namespace openshift-console (previous: false). I0417 18:41:35.291633 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-78b9644658-c49xn pod in namespace openshift-console (previous: false). I0417 18:41:35.491559 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-78b9644658-d5472 pod in namespace openshift-console (previous: false). I0417 18:41:35.692017 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-5c54f4944f-27sd8 pod in namespace openshift-console (previous: false). I0417 18:41:35.892650 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-5c54f4944f-zjhp8 pod in namespace openshift-console (previous: false). I0417 18:41:36.092764 1 tasks_processing.go:74] worker 26 stopped. I0417 18:41:36.093515 1 recorder.go:75] Recording events/openshift-monitoring with fingerprint=b22843ef493cbdf69d3354bea88b18e0710b802f71e974e82b4a510f5022cf0e I0417 18:41:36.093565 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=650cada2fce01eeaf5d0311985492292e8845b6be5b7d5a4cfb38f2cde3a05bd I0417 18:41:36.093682 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=f572337f20dab7b33efa7a7ccf870e307e1004061ef80684ceafcb616ba5add1 I0417 18:41:36.093780 1 recorder.go:75] Recording events/openshift-console with fingerprint=30be30cd7c4b83f26ddf0bb81b51b90a28bcb99d5614c082da7cc6d8a96ff524 I0417 18:41:36.093793 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/init-config-reloader_current.log with fingerprint=1b8bc22ce509aaa9b3bc5cdfcd3fe433610514cc419669f4e761850c9526e1d7 I0417 18:41:36.093805 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/alertmanager_current.log with fingerprint=4a8d21dafdce19a0097c4de594202bc19dbd8d33c76042d5dbb9fe76adc047fb I0417 18:41:36.093820 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/config-reloader_current.log with fingerprint=6f55472eb28a263ddada182f1427c465260e32fccf1c4cb5cc7da0c46c50e7f2 I0417 18:41:36.093828 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-web_current.log with fingerprint=6da6ff3486f431cdb5c857dcc60edfb67a34e0ab847b69cf76eea5fc80308b58 I0417 18:41:36.093833 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy_current.log with fingerprint=8a884c47fe92bc2e2ccccc92c8762214187dd9dc6a16c7173ec7f3a8efcbccbb I0417 18:41:36.093840 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-metric_current.log with fingerprint=3fd776a6bd72c83b8484a477915e9a34faeedf363f61cd92bc4831cc9a4a79b3 I0417 18:41:36.093844 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/prom-label-proxy_current.log with fingerprint=35b107c186b16c78eaba47dcd75c55d2bda55d29a8282bb3634cd0a51595b92f I0417 18:41:36.093881 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/init-config-reloader_current.log with fingerprint=2c8efd9330c2d4e3daf4d4c62aeab225595e5b5ca7ef1432274b60d5c97e2cc4 I0417 18:41:36.093894 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/alertmanager_current.log with fingerprint=153ac3dd45f8e2788585920bf4cbb9962516b69ac5f65f48fdbd4af9306e7b98 I0417 18:41:36.093906 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/config-reloader_current.log with fingerprint=9a00c35e512169155d098e75c909a773e6b0fc33eda3c91b193528df3cc69d71 I0417 18:41:36.093912 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-web_current.log with fingerprint=5ad77a21c014aee1fdce8761d88c0eb9873eb9659971cedf118d04f6a1f734dc I0417 18:41:36.093918 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy_current.log with fingerprint=c25dcfdbe6e2d771ea4da9f48eeb7ce374818a4493ac82218db64b23fcb06994 I0417 18:41:36.093924 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-metric_current.log with fingerprint=cf580fa969b6b58559be6dcf27fc1f8ae6cd796d5b1e731645ade94e806f4637 I0417 18:41:36.093928 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/prom-label-proxy_current.log with fingerprint=4a2010b638186da0ab3f9a2d0aa3e1f785181540550f24571fb605d860b7247c I0417 18:41:36.094072 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/cluster-monitoring-operator-557b74df6-l8xzw/cluster-monitoring-operator_current.log with fingerprint=7f4bdf7297da38573aba889b6f6e887c87ecd7d16e7bf895205b5114748d0b75 I0417 18:41:36.094098 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-6dfffbb599-pfdhn/kube-state-metrics_current.log with fingerprint=02200d59ab7d24fc9900c844408acbb9c0daaadedf9daf6349275d59d80e9bde I0417 18:41:36.094109 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-6dfffbb599-pfdhn/kube-rbac-proxy-main_current.log with fingerprint=32f3b597d3bff67875c6f7f7d33e5b96648f32430cf53b2a05706356bc83d7f7 I0417 18:41:36.094115 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-6dfffbb599-pfdhn/kube-rbac-proxy-self_current.log with fingerprint=eb93ddcc28de5a9082a630d50112ac03a3a50a0867db8e59968deb5148f4e247 I0417 18:41:36.094123 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-59d7f65bd7-92jcr/metrics-server_current.log with fingerprint=e52cbadb191ad5edb0c01cc222715c0b81f2e740a2062c7dae96098c76279391 I0417 18:41:36.094130 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-59d7f65bd7-m76zh/metrics-server_current.log with fingerprint=6daf7ff98d62a6d097b5e3ffc69dc17c76975908a4eb4652f076ba2a7dcab617 I0417 18:41:36.094141 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-5cbb48bf6b-9nn8c/monitoring-plugin_current.log with fingerprint=5fdac49609cf32799f463b52e427101df3eaa4b80ef9e3389369a9f8c875b4fb I0417 18:41:36.094152 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-5cbb48bf6b-hxphz/monitoring-plugin_current.log with fingerprint=886efa33981cd04a4f487101c854ed178aff5df47245a57b966b97ff8fd0c8cd I0417 18:41:36.094179 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-dtd9l/node-exporter_current.log with fingerprint=ae805ab945ff8bc72c63c58bff821cc9b315cd7b2167e4a1c6a27d8c6e1302c5 I0417 18:41:36.094189 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-dtd9l/kube-rbac-proxy_current.log with fingerprint=66a939696ee7365dca7aedcd66aef8fbeb7b41b705711e0832365705c2fc5245 I0417 18:41:36.094215 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-jl8c7/node-exporter_current.log with fingerprint=7ecb58fd9b88160bdc914455f54b6d2095cbf2467136b3df0ab6b1c25abb990c I0417 18:41:36.094224 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-jl8c7/kube-rbac-proxy_current.log with fingerprint=463fb82db66a9780855e167e9d3584d2082f6651fc3eae470cfb7aadfb7a86ef I0417 18:41:36.094251 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-msqvr/node-exporter_current.log with fingerprint=5a50e6af066dec7391ae4aa275e26aeb0d479252cbe9758e50af48cba4ee8c8b I0417 18:41:36.094259 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-msqvr/kube-rbac-proxy_current.log with fingerprint=53ab28f701679e1ec399314d6e5290194d7fbe4105d66e36c9dea7fa7408e149 I0417 18:41:36.094266 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-b8587444b-vw876/kube-rbac-proxy-main_current.log with fingerprint=19493f58a6b291fefa34365676e221a0b4389c9046d9cd06d53a82032dadf213 I0417 18:41:36.094272 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-b8587444b-vw876/kube-rbac-proxy-self_current.log with fingerprint=f33b43eb6fefc23134218f229b6b3e4b4f7f687c595fdbebb1a5994d35474ebb I0417 18:41:36.094283 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-b8587444b-vw876/openshift-state-metrics_current.log with fingerprint=8d39a93c1b968553ef8f7df2375a7cbdd4e9880bb3c79935525febeaf3abadb7 I0417 18:41:36.094290 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/init-config-reloader_current.log with fingerprint=f4cc74e4949a02b0805fdb88188702c11d83180143c579855d10373815762e48 I0417 18:41:36.094333 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/prometheus_current.log with fingerprint=c7d0ac56ce1ed7e8446853cc6d239bfe09f1eab028576656578ddc5f84c834f2 I0417 18:41:36.094348 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/config-reloader_current.log with fingerprint=fbb1e94242922c18596981be91bcc936b1f36fe463ec61bd0297af42c87c561f I0417 18:41:36.094359 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/thanos-sidecar_current.log with fingerprint=aeec146c2bf1e110b7636b472df2b81332e71067641ed328188fc5c6a29512f8 I0417 18:41:36.094366 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-web_current.log with fingerprint=661b07faa8c9495a7ba5ce160fd05f19af159278de7b53ec8b594df10ed44d34 I0417 18:41:36.094373 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy_current.log with fingerprint=4cd977053736a0b710219baf608e014ead257273f7a2dcd95a835a659c020369 I0417 18:41:36.094379 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-thanos_current.log with fingerprint=1e480f77431eed7104f4e23365fef1eb556ae6e01c8b2310c7fdea1783b07397 I0417 18:41:36.094393 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/init-config-reloader_current.log with fingerprint=6966dacdc1821d4398b487298ddd51aa22fde2e497619226f19edeef236c7e6a I0417 18:41:36.094449 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/prometheus_current.log with fingerprint=e4cd636e776048e7c3280c44de530ff87413fa0278dba6bb04726769cf5dc6dd I0417 18:41:36.094465 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/config-reloader_current.log with fingerprint=6e996b034227bdf328079bca4f3c126416c5ea9360447d0f9f083cd98e5b13ff I0417 18:41:36.094475 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/thanos-sidecar_current.log with fingerprint=97d4cb51e70e46ddb50df83181d8ad8093d37caac9d559badfd2b42766a5588e I0417 18:41:36.094482 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-web_current.log with fingerprint=cebfcfb437206c17ce7953b84a028a68161c3b5381f298c5f2cb88b83ce577be I0417 18:41:36.094488 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy_current.log with fingerprint=385f35e178fe592e44aeadcd3b6aa0b0e0c8b75908c7c1d1052a0201cb26afed I0417 18:41:36.094495 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-thanos_current.log with fingerprint=e5d32e1e9424f91ff42ebb5b09c240c669c66e7742712d5fdab8e6a078b06801 I0417 18:41:36.094574 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-76b589d668-chpmc/prometheus-operator_current.log with fingerprint=9d3b72a10a2b01964bd3561b135a794209a2ca28cc2c5c8e5e8c95414da43b26 I0417 18:41:36.094582 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-76b589d668-chpmc/kube-rbac-proxy_current.log with fingerprint=6bd0f4913ae3bf49886a89cbd40529a434e01e928fbc7f7ac5234607ea95c342 I0417 18:41:36.094590 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-59b79769fb-5vngs/prometheus-operator-admission-webhook_current.log with fingerprint=360c5c07349d420b721f5b5bddaa492a6307af912fbfc55510e65f38567602e9 I0417 18:41:36.094596 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-59b79769fb-xbhnn/prometheus-operator-admission-webhook_current.log with fingerprint=b388471cb1a96809090c3fe28f006b1263ee0d36ee784373beda8f5c06551607 I0417 18:41:36.094657 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/telemeter-client-69c7bc98d9-pqxkw/telemeter-client_current.log with fingerprint=70d1f42406b80390d7f104f39a09344768a32e148590a7b9f07c42a980fd9798 I0417 18:41:36.094667 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/telemeter-client-69c7bc98d9-pqxkw/reload_current.log with fingerprint=56acca1d977ada5909209875f6b1d34b2ab25438d7ee59ef0066602f84df6202 I0417 18:41:36.094673 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/telemeter-client-69c7bc98d9-pqxkw/kube-rbac-proxy_current.log with fingerprint=de7761a37887ceb0ec57d5b1e691fd2b9254108b178b4417089c84f38636b2bd I0417 18:41:36.094683 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-gbqmj/thanos-query_current.log with fingerprint=dbf5c0c4a78aa738ff51f436c230668308212247309c75428873fd9efe0ea7cc I0417 18:41:36.094705 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-gbqmj/kube-rbac-proxy-web_current.log with fingerprint=087884221c28c36fab16ea1d5918b5a61d5f84690811859b6795f646c77fadc0 I0417 18:41:36.094711 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-gbqmj/kube-rbac-proxy_current.log with fingerprint=b8e90800b15ae16342c963c6565472221011d872709bd96a87e35e7113227c64 I0417 18:41:36.094716 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-gbqmj/prom-label-proxy_current.log with fingerprint=967b24932151e4c71dbb940d50a2d03c889dffdaa1244c95c6cbbc027ad64ccf I0417 18:41:36.094724 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-gbqmj/kube-rbac-proxy-rules_current.log with fingerprint=fc8e7fc836d571408b9cd2ddd5e982ba8082d10451d186ee433ff5753f56d02a I0417 18:41:36.094731 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-gbqmj/kube-rbac-proxy-metrics_current.log with fingerprint=a7219d82dd3801c7b43d37f0e121ee5157a66dbe6756e8379aa049373472feb6 I0417 18:41:36.094742 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-nlhk2/thanos-query_current.log with fingerprint=818ebaf89d36fb540c0423f8bc3d03a05675f32e866e9dcc8d1d4ffb708b1e1b I0417 18:41:36.094749 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-nlhk2/kube-rbac-proxy-web_current.log with fingerprint=47074208a545da8c396fa6a071e56a58e05c38e56a433eccbe625a3161e6510b I0417 18:41:36.094755 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-nlhk2/kube-rbac-proxy_current.log with fingerprint=1b5281a2fa4e601b751c0a9867fb8a69d6c3655477ac1f2db8a465c5a6d96dba I0417 18:41:36.094759 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-nlhk2/prom-label-proxy_current.log with fingerprint=4ae9b8f255c0e3779af910a30a828752a605054e0a4f1f4cf0f5c38c964d3f8b I0417 18:41:36.094765 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-nlhk2/kube-rbac-proxy-rules_current.log with fingerprint=3c96a9f8465cb95e48a40a600521ed8f768a9f2cbd14aba8b0ae731cc1ee1343 I0417 18:41:36.094772 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-7ccdf587fd-nlhk2/kube-rbac-proxy-metrics_current.log with fingerprint=3973bdf6e9aeb998d89a7322f0a2d99e5ccc88ed065acf82f7075c27bb182245 I0417 18:41:36.094778 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-7f449dfbfd-6825n/networking-console-plugin_current.log with fingerprint=1741a4b8583596ac324d84c80c2e5492df4538bf8220d296dd8c5f496c23e2de I0417 18:41:36.094784 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-7f449dfbfd-pdcx6/networking-console-plugin_current.log with fingerprint=22be793ba1488a3aa2463854eb95e977428010103617aa6e72f652870185c30b I0417 18:41:36.094931 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf with fingerprint=87f34cd89658e4b7fd915a56393d3b142fb3cc08359c9e8150cd28c231df5472 E0417 18:41:36.094949 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json" was already recorded and had the fingerprint "87f34cd89658e4b7fd915a56393d3b142fb3cc08359c9e8150cd28c231df5472", overwriting with the record having fingerprint "87f34cd89658e4b7fd915a56393d3b142fb3cc08359c9e8150cd28c231df5472" W0417 18:41:36.094961 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json" because of the warning: warning: the record with the same fingerprint "87f34cd89658e4b7fd915a56393d3b142fb3cc08359c9e8150cd28c231df5472" was already recorded at path "config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json" I0417 18:41:36.094979 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-66dbbb6468-44nzf/console-operator_previous.log with fingerprint=62454019d11e9b6e3274f7e609ab787af49184d39745b7063beb6dc5fc9b495a I0417 18:41:36.095178 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-66dbbb6468-44nzf/console-operator_current.log with fingerprint=bb131cc111420b62d3f11c5efd29eb1888d91dd01946afefe2c11bae2baccd02 I0417 18:41:36.095193 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-774cbc7cbd-8j2zj/console_current.log with fingerprint=e035ec0a6af065773955df288f9ca542465f1867769dab4924696efe24a6b503 I0417 18:41:36.095205 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-78b9644658-c49xn/console_current.log with fingerprint=1ec5dc2c314c12df4c99b9eced223bda20fb7ea1f1da105a3476b9047708a095 I0417 18:41:36.095218 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-78b9644658-d5472/console_current.log with fingerprint=86cd13caf52c841d8e6df6beeec65bc5efa18fcdc0babba9939b20148a8cd8ec I0417 18:41:36.095245 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-5c54f4944f-27sd8/download-server_current.log with fingerprint=6de8d280bd990ba41fbd8e9628121001b625fbbc5d51800616725776d40279f6 I0417 18:41:36.095275 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-5c54f4944f-zjhp8/download-server_current.log with fingerprint=9a644f3e8d3130993e09ab7e12c40390df2bd58d721f5ea1b70ec48a3a2dd371 I0417 18:41:36.095282 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 15.250648873s to process 78 records E0417 18:41:36.095350 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 15.256s with: function \"metrics\" failed with an error, function \"tsdb_status\" failed with an error, function \"silenced_alerts\" failed with an error, function \"active_alerts\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json\"" I0417 18:41:36.096470 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "metrics" failed with an error, function "tsdb_status" failed with an error, function "silenced_alerts" failed with an error, function "active_alerts" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-66dbbb6468-44nzf.json" I0417 18:41:36.096488 1 periodic.go:216] Running workloads gatherer I0417 18:41:36.096506 1 tasks_processing.go:45] number of workers: 2 I0417 18:41:36.096518 1 tasks_processing.go:69] worker 1 listening for tasks. I0417 18:41:36.096524 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0417 18:41:36.096526 1 tasks_processing.go:69] worker 0 listening for tasks. I0417 18:41:36.096606 1 tasks_processing.go:71] worker 0 working on workload_info task. I0417 18:41:36.103309 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-216.ec2.internal... I0417 18:41:36.103316 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-177.ec2.internal... I0417 18:41:36.103330 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-68.ec2.internal... I0417 18:41:36.125980 1 tasks_processing.go:74] worker 1 stopped. I0417 18:41:36.125998 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 29.440964ms to process 0 records I0417 18:41:40.507919 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0417 18:41:40.534930 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0417 18:41:40.543313 1 gather_workloads_info.go:387] No image sha256:730d1b6988025bef0daa3a9a5d8467ec4a26b0382cc52f91c3375b4590d3518a (10ms) I0417 18:41:40.551272 1 gather_workloads_info.go:387] No image sha256:a043239802b3eb8b323d285193d2527fad0ecec98ca91d188a3472a2fac8ae04 (8ms) I0417 18:41:40.559721 1 gather_workloads_info.go:387] No image sha256:5940cc34b322325618929b1e0eb3864dae73d54c8f5b474e218dfb28123a61c7 (8ms) I0417 18:41:40.567527 1 gather_workloads_info.go:387] No image sha256:572ebf7c7fab1a518582f3ca46c866730423cce6e8a19f614dc15fd61bf0c65a (8ms) I0417 18:41:40.577624 1 gather_workloads_info.go:398] Found image sha256:23204877f166455ce8f9de9b91a467eeda576f172d5f96124a68aa986c4775d6 (10ms) I0417 18:41:40.587590 1 gather_workloads_info.go:387] No image sha256:3380fecebaeee139c1607fbcd9346cebc92806679568e2244a8905ef0f52aad2 (10ms) I0417 18:41:40.595938 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (8ms) I0417 18:41:40.604376 1 gather_workloads_info.go:387] No image sha256:5808401268394502d335281ea1a294b07210461b986b58f91d7d1f29c0029c6d (8ms) I0417 18:41:40.612172 1 gather_workloads_info.go:387] No image sha256:89277d8d4560d71db88c2dcc67c992a24544ca21810920b609c1d49d53b4a287 (8ms) I0417 18:41:40.619874 1 gather_workloads_info.go:387] No image sha256:5aaea0419169e55832cc27acfe0fe3b9513a343d6bdf71d3da1575ed322245d0 (8ms) I0417 18:41:40.642589 1 gather_workloads_info.go:398] Found image sha256:673ebc8cc22c56c8f410e011b2fa950d28cf7b6420e17fdb6580d6cb10523384 (23ms) I0417 18:41:40.743730 1 gather_workloads_info.go:387] No image sha256:ab60623bb32f7e75fca71ef65137731cae347a21c7a4091dfd583fa00732721c (101ms) I0417 18:41:40.845015 1 gather_workloads_info.go:387] No image sha256:52db05084f2a2e7d7bf4040e4865df88ec4074efbbe59a78d861be2c5ab2aa40 (101ms) I0417 18:41:40.943734 1 gather_workloads_info.go:387] No image sha256:5ff204630794311b0b37fa7b197b933ad85d76a481bf7cdb3bcbada08f0cdcbf (99ms) I0417 18:41:41.043271 1 gather_workloads_info.go:387] No image sha256:5d6a2b8d2508c7da71192661e7c0dcf1335f48b6336a20189abffbe79eeff962 (100ms) I0417 18:41:41.144126 1 gather_workloads_info.go:387] No image sha256:48e883a3932aea9457f55cd4628d21397d429d4828f42f95d1c903d9d5395bde (101ms) I0417 18:41:41.243446 1 gather_workloads_info.go:387] No image sha256:9d6d63dce784df5863e6a0ba4c11317a26c9d756d4e62d47759c97190a71cce5 (99ms) I0417 18:41:41.343399 1 gather_workloads_info.go:387] No image sha256:5b6212b8f539f08e78417d8a4b7485ca0b4e7927cacd7b752742a28841bc8ccd (100ms) I0417 18:41:41.447914 1 gather_workloads_info.go:387] No image sha256:084aa9b0f8a6d478549dc384d4e66da13ee9b25cc98531da861cc19dee2a9e8f (104ms) I0417 18:41:41.546772 1 gather_workloads_info.go:387] No image sha256:1eb6a6ee3c66ac15be6bc21bb6015c595c11c15b9dfd21bd9f5ce7f7b990ebaa (99ms) I0417 18:41:41.643310 1 gather_workloads_info.go:387] No image sha256:1f118a4ec0b92cb88c8544ea31d2b9adfc7006eeddb7a8d260150ecbce4adb8c (97ms) I0417 18:41:41.744081 1 gather_workloads_info.go:387] No image sha256:8f881f6f0498eaba860d711dc8e9dc066db8728f22abedb82e1dc9a8e1c30046 (101ms) I0417 18:41:41.842983 1 gather_workloads_info.go:387] No image sha256:55b1db6038c5beaed54c626e3343b7a8589cc0be8dc41d1a66b4deab766ff520 (99ms) I0417 18:41:41.943409 1 gather_workloads_info.go:387] No image sha256:15e1b5f763553e86d0230a751b0783e3b990438fd07cf96eed164bccb72ed564 (100ms) I0417 18:41:42.044005 1 gather_workloads_info.go:387] No image sha256:5c6d21c3f97366bc7ab57031cc027b67405a684bd804ce364ed5998b0685eaca (101ms) I0417 18:41:42.143938 1 gather_workloads_info.go:387] No image sha256:000105ef5150e7079b90a613fb9e6193e2a6ef9b1908d2dce44f2395d4fd070f (100ms) I0417 18:41:42.243501 1 gather_workloads_info.go:387] No image sha256:d64bea34bf3e1bb0b3a701c3ff14e66665afc1b050f28124ad7e6888eaec3a81 (100ms) I0417 18:41:42.343625 1 gather_workloads_info.go:387] No image sha256:0b2e6536a0ca3afc5e64a218087a6287c74b77a359075a385f1949ca5bd51b22 (100ms) I0417 18:41:42.443115 1 gather_workloads_info.go:387] No image sha256:15928a5b803fa00781b408dbc4c3bfa7c5287b18584ba31f817def8a4430c725 (99ms) I0417 18:41:42.543576 1 gather_workloads_info.go:387] No image sha256:5ac9c549d65fc1d8bc900773bebee43e9192bcec1bb5fa46afb4597230c16ac7 (100ms) I0417 18:41:42.643326 1 gather_workloads_info.go:387] No image sha256:08c5a78c8a5af04c549e2273aaf4bb452a75bf038d68aa9d01bb2aff66c30e90 (100ms) I0417 18:41:42.745022 1 gather_workloads_info.go:387] No image sha256:0acfe677ed58031f8381cb8a4f2d8917ec4cfa95485e2a46529dffae54c75ace (102ms) I0417 18:41:42.843030 1 gather_workloads_info.go:387] No image sha256:637d41f067a5239096fc22b135181cda5113da833f1370e7a73965e83792e93a (98ms) I0417 18:41:42.943859 1 gather_workloads_info.go:387] No image sha256:48f5559439bea030a6423652da448c25fff0c62ba293eb521843c1a8eb89ddb0 (101ms) I0417 18:41:43.043531 1 gather_workloads_info.go:387] No image sha256:0a4dfb8d4c1b3849319d45b4c54dff26a7238a2c08fcaa121f93073e95ab12e8 (100ms) I0417 18:41:43.142484 1 gather_workloads_info.go:387] No image sha256:17aefdc2f4752c23dc8f0947bb01fc34635822eeb18f2d8a5eddf8211b84d450 (99ms) I0417 18:41:43.244515 1 gather_workloads_info.go:387] No image sha256:3bba1358d4a0ae878ff491c0c2cbfffe60649e110b40342b878fe8fa332f8858 (102ms) I0417 18:41:43.345097 1 gather_workloads_info.go:387] No image sha256:30e597ec5d6bb96ff70a4f8688c748b659cd4fd5d73d222e8701821d236795c5 (101ms) I0417 18:41:43.443723 1 gather_workloads_info.go:387] No image sha256:25c148fd380b1a9db3f6039d2e0eabc489a954921452391390ba9192b2325678 (99ms) I0417 18:41:43.542792 1 gather_workloads_info.go:387] No image sha256:9ef6dc311b880a6442b7a3a19006ddd955c7da36850bf000f5c9b68fd0862dd4 (99ms) I0417 18:41:43.643193 1 gather_workloads_info.go:387] No image sha256:934d8e8c50f3c609b8eea80d1051111fe3d066fe8c65c79572072ae55fcb0a86 (100ms) I0417 18:41:43.743914 1 gather_workloads_info.go:387] No image sha256:6814f5fced0ef219d06374011c68a11a3da788a764a00a69ade435466d9ee240 (101ms) I0417 18:41:43.844262 1 gather_workloads_info.go:387] No image sha256:53ff8efdd8f563421ebc1859ab021a3bbc1306f82d4ff5d9c21bc60ec5f57898 (100ms) I0417 18:41:43.944157 1 gather_workloads_info.go:387] No image sha256:875c77e5d144f03fb91d8cee0259f6966683ca88d1bf818dbf4652c16b70312c (100ms) I0417 18:41:44.043640 1 gather_workloads_info.go:387] No image sha256:6c6d412ccf0c3d6a14645a79df6c4658c0f8725d7954cb83c3d2f0116499345a (99ms) I0417 18:41:44.144047 1 gather_workloads_info.go:387] No image sha256:bdeb1c455cffccdb6c90db238e9d371de06340b1d6105d47f66d87293f878ac1 (100ms) I0417 18:41:44.243616 1 gather_workloads_info.go:387] No image sha256:15677f0b70e6aa2dfaf088e45fc1a425c22bf6fda326b8116f87e88b6694dfab (100ms) I0417 18:41:44.243651 1 tasks_processing.go:74] worker 0 stopped. I0417 18:41:44.243971 1 recorder.go:75] Recording config/workload_info with fingerprint=93abe93a3428a39eee1166ec361b57b2d9ec5ccaddab2f11ec8a9c82dc9c3378 I0417 18:41:44.243987 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.14702805s to process 1 records I0417 18:41:44.244002 1 periodic.go:263] Periodic gather workloads completed in 8.147s I0417 18:41:44.244010 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0417 18:41:44.244015 1 periodic.go:216] Running conditional gatherer I0417 18:41:44.249626 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules I0417 18:41:44.482652 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0417 18:41:44.485482 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.153.210:9091: connect: connection refused I0417 18:41:44.485584 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0417 18:41:44.492192 1 conditional_gatherer.go:392] cluster version is '4.21.9' E0417 18:41:44.492209 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492216 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492221 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492225 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492230 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492235 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492239 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492243 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492246 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0417 18:41:44.492253 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0417 18:41:44.492273 1 tasks_processing.go:45] number of workers: 3 I0417 18:41:44.492288 1 tasks_processing.go:69] worker 2 listening for tasks. I0417 18:41:44.492297 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0417 18:41:44.492297 1 tasks_processing.go:69] worker 0 listening for tasks. I0417 18:41:44.492311 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0417 18:41:44.492311 1 tasks_processing.go:69] worker 1 listening for tasks. I0417 18:41:44.492317 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0417 18:41:44.492320 1 tasks_processing.go:74] worker 1 stopped. I0417 18:41:44.492400 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0417 18:41:44.492419 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.749µs to process 1 records I0417 18:41:44.492463 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0417 18:41:44.492476 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.388µs to process 1 records I0417 18:41:44.492485 1 tasks_processing.go:74] worker 0 stopped. I0417 18:41:44.492576 1 tasks_processing.go:74] worker 2 stopped. I0417 18:41:44.492591 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 251.177µs to process 0 records I0417 18:41:44.492608 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0417 18:41:44.492625 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0417 18:41:44.521542 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=589b4f5dbff56726b621e8b170fcfcc0ca60ccac30ce8b62410c955203cc3cb5 I0417 18:41:44.521754 1 diskrecorder.go:70] Writing 183 records to /var/lib/insights-operator/insights-2026-04-17-184144.tar.gz I0417 18:41:44.535972 1 diskrecorder.go:51] Wrote 183 records to disk in 14ms I0417 18:41:44.536017 1 periodic.go:285] Gathering cluster info every 2h0m0s I0417 18:41:44.536038 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0417 18:42:50.840184 1 diskrecorder.go:223] Found files to send: insights-2026-04-17-184144.tar.gz I0417 18:42:50.840216 1 insightsuploader.go:150] Checking archives to upload periodically every 15m4.500669817s I0417 18:42:50.840225 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0417 18:42:50.859638 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0417 18:42:51.115719 1 requests.go:87] Successfully reported id=2026-04-17T18:42:50Z x-rh-insights-request-id=480707bd8e3a4cf9bd17982ec70946c8, wrote=94380 I0417 18:42:51.115861 1 insightsuploader.go:187] Uploaded report successfully in 275.626937ms I0417 18:42:51.115886 1 controller.go:129] Initializing last reported time to 2026-04-17T18:42:50Z I0417 18:42:51.115961 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0417 18:42:51.115976 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0417 18:42:51.115985 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0417 18:42:51.121638 1 controller.go:554] The operator is healthy I0417 18:43:20.854535 1 controller.go:554] The operator is healthy I0417 18:43:53.163260 1 insightsreport.go:137] Pulling report from smart-proxy I0417 18:43:53.163295 1 insightsreport.go:149] Retrieving report I0417 18:43:53.169878 1 requests.go:110] Retrieving report for cluster: 2c9c1877-c703-48cd-a66a-5bc683f3d846 I0417 18:43:53.169890 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/2c9c1877-c703-48cd-a66a-5bc683f3d846/reports I0417 18:43:53.173043 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/2c9c1877-c703-48cd-a66a-5bc683f3d846/reports I0417 18:43:53.329715 1 insightsreport.go:184] Report retrieved I0417 18:43:53.359573 1 insightsreport.go:239] Report retrieved correctly I0417 18:45:20.852202 1 controller.go:554] The operator is healthy I0417 18:46:20.833250 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0417 18:46:20.844181 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0417 18:46:20.844201 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0417 18:46:20.850501 1 secretconfigobserver.go:119] support secret does not exist I0417 18:47:20.851779 1 controller.go:554] The operator is healthy I0417 18:49:20.852396 1 controller.go:554] The operator is healthy I0417 18:51:20.828863 1 configmapobserver.go:84] configmaps "insights-config" not found I0417 18:51:20.833565 1 configmapobserver.go:84] configmaps "insights-config" not found I0417 18:51:20.850906 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0417 18:51:20.851477 1 controller.go:554] The operator is healthy I0417 18:51:20.855623 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0417 18:51:20.855639 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0417 18:51:20.860002 1 secretconfigobserver.go:119] support secret does not exist I0417 18:53:20.851641 1 controller.go:554] The operator is healthy I0417 18:55:20.851827 1 controller.go:554] The operator is healthy I0417 18:56:20.860580 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0417 18:56:20.866615 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0417 18:56:20.866637 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0417 18:56:20.871190 1 secretconfigobserver.go:119] support secret does not exist I0417 18:57:20.851521 1 controller.go:554] The operator is healthy I0417 18:58:00.618354 1 insightsuploader.go:146] Nothing to report since 2026-04-17T18:42:50Z I0417 18:59:20.867656 1 controller.go:554] The operator is healthy I0417 19:01:20.847326 1 configmapobserver.go:84] configmaps "insights-config" not found I0417 19:01:20.851094 1 controller.go:554] The operator is healthy I0417 19:01:20.851764 1 configmapobserver.go:84] configmaps "insights-config" not found I0417 19:01:20.872073 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0417 19:01:20.877001 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0417 19:01:20.877019 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0417 19:01:20.881172 1 secretconfigobserver.go:119] support secret does not exist I0417 19:03:20.866995 1 controller.go:554] The operator is healthy