I0702 11:46:54.995629 1 cmd.go:253] Using service-serving-cert provided certificates I0702 11:46:54.996077 1 observer_polling.go:159] Starting file observer W0702 11:46:55.015958 1 builder.go:272] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-5ff5cb4f99" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0702 11:46:55.505079 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0702 11:46:55.505327 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0702 11:46:55.506367 1 secure_serving.go:57] Forcing use of http/1.1 only W0702 11:46:55.506443 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0702 11:46:55.506481 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0702 11:46:55.506510 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0702 11:46:55.506542 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0702 11:46:55.506566 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0702 11:46:55.506589 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0702 11:46:55.506897 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0702 11:46:55.510750 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes CPMSMachineNamePrefix ChunkSizeMiB CloudDualStackNodeIPs ConsolePluginContentSecurityPolicy DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags GatewayAPI GatewayAPIController HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy OnClusterBuild PersistentIPsForVirtualization PrivateHostedZoneAWS RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController VSphereDriverConfiguration VSphereMultiVCenters ValidatingAdmissionPolicy AWSClusterHostedDNS AutomatedEtcdBackup BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNS GCPCustomAPIEndpoints HighlyAvailableArbiter ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements SELinuxChangePolicy SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerification SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0702 11:46:55.510774 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ChunkSizeMiB", "CloudDualStackNodeIPs", "ConsolePluginContentSecurityPolicy", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "GatewayAPI", "GatewayAPIController", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "OnClusterBuild", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "VSphereDriverConfiguration", "VSphereMultiVCenters", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AutomatedEtcdBackup", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "HighlyAvailableArbiter", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "SELinuxChangePolicy", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0702 11:46:55.511315 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0702 11:46:55.511327 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0702 11:46:55.511329 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0702 11:46:55.511344 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0702 11:46:55.511348 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0702 11:46:55.511337 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0702 11:46:55.511548 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0702 11:46:55.511800 1 secure_serving.go:213] Serving securely on [::]:8443 I0702 11:46:55.511833 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" E0702 11:46:55.513547 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18be7684dd96d658 openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdditionalRoutingCapabilities\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"CPMSMachineNamePrefix\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"ConsolePluginContentSecurityPolicy\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"GatewayAPI\", \"GatewayAPIController\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MetricsCollectionProfiles\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"OnClusterBuild\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SetEIPForNLBIngressController\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ClusterVersionOperatorConfiguration\", \"DNSNameResolver\", \"DualReplica\", \"DyanmicServiceEndpointIBMCloud\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"Example2\", \"ExternalOIDCWithUIDAndExtraClaimMappings\", \"GCPClusterHostedDNS\", \"GCPCustomAPIEndpoints\", \"HighlyAvailableArbiter\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"KMSEncryptionProvider\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NewOLMCatalogdAPIV1Metas\", \"NewOLMOwnSingleNamespace\", \"NewOLMPreflightPermissionChecks\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"SELinuxChangePolicy\", \"SELinuxMount\", \"ShortCertRotation\", \"SignatureStores\", \"SigstoreImageVerification\", \"SigstoreImageVerificationPKI\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereConfigurableMaxAllowedBlockVolumesPerNode\", \"VSphereHostVMGroupZonal\", \"VSphereMultiDisk\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-07-02 11:46:55.510705752 +0000 UTC m=+0.553547696,LastTimestamp:2026-07-02 11:46:55.510705752 +0000 UTC m=+0.553547696,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W0702 11:46:55.515748 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0702 11:46:55.515772 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0702 11:46:55.515869 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0702 11:46:55.519841 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 11:46:55.519890 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0702 11:46:55.524034 1 secretconfigobserver.go:119] support secret does not exist I0702 11:46:55.527908 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 11:46:55.531601 1 secretconfigobserver.go:119] support secret does not exist I0702 11:46:55.535087 1 recorder.go:156] Pruning old reports every 8h25m23s, max age is 288h0m0s I0702 11:46:55.539383 1 periodic.go:212] Running clusterconfig gatherer I0702 11:46:55.539402 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0702 11:46:55.539411 1 insightsreport.go:296] Starting report retriever I0702 11:46:55.539419 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0702 11:46:55.539383 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0702 11:46:55.539431 1 tasks_processing.go:45] number of workers: 32 I0702 11:46:55.539439 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0702 11:46:55.539452 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 11:46:55.539467 1 tasks_processing.go:69] worker 4 listening for tasks. I0702 11:46:55.539470 1 tasks_processing.go:69] worker 6 listening for tasks. I0702 11:46:55.539473 1 tasks_processing.go:71] worker 4 working on metrics task. I0702 11:46:55.539470 1 tasks_processing.go:71] worker 0 working on monitoring_persistent_volumes task. I0702 11:46:55.539483 1 tasks_processing.go:69] worker 7 listening for tasks. I0702 11:46:55.539485 1 tasks_processing.go:69] worker 9 listening for tasks. I0702 11:46:55.539490 1 tasks_processing.go:69] worker 5 listening for tasks. I0702 11:46:55.539495 1 tasks_processing.go:69] worker 3 listening for tasks. I0702 11:46:55.539500 1 tasks_processing.go:69] worker 2 listening for tasks. I0702 11:46:55.539496 1 tasks_processing.go:69] worker 10 listening for tasks. I0702 11:46:55.539514 1 tasks_processing.go:71] worker 2 working on oauths task. I0702 11:46:55.539517 1 tasks_processing.go:71] worker 10 working on openstack_dataplanenodesets task. I0702 11:46:55.539524 1 tasks_processing.go:69] worker 13 listening for tasks. I0702 11:46:55.539527 1 tasks_processing.go:69] worker 23 listening for tasks. I0702 11:46:55.539536 1 tasks_processing.go:69] worker 24 listening for tasks. I0702 11:46:55.539536 1 tasks_processing.go:69] worker 18 listening for tasks. I0702 11:46:55.539542 1 tasks_processing.go:69] worker 19 listening for tasks. I0702 11:46:55.539510 1 tasks_processing.go:69] worker 22 listening for tasks. I0702 11:46:55.539549 1 tasks_processing.go:69] worker 14 listening for tasks. I0702 11:46:55.539549 1 tasks_processing.go:69] worker 25 listening for tasks. I0702 11:46:55.539557 1 tasks_processing.go:69] worker 20 listening for tasks. I0702 11:46:55.539556 1 tasks_processing.go:69] worker 26 listening for tasks. I0702 11:46:55.539535 1 tasks_processing.go:69] worker 17 listening for tasks. I0702 11:46:55.539569 1 tasks_processing.go:69] worker 15 listening for tasks. I0702 11:46:55.539571 1 tasks_processing.go:69] worker 28 listening for tasks. I0702 11:46:55.539573 1 tasks_processing.go:69] worker 16 listening for tasks. I0702 11:46:55.539563 1 tasks_processing.go:69] worker 27 listening for tasks. I0702 11:46:55.539577 1 tasks_processing.go:71] worker 6 working on openshift_logging task. I0702 11:46:55.539581 1 tasks_processing.go:71] worker 7 working on version task. I0702 11:46:55.539586 1 tasks_processing.go:69] worker 31 listening for tasks. I0702 11:46:55.539586 1 tasks_processing.go:69] worker 30 listening for tasks. I0702 11:46:55.539595 1 tasks_processing.go:71] worker 5 working on validating_webhook_configurations task. I0702 11:46:55.539508 1 tasks_processing.go:69] worker 21 listening for tasks. I0702 11:46:55.539606 1 tasks_processing.go:71] worker 30 working on overlapping_namespace_uids task. I0702 11:46:55.539611 1 tasks_processing.go:71] worker 21 working on nodes task. I0702 11:46:55.539516 1 tasks_processing.go:69] worker 12 listening for tasks. I0702 11:46:55.539580 1 tasks_processing.go:69] worker 29 listening for tasks. I0702 11:46:55.539474 1 tasks_processing.go:69] worker 8 listening for tasks. I0702 11:46:55.539656 1 tasks_processing.go:71] worker 25 working on silenced_alerts task. I0702 11:46:55.539669 1 tasks_processing.go:71] worker 8 working on olm_operators task. I0702 11:46:55.539829 1 tasks_processing.go:71] worker 3 working on container_runtime_configs task. I0702 11:46:55.539869 1 tasks_processing.go:71] worker 23 working on nodenetworkconfigurationpolicies task. I0702 11:46:55.539893 1 tasks_processing.go:71] worker 12 working on machine_configs task. I0702 11:46:55.540040 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 11:46:55.540107 1 tasks_processing.go:71] worker 28 working on container_images task. I0702 11:46:55.539863 1 tasks_processing.go:71] worker 13 working on jaegers task. I0702 11:46:55.539502 1 tasks_processing.go:69] worker 11 listening for tasks. I0702 11:46:55.540413 1 tasks_processing.go:71] worker 11 working on operators task. I0702 11:46:55.540110 1 tasks_processing.go:71] worker 1 working on cluster_apiserver task. I0702 11:46:55.539877 1 tasks_processing.go:71] worker 15 working on openstack_controlplanes task. I0702 11:46:55.539882 1 tasks_processing.go:71] worker 20 working on openshift_machine_api_events task. I0702 11:46:55.539885 1 tasks_processing.go:71] worker 26 working on cost_management_metrics_configs task. I0702 11:46:55.539888 1 tasks_processing.go:71] worker 17 working on operators_pods_and_events task. I0702 11:46:55.540109 1 tasks_processing.go:71] worker 29 working on install_plans task. I0702 11:46:55.540118 1 tasks_processing.go:71] worker 27 working on storage_classes task. I0702 11:46:55.540123 1 tasks_processing.go:71] worker 16 working on lokistack task. I0702 11:46:55.540126 1 tasks_processing.go:71] worker 24 working on image_pruners task. I0702 11:46:55.540129 1 tasks_processing.go:71] worker 31 working on node_logs task. I0702 11:46:55.540131 1 tasks_processing.go:71] worker 18 working on sap_pods task. I0702 11:46:55.540135 1 tasks_processing.go:71] worker 22 working on feature_gates task. I0702 11:46:55.540136 1 tasks_processing.go:71] worker 14 working on pdbs task. I0702 11:46:55.539577 1 tasks_processing.go:71] worker 9 working on machine_config_pools task. I0702 11:46:55.540139 1 tasks_processing.go:71] worker 19 working on config_maps task. I0702 11:46:55.544030 1 tasks_processing.go:71] worker 6 working on image task. I0702 11:46:55.544054 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 4.44144ms to process 0 records E0702 11:46:55.544869 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:44528->172.30.0.10:53: read: connection refused I0702 11:46:55.544887 1 tasks_processing.go:71] worker 25 working on active_alerts task. E0702 11:46:55.544900 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:32991->172.30.0.10:53: read: connection refused E0702 11:46:55.544909 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:44528->172.30.0.10:53: read: connection refused I0702 11:46:55.544930 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 5.224141ms to process 0 records E0702 11:46:55.544940 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:32991->172.30.0.10:53: read: connection refused I0702 11:46:55.544956 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 5.439968ms to process 0 records I0702 11:46:55.544976 1 tasks_processing.go:71] worker 4 working on openstack_dataplanedeployments task. E0702 11:46:55.548382 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:47540->172.30.0.10:53: read: connection refused I0702 11:46:55.548400 1 tasks_processing.go:71] worker 25 working on pod_network_connectivity_checks task. E0702 11:46:55.548413 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:47540->172.30.0.10:53: read: connection refused I0702 11:46:55.548430 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 3.504303ms to process 0 records I0702 11:46:55.549332 1 tasks_processing.go:71] worker 23 working on sap_datahubs task. I0702 11:46:55.549341 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 9.403535ms to process 0 records I0702 11:46:55.549375 1 tasks_processing.go:71] worker 13 working on machine_autoscalers task. I0702 11:46:55.549412 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 9.01177ms to process 0 records I0702 11:46:55.549465 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 9.529994ms to process 0 records I0702 11:46:55.549479 1 tasks_processing.go:71] worker 3 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0702 11:46:55.550344 1 tasks_processing.go:71] worker 10 working on dvo_metrics task. I0702 11:46:55.550357 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 10.817289ms to process 0 records I0702 11:46:55.550467 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 9.661599ms to process 0 records I0702 11:46:55.550491 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 9.48385ms to process 0 records I0702 11:46:55.550506 1 tasks_processing.go:71] worker 26 working on crds task. I0702 11:46:55.550616 1 tasks_processing.go:71] worker 2 working on openstack_version task. I0702 11:46:55.550689 1 tasks_processing.go:71] worker 15 working on machine_sets task. I0702 11:46:55.552849 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0702 11:46:55.552920 1 controller.go:317] The initial operator extension status is healthy I0702 11:46:55.552960 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0702 11:46:55.552990 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0702 11:46:55.553017 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0702 11:46:55.553278 1 controller.go:458] The operator is still being initialized I0702 11:46:55.553332 1 controller.go:481] The operator is healthy I0702 11:46:55.553870 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0702 11:46:55.554139 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s W0702 11:46:55.554431 1 operator.go:287] started I0702 11:46:55.554908 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0702 11:46:55.554955 1 recorder.go:70] Recording config/oauth with fingerprint=18a65bbdab23a7a065d8929150c3cbc01a15c5225eec3e8f529979eaa77087c7 I0702 11:46:55.554977 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 11.071478ms to process 1 records I0702 11:46:55.554989 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 10.585599ms to process 0 records I0702 11:46:55.554996 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 12.151517ms to process 0 records I0702 11:46:55.555004 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 13.427844ms to process 0 records I0702 11:46:55.555012 1 tasks_processing.go:71] worker 16 working on nodenetworkstates task. I0702 11:46:55.555248 1 tasks_processing.go:71] worker 0 working on ingress_certificates task. I0702 11:46:55.555290 1 recorder.go:70] Recording config/apiserver with fingerprint=a5bc7658a78bd70ea567378f522cf341792bb1e9af4a85eeafea14d083a16079 I0702 11:46:55.555292 1 tasks_processing.go:71] worker 20 working on machine_healthchecks task. I0702 11:46:55.555302 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 13.215207ms to process 1 records I0702 11:46:55.555385 1 tasks_processing.go:71] worker 1 working on sap_config task. I0702 11:46:55.555678 1 tasks_processing.go:71] worker 21 working on certificate_signing_requests task. I0702 11:46:55.555773 1 recorder.go:70] Recording config/node/ip-10-0-0-159.ec2.internal with fingerprint=291f73b91bab616252797a2f8432b368891f6a87358d8476646199512f3eda50 I0702 11:46:55.555859 1 recorder.go:70] Recording config/node/ip-10-0-1-122.ec2.internal with fingerprint=335beff0957b1ff5fc0fe3db3f2faa5a3950fd505f34b7fbe3079641065714a1 I0702 11:46:55.555937 1 recorder.go:70] Recording config/node/ip-10-0-2-220.ec2.internal with fingerprint=1c6b17be25b181212b6258e14298128ea99c3bbc6a34cdd66fa470d8aa8dad21 I0702 11:46:55.555951 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 14.227748ms to process 3 records I0702 11:46:55.556016 1 tasks_processing.go:71] worker 5 working on proxies task. I0702 11:46:55.556200 1 recorder.go:70] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=2788dc9b56bfa6be625f751d3649a89c46d22ced84d035ecf1f77d4657b999fa I0702 11:46:55.556279 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=d27720121b27ff2b50b40f7c1839747149c05f456b6d65e4f47cc0eefb5dfb47 I0702 11:46:55.556379 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=cb2ea5ec5f8c8a38221356cc56c60d871282aef8bd1d68fdaefc431785db867e I0702 11:46:55.556405 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=b3fd65357c9b944f11edd5e423ab037baba4156e6d0358efad087d5218964599 I0702 11:46:55.556438 1 recorder.go:70] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=c4063ea1fe62ee27804de52e95ab17c21da25028cb12e079ba2919188f68746d I0702 11:46:55.556470 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=2c2ca0bf798b1aa7f078694c3ee1465c55e1084c8329268bda2e091626056291 I0702 11:46:55.556513 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=4718ed313f2f12f90ea13b27ec84c51e956e1c696ab6d3da0b0e04be7c356ebe I0702 11:46:55.556544 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=55b33455eab6c45aeff1689d8c70fcce7a780470ed7c19086a673f04c308c281 I0702 11:46:55.556574 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=4e80c4a57577b9940f83da3eebba786be3a0e3f3cf7c30ad9e44d32ddea58df6 I0702 11:46:55.556622 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=e4dc20a578e176e3bcc81673f7e4ee641006e53f42f671d6255a552db0d1589c I0702 11:46:55.556655 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=bd1f863cb28016a669a37b62fc26796a3eb84799d6f1c10246405143d945e2c9 I0702 11:46:55.556723 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=d8a7ad1a1ac30a37351a460cc68a0d5886b7f44870f3ca20f9dfe176b16a5728 I0702 11:46:55.556769 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=c39d5a7104a098c9f846c393285637f89b068d20957a5ac21a75054f20f38f57 I0702 11:46:55.556784 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 15.400522ms to process 13 records I0702 11:46:55.556872 1 tasks_processing.go:71] worker 24 working on ceph_cluster task. I0702 11:46:55.557081 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=1b9ce8370f21ef803a8f8d23ceb21ba0abdeb75d8a2c19ee3de51691ef818170 I0702 11:46:55.557091 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 14.293542ms to process 1 records I0702 11:46:55.557110 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0702 11:46:55.557119 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 16.097731ms to process 1 records I0702 11:46:55.557127 1 tasks_processing.go:71] worker 30 working on service_accounts task. I0702 11:46:55.557553 1 tasks_processing.go:71] worker 31 working on tsdb_status task. I0702 11:46:55.557567 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 16.102532ms to process 0 records I0702 11:46:55.557592 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 15.651552ms to process 0 records I0702 11:46:55.557598 1 tasks_processing.go:71] worker 9 working on infrastructures task. I0702 11:46:55.557606 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 15.998261ms to process 0 records I0702 11:46:55.557614 1 tasks_processing.go:71] worker 18 working on image_registries task. I0702 11:46:55.557806 1 tasks_processing.go:71] worker 27 working on networks task. I0702 11:46:55.557903 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=d85b424130e3332381f45196e45423dc02b4e5d16c9e81d578d1240d0a824f39 I0702 11:46:55.557934 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=947f7baf2705137ea3490bddf3de321f01d283230d068dde07a82cdfdf517f7a I0702 11:46:55.557946 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 16.531492ms to process 2 records I0702 11:46:55.558583 1 tasks_processing.go:71] worker 12 working on authentication task. E0702 11:46:55.558587 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0702 11:46:55.558734 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0702 11:46:55.558771 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 18.590853ms to process 1 records E0702 11:46:55.558812 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0702 11:46:55.558839 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 10.172732ms to process 0 records I0702 11:46:55.558852 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 13.56874ms to process 0 records I0702 11:46:55.558860 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 9.280927ms to process 0 records I0702 11:46:55.558868 1 tasks_processing.go:71] worker 23 working on aggregated_monitoring_cr_names task. I0702 11:46:55.558899 1 tasks_processing.go:71] worker 4 working on support_secret task. I0702 11:46:55.558955 1 tasks_processing.go:71] worker 25 working on schedulers task. I0702 11:46:55.559474 1 tasks_processing.go:71] worker 13 working on mutating_webhook_configurations task. I0702 11:46:55.559496 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 10.085265ms to process 0 records I0702 11:46:55.559732 1 tasks_processing.go:71] worker 22 working on clusterroles task. I0702 11:46:55.559892 1 recorder.go:70] Recording config/featuregate with fingerprint=69edddf36b1c7fa14488f53a6dfb9df78e2abf2c1e6a45d729ce9892003b1de0 I0702 11:46:55.559911 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 18.086777ms to process 1 records E0702 11:46:55.561923 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:48415->172.30.0.10:53: read: connection refused I0702 11:46:55.561943 1 tasks_processing.go:71] worker 31 working on storage_cluster task. E0702 11:46:55.561956 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:48415->172.30.0.10:53: read: connection refused I0702 11:46:55.561972 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.374075ms to process 0 records I0702 11:46:55.566660 1 tasks_processing.go:71] worker 2 working on ingress task. I0702 11:46:55.566662 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 16.027397ms to process 0 records I0702 11:46:55.566697 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 15.965151ms to process 0 records I0702 11:46:55.566704 1 tasks_processing.go:71] worker 15 working on machines task. I0702 11:46:55.572317 1 tasks_processing.go:74] worker 14 stopped. I0702 11:46:55.572413 1 recorder.go:70] Recording config/pdbs/openshift-console/console with fingerprint=6426c0ee5c4c3e1f8c1c71e58547835dee6f16d50aad912800cccf93a75c6d90 I0702 11:46:55.572438 1 recorder.go:70] Recording config/pdbs/openshift-console/downloads with fingerprint=e4208f5506a4eb6b78053b98f6750700ee3d6dadc4d82a7ee5e72805c348dc0b I0702 11:46:55.572458 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=e8eb271e7774693023aa814f3eafbef2736067d71f4a9c40246ab46f1a9e5e1f I0702 11:46:55.572476 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=575e8abd850deff231d22bbea3e0a545370449bc107ba6472b6417551d51b305 I0702 11:46:55.572494 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=eedb030c720ee8376154a41196a390a674c540547561eea3689094614d974910 I0702 11:46:55.572513 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=11912fa18e68d206eaa132b2cf14df265f09cd38338540cc12ea5bb33bb0acec I0702 11:46:55.572535 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=9c145965151a5af6479d86dea3bad7d7a972d3f1ec8f01ccf492926145700807 I0702 11:46:55.572559 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=bb74d075547c2853cc06a0908c3250cea363bfb3034dffced3f9f21b869c5bd4 I0702 11:46:55.572576 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=81dc0b5b1845da0b4f626117aa451a75b4f5842b5a2ba9fffccb243402960d73 I0702 11:46:55.572608 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=a42994de7ca03e8286bdcb43bd943c2aa9fdcd683216610c33b08369d7bb5f39 I0702 11:46:55.572617 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 30.564336ms to process 10 records I0702 11:46:55.572700 1 tasks_processing.go:74] worker 6 stopped. I0702 11:46:55.572703 1 recorder.go:70] Recording config/image with fingerprint=d554a88dd57e5b02211aa7fe192c6c4eaf8fc85a911519e24078390059648b4d I0702 11:46:55.572720 1 gather.go:177] gatherer "clusterconfig" function "image" took 28.365222ms to process 1 records I0702 11:46:55.573901 1 tasks_processing.go:74] worker 16 stopped. I0702 11:46:55.573922 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 18.873723ms to process 0 records I0702 11:46:55.574690 1 tasks_processing.go:74] worker 1 stopped. I0702 11:46:55.574703 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 19.254696ms to process 0 records E0702 11:46:55.574741 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0702 11:46:55.574748 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 19.411772ms to process 0 records I0702 11:46:55.574755 1 tasks_processing.go:74] worker 20 stopped. I0702 11:46:55.575030 1 tasks_processing.go:74] worker 5 stopped. I0702 11:46:55.575972 1 recorder.go:70] Recording config/proxy with fingerprint=5ade45deba3bebd9826b4a93716d725c2be8f7c384a0ca16d3b22f8c5dbd5214 I0702 11:46:55.576062 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 18.99952ms to process 1 records I0702 11:46:55.576657 1 tasks_processing.go:74] worker 7 stopped. I0702 11:46:55.577935 1 recorder.go:70] Recording config/version with fingerprint=61e6caa30f93a6d2bfff05f862fb4799573c644f9e313b462d70fce4b009046b I0702 11:46:55.577997 1 recorder.go:70] Recording config/id with fingerprint=fbb4f2012d5b50ee492a3b17087be259b21435870e386b0e4ce41c95b84abdc5 I0702 11:46:55.578015 1 gather.go:177] gatherer "clusterconfig" function "version" took 37.065698ms to process 2 records E0702 11:46:55.578034 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0702 11:46:55.578042 1 gather.go:177] gatherer "clusterconfig" function "machines" took 10.633722ms to process 0 records E0702 11:46:55.578052 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0702 11:46:55.578055 1 tasks_processing.go:74] worker 15 stopped. I0702 11:46:55.578062 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 18.783787ms to process 0 records I0702 11:46:55.578162 1 tasks_processing.go:74] worker 4 stopped. I0702 11:46:55.578269 1 tasks_processing.go:74] worker 12 stopped. I0702 11:46:55.578444 1 recorder.go:70] Recording config/authentication with fingerprint=928f117f36c27a3d2b500b41d5aeb99f18ce82a20fde175d39c78af030e12830 I0702 11:46:55.578462 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 19.239252ms to process 1 records I0702 11:46:55.578654 1 tasks_processing.go:74] worker 25 stopped. I0702 11:46:55.578666 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=43cbbe62c68ab092b7a3741d825aa6da0b5e36ab05057738a2178cf28f426117 I0702 11:46:55.578763 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 19.067145ms to process 1 records I0702 11:46:55.578941 1 tasks_processing.go:74] worker 2 stopped. I0702 11:46:55.579277 1 recorder.go:70] Recording config/ingress with fingerprint=8f300ddf699c6672a70dea1812585831c2d9601ce80bc664ad0daff6b7f39c20 I0702 11:46:55.579342 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 11.413313ms to process 1 records I0702 11:46:55.579395 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 21.577184ms to process 0 records I0702 11:46:55.579480 1 tasks_processing.go:74] worker 24 stopped. I0702 11:46:55.579528 1 tasks_processing.go:74] worker 9 stopped. I0702 11:46:55.581058 1 recorder.go:70] Recording config/infrastructure with fingerprint=24845748acc6a748fa784e26c6de3f26209b8ab7db2a4fdd137aa7f2053c675f I0702 11:46:55.581120 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 21.667916ms to process 1 records I0702 11:46:55.581366 1 recorder.go:70] Recording config/network with fingerprint=2c502579fa4207f5376accda8ee0caea41fee3fc18d9a9ca20b7ce37fdf1ea15 I0702 11:46:55.581410 1 gather.go:177] gatherer "clusterconfig" function "networks" took 21.690903ms to process 1 records I0702 11:46:55.581441 1 tasks_processing.go:74] worker 27 stopped. W0702 11:46:55.581651 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 11:46:55.581703 1 tasks_processing.go:74] worker 31 stopped. I0702 11:46:55.581720 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 19.750875ms to process 0 records I0702 11:46:55.582533 1 tasks_processing.go:74] worker 13 stopped. I0702 11:46:55.582720 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=349ae00a46c6aa37f21024218e787d08aa377241cb4bd35c3a1485d6a4325309 I0702 11:46:55.582788 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=f112806b2a11eea842cfb1cde4f3e7287fb0abeb339c4000d5d0611e4ef351b0 I0702 11:46:55.582877 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=559d57520f1080ead89f44a3bc7737c0aa8b2461110a171666e430def00e7d8c I0702 11:46:55.582908 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 23.044098ms to process 3 records I0702 11:46:55.583033 1 recorder.go:70] Recording config/olm_operators with fingerprint=53797183f718583ff4f2c346ea0179c89b9afd7a4b74f0cb93a47bac92eaa89c I0702 11:46:55.583048 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 42.881957ms to process 1 records I0702 11:46:55.583296 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0702 11:46:55.583351 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0702 11:46:55.583361 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0702 11:46:55.583369 1 controller.go:203] Source scaController *sca.Controller is not ready I0702 11:46:55.583376 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0702 11:46:55.583401 1 controller.go:458] The operator is still being initialized I0702 11:46:55.583410 1 controller.go:481] The operator is healthy I0702 11:46:55.583434 1 tasks_processing.go:74] worker 18 stopped. I0702 11:46:55.583328 1 tasks_processing.go:74] worker 8 stopped. I0702 11:46:55.583579 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=5479afc022850d0cf5fc11df0138b4efb01845b77dc8da0adf3c3fb9f7151bad I0702 11:46:55.583595 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 25.326745ms to process 1 records I0702 11:46:55.584095 1 tasks_processing.go:74] worker 21 stopped. I0702 11:46:55.584113 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 28.350116ms to process 0 records E0702 11:46:55.593589 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27db0d3d3f-418f-4886-b820-640f093a78f8%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:35425->172.30.0.10:53: read: connection refused I0702 11:46:55.593608 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27db0d3d3f-418f-4886-b820-640f093a78f8%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:35425->172.30.0.10:53: read: connection refused I0702 11:46:55.593806 1 tasks_processing.go:74] worker 22 stopped. I0702 11:46:55.594064 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=e94702662155b666525e0e47c119621e53756d340df1884f5916dfcad3ebdd04 I0702 11:46:55.594222 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=011df4642584a213f145b5b4d5a191a04830e8258027ab3311e9f75a3280b1f8 I0702 11:46:55.594234 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 34.059532ms to process 2 records I0702 11:46:55.599312 1 tasks_processing.go:74] worker 26 stopped. I0702 11:46:55.600058 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=5d5df7c0ae3e62d6264488ae8bee86becfe86d991449f1335f128fed64bb6cd4 I0702 11:46:55.600321 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=d6af94050319f75e2a5477ef6a07610d2c8ca9b0997f0f52b324ad48bf4a77a9 I0702 11:46:55.600340 1 gather.go:177] gatherer "clusterconfig" function "crds" took 48.789632ms to process 2 records I0702 11:46:55.600355 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 41.044926ms to process 0 records I0702 11:46:55.600367 1 tasks_processing.go:74] worker 23 stopped. I0702 11:46:55.601351 1 tasks_processing.go:74] worker 28 stopped. I0702 11:46:55.602035 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0702 11:46:55.603260 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm with fingerprint=486a3ce1203d36dd698092e666185d6fbf85f7f82bea1bca45e89d8e0d5cb803 I0702 11:46:55.603502 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-qgtjj with fingerprint=ce32140d435afc03c8e99bcdd23d4cc202aba98153fdf2fe43f5abd30cef8000 I0702 11:46:55.603633 1 recorder.go:70] Recording config/running_containers with fingerprint=5296f2d6e0b52796c71215f1ab15a0ab3d11817213d0f49e24f143a9e12f0274 I0702 11:46:55.603651 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 61.179212ms to process 3 records W0702 11:46:55.605791 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:47385->172.30.0.10:53: read: connection refused I0702 11:46:55.605805 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:47385->172.30.0.10:53: read: connection refused I0702 11:46:55.606075 1 tasks_processing.go:74] worker 3 stopped. I0702 11:46:55.606088 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 56.58712ms to process 0 records I0702 11:46:55.608048 1 tasks_processing.go:74] worker 19 stopped. E0702 11:46:55.608062 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0702 11:46:55.608068 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0702 11:46:55.608071 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0702 11:46:55.608080 1 recorder.go:70] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=295a2bc7810c628251f1f312c209b5eb4da888326706c635ac0c12aec0969e17 I0702 11:46:55.608122 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0702 11:46:55.608133 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0702 11:46:55.608138 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=fe7abfa7f7aea852d8bca6b7df9b4d5de32045254ff2774f9bd30f0b6dcb7dc4 I0702 11:46:55.608148 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0702 11:46:55.608209 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0702 11:46:55.608218 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0702 11:46:55.608223 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 66.073218ms to process 7 records I0702 11:46:55.608802 1 tasks_processing.go:74] worker 0 stopped. E0702 11:46:55.608813 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0702 11:46:55.608823 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2r9viipd2t2o1jdilrhipalebtm49bmv-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2r9viipd2t2o1jdilrhipalebtm49bmv-primary-cert-bundle-secret" not found I0702 11:46:55.608881 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=0ef7b26da8152365b61c9edb83c8c3fc12565b076867bb316e528aeab296be21 I0702 11:46:55.608894 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 53.537957ms to process 1 records I0702 11:46:55.611418 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0702 11:46:55.611429 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0702 11:46:55.611434 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0702 11:46:55.616117 1 base_controller.go:82] Caches are synced for ConfigController I0702 11:46:55.616130 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0702 11:46:55.620172 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 11:46:55.656019 1 base_controller.go:82] Caches are synced for LoggingSyncer I0702 11:46:55.656033 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0702 11:46:56.397985 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" W0702 11:46:56.578942 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 11:46:56.598240 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0702 11:46:56.789013 1 gather_cluster_operator_pods_and_events.go:121] Found 24 pods with 27 containers I0702 11:46:56.789029 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 932067 bytes I0702 11:46:56.789236 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-85b84f584d-mn8sm pod in namespace openshift-console-operator (previous: true). I0702 11:46:57.015139 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-85b84f584d-mn8sm pod in namespace openshift-console-operator (previous: false). I0702 11:46:57.198425 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-8dd7dc4f4-9sdkp pod in namespace openshift-console (previous: false). I0702 11:46:57.394340 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-8dd7dc4f4-khzl8 pod in namespace openshift-console (previous: false). I0702 11:46:57.399218 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0702 11:46:57.578353 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 11:46:57.610352 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-bb847466f-bfdfp pod in namespace openshift-console (previous: false). I0702 11:46:57.808175 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-7b98f8c767-559k2 pod in namespace openshift-console (previous: false). I0702 11:46:57.991944 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-7b98f8c767-s57jh pod in namespace openshift-console (previous: false). I0702 11:46:58.000677 1 tasks_processing.go:74] worker 11 stopped. I0702 11:46:58.000814 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=569968043f8e1dae134daca88644d9ea5483ea33b48339327203711327693ce6 I0702 11:46:58.000865 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0702 11:46:58.000897 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=3f245ff7d4bdf90bd5407e6017bf663075212547824344a421431302db30b549 I0702 11:46:58.000908 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0702 11:46:58.000932 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=650e00aac4a4ac8a2423c939268e0e89dee799c16a78bc913e0f30eccd79c5c7 I0702 11:46:58.000953 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0702 11:46:58.000981 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=9bd59e88d42561650e3da32c121a5131b9c3994ca9403954a75a4fcbcc6dd94b I0702 11:46:58.001009 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=93977a945225ec723d391fce5c886a8bbad864f7f0826194bd7be0708c07953a I0702 11:46:58.001041 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=b590604f246a0239422ac60f5493b43195801670d48f95702eddd1de3f7c19c5 I0702 11:46:58.001074 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=52091d77f10a08c6fb2ee0ec23a78c6895e3c2aff9e7f8e7432b1652a794123a I0702 11:46:58.001423 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e56866a28883a7d7f7ced90d9713ceeab3dfa200cadf8cc2b0ee7e942f30cb6a I0702 11:46:58.001450 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=eb91f4d26fad989afe1b955f0780906385fc9149d85957a30745db75933b0083 I0702 11:46:58.001461 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0702 11:46:58.001478 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=d19346625df4521c9e7490af8245b37924c7a4d2451c013193d4d3cc00c9764a I0702 11:46:58.001490 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0702 11:46:58.001512 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=d0a5c2401e3eba3149be9906486395e584f868116eba6a7a3a53a351aede2e2c I0702 11:46:58.001525 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0702 11:46:58.001545 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=3134b22d5c3148278b88b97ded8ca7d8a29e636e13752ab97b8d007c8361b1bd I0702 11:46:58.001556 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0702 11:46:58.001580 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=a568e174b478d5f5e39f8cd11488105a20ba19539d0dc9d1ca9dfee888485a8e I0702 11:46:58.001694 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=614c0d0f9a4cbd4db03f86b1589ba01eeeded6cabbb9eaef5d035be6793462ad I0702 11:46:58.001729 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0702 11:46:58.001758 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=4f56d5db185029de0ebaf829575faa70c11017f1f417a8b6638111abf585589d I0702 11:46:58.001783 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=e5540027ebf30d56e9974503f3b66e8cf9344a7c095e2248ba0706c69bc9c0a1 I0702 11:46:58.001795 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0702 11:46:58.001812 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=ceb6813bca90b5666ca4962026286fc94eaa1b5c976003729be46b75d36edc31 I0702 11:46:58.001827 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0702 11:46:58.001845 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=5b015f28aea8edaca1dc89c1852ccae48b9a3fc9cb9e35761c13069e8afcdaa6 I0702 11:46:58.001861 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=bff9f49907e17a607c6ee9eca2e2342078b0184fa8fd2bc25dbab33a2162395d I0702 11:46:58.001877 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=e079f8bf5e0418cba574e6d95dab14c645df577f88cf9edee680b98e04842692 I0702 11:46:58.001898 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=753eda47bb2aa7be3d8b1d172af0e93a11a44d2c9117fed0d7dff68885c2509e I0702 11:46:58.001939 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=9527ddb8ae147fdaae70bceebf5c1cba1c6b07b2236f2ab0ac14ab8e4aa1cae2 I0702 11:46:58.001955 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0702 11:46:58.001990 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=b9956e217b3efa0ce591536dc39ce982927a379246deeeecd1467793c56a095e I0702 11:46:58.002010 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0702 11:46:58.002019 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0702 11:46:58.002027 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.46024882s to process 36 records I0702 11:46:58.197263 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-hh99n pod in namespace openshift-dns (previous: false). I0702 11:46:58.394167 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-hh99n pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-hh99n\" is waiting to start: ContainerCreating" I0702 11:46:58.394205 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-hh99n\" is waiting to start: ContainerCreating" I0702 11:46:58.394216 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-hh99n pod in namespace openshift-dns (previous: false). W0702 11:46:58.578272 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 11:46:58.593063 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-hh99n pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-hh99n\" is waiting to start: ContainerCreating" I0702 11:46:58.593082 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-hh99n\" is waiting to start: ContainerCreating" I0702 11:46:58.593113 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-t6tjk pod in namespace openshift-dns (previous: false). I0702 11:46:58.791387 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-t6tjk pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-t6tjk\" is waiting to start: ContainerCreating" I0702 11:46:58.791406 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-t6tjk\" is waiting to start: ContainerCreating" I0702 11:46:58.791415 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-t6tjk pod in namespace openshift-dns (previous: false). I0702 11:46:58.991358 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-t6tjk pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-t6tjk\" is waiting to start: ContainerCreating" I0702 11:46:58.991377 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-t6tjk\" is waiting to start: ContainerCreating" I0702 11:46:58.991407 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-whrj7 pod in namespace openshift-dns (previous: false). I0702 11:46:59.192314 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-whrj7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-whrj7\" is waiting to start: ContainerCreating" I0702 11:46:59.192334 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-whrj7\" is waiting to start: ContainerCreating" I0702 11:46:59.192344 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-whrj7 pod in namespace openshift-dns (previous: false). I0702 11:46:59.391918 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-whrj7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-whrj7\" is waiting to start: ContainerCreating" I0702 11:46:59.391936 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-whrj7\" is waiting to start: ContainerCreating" I0702 11:46:59.391950 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-kr8lm pod in namespace openshift-dns (previous: false). W0702 11:46:59.578097 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 11:46:59.594433 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 11:46:59.594448 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-n9kd2 pod in namespace openshift-dns (previous: false). I0702 11:46:59.792478 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 11:46:59.792496 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-zd7rp pod in namespace openshift-dns (previous: false). I0702 11:46:59.992368 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 11:46:59.992384 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6b6658f55c-d7lb8 pod in namespace openshift-image-registry (previous: false). I0702 11:47:00.193929 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6b6658f55c-xgt9v pod in namespace openshift-image-registry (previous: false). I0702 11:47:00.394218 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-b6bd5b5d9-k4wms pod in namespace openshift-image-registry (previous: false). W0702 11:47:00.578397 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0702 11:47:00.578422 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0702 11:47:00.578439 1 tasks_processing.go:74] worker 10 stopped. E0702 11:47:00.578448 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0702 11:47:00.578461 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0702 11:47:00.578470 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.028027073s to process 1 records I0702 11:47:00.592328 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-48mxx pod in namespace openshift-image-registry (previous: false). I0702 11:47:00.794790 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-6d9sj pod in namespace openshift-image-registry (previous: false). I0702 11:47:00.993095 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-b6wfk pod in namespace openshift-image-registry (previous: false). I0702 11:47:01.194659 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-69cd788b57-gcbs8 pod in namespace openshift-ingress (previous: false). I0702 11:47:01.394587 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7995c9cd99-jkpnj pod in namespace openshift-ingress (previous: false). I0702 11:47:01.592349 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7995c9cd99-knss5 pod in namespace openshift-ingress (previous: false). I0702 11:47:01.795141 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dd95b pod in namespace openshift-ingress-canary (previous: false). I0702 11:47:01.991307 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dd95b pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dd95b\" is waiting to start: ContainerCreating" I0702 11:47:01.991327 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dd95b\" is waiting to start: ContainerCreating" I0702 11:47:01.991357 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-ghsb8 pod in namespace openshift-ingress-canary (previous: false). I0702 11:47:02.192219 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-ghsb8 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-ghsb8\" is waiting to start: ContainerCreating" I0702 11:47:02.192236 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-ghsb8\" is waiting to start: ContainerCreating" I0702 11:47:02.192265 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-v6w9j pod in namespace openshift-ingress-canary (previous: false). I0702 11:47:02.393211 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-v6w9j pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-v6w9j\" is waiting to start: ContainerCreating" I0702 11:47:02.393230 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-v6w9j\" is waiting to start: ContainerCreating" I0702 11:47:02.393248 1 tasks_processing.go:74] worker 17 stopped. I0702 11:47:02.393440 1 recorder.go:70] Recording events/openshift-console-operator with fingerprint=2a1ded8291b19e76de1ffbd2d3adaf77293bc7e8d3bef9e39d09d9d84a66d908 I0702 11:47:02.393548 1 recorder.go:70] Recording events/openshift-console with fingerprint=1dd33de0b1455ff5eae57e1ed8df330eb747ab23b0d2c3881f39d7f89151a8f3 I0702 11:47:02.393573 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=04c28673b5cd8a41c66639e4f176c2aa5625e2876409a72106b1fb7afd834758 I0702 11:47:02.393619 1 recorder.go:70] Recording events/openshift-dns with fingerprint=db6f4f541a4fc48310a13494cedc26cf22c2b2a77c617a706597265e20a6d3ae I0702 11:47:02.393774 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=a0c6ce817184a4ec49eba9690039e481f768e26fcbc62f8a8dc3178335d1eb15 I0702 11:47:02.393808 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=6423c757f8c73ce441bd46e88200f1b9a290201c8ce6fbd77407f0e83928fc3a I0702 11:47:02.393881 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=80988449f63c0266ae2983cdc030f8b758b484c0472f13b77d99df1aaeef8ca1 I0702 11:47:02.393899 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=2729bc5f841136ac8640cca39f102e7948c1312de01e941a4468d2248bd36dbb I0702 11:47:02.394029 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm with fingerprint=486a3ce1203d36dd698092e666185d6fbf85f7f82bea1bca45e89d8e0d5cb803 E0702 11:47:02.394046 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json" was already recorded and had the fingerprint "486a3ce1203d36dd698092e666185d6fbf85f7f82bea1bca45e89d8e0d5cb803", overwriting with the record having fingerprint "486a3ce1203d36dd698092e666185d6fbf85f7f82bea1bca45e89d8e0d5cb803" W0702 11:47:02.394056 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json" because of the warning: warning: the record with the same fingerprint "486a3ce1203d36dd698092e666185d6fbf85f7f82bea1bca45e89d8e0d5cb803" was already recorded at path "config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json" I0702 11:47:02.394067 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-85b84f584d-mn8sm/console-operator_previous.log with fingerprint=083c012f872cf8ecf6390cd72d78db8754c7c80aa0538a4f78566b8eab671042 I0702 11:47:02.394283 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-85b84f584d-mn8sm/console-operator_current.log with fingerprint=1de0bbde0ecfd2a17b20919262fcaefe1387a6ac06d42f2406c87dc20b1b2dbb I0702 11:47:02.394297 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-8dd7dc4f4-9sdkp/console_current.log with fingerprint=0949349070cf546c8618d835c8d0da81ff0e1bb723ee2f89ca06d3735b057ed8 I0702 11:47:02.394307 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-8dd7dc4f4-khzl8/console_current.log with fingerprint=fef83b67d809ea1ac6e695d4b60fdf42340c4dfa5eebbb23ec82023de00340c4 I0702 11:47:02.394325 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-bb847466f-bfdfp/console_current.log with fingerprint=95427663fa7b7af5469cece60d0ca1aa63a47ee962f1072e9abf069f947176a1 I0702 11:47:02.394333 1 recorder.go:70] Recording config/pod/openshift-console/logs/downloads-7b98f8c767-559k2/download-server_current.log with fingerprint=7968a40106430c09247240b5b53c5894d87b6b711ce0648c546357f34026241a I0702 11:47:02.394340 1 recorder.go:70] Recording config/pod/openshift-console/logs/downloads-7b98f8c767-s57jh/download-server_current.log with fingerprint=fae3627570ce8d31040a094d8c72f81aecd13acd8f39063219d2f35602ac9db7 I0702 11:47:02.394433 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-hh99n with fingerprint=ccca651ccad4e246dcc338faa017856cb7317292d589a01f1294e173fe6d876c I0702 11:47:02.394503 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-t6tjk with fingerprint=2fb310038eca6408058ec3bc3bfe07d80ea001f1e1b7cb6f77ed684ec635f197 I0702 11:47:02.394570 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-whrj7 with fingerprint=1c9331e6b45583cd1fec27b57b9a8a8d4cfc6820ee2d9995ac6963c154ededb7 I0702 11:47:02.394590 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-6b6658f55c-d7lb8/registry_current.log with fingerprint=0eb1a737bd48107f9949b03abbc5fc7283ff982450ae546d37af1c01ddc9a38b I0702 11:47:02.394620 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-6b6658f55c-xgt9v/registry_current.log with fingerprint=0f257a7137f7ff11edf4caa312b5793b0b763ceff110637a6fba330aecad33bc I0702 11:47:02.394632 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-b6bd5b5d9-k4wms/registry_current.log with fingerprint=cc56e90d6c9a40d3dceb0b76ebbb81290a12261fe0bc02e2a9fcc6d031654eb9 I0702 11:47:02.394638 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/node-ca-48mxx/node-ca_current.log with fingerprint=8ee35a25260adac8ed1beef48f59c4e1996a2ac02ee00fa9331db67ff48f29f4 I0702 11:47:02.394643 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/node-ca-6d9sj/node-ca_current.log with fingerprint=734237cc33bfe92ae3e027b4a4e571c3745055e9ab5d21f7d2a57f8bdf596da3 I0702 11:47:02.394648 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/node-ca-b6wfk/node-ca_current.log with fingerprint=47d8a441ead638ec0dfe61ced0ef27835d0738ed0d4b89c3ebc180fb940789ce I0702 11:47:02.394674 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-69cd788b57-gcbs8/router_current.log with fingerprint=c955acde6b033ffd85fd7fdc99618e08c26105cf1e450ffb3e53401e8fb21a3f I0702 11:47:02.394706 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-7995c9cd99-jkpnj/router_current.log with fingerprint=743341e432f9e052c0e7348f8b6be9dbc7214d10c2a75b1332706eab65d27908 I0702 11:47:02.394738 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-7995c9cd99-knss5/router_current.log with fingerprint=764069e229c41ec56a5e549dc53088006f5bd5fcccac2b0e534b30128cd1475f I0702 11:47:02.394795 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-dd95b with fingerprint=0f674b90ab1c1e7b36f5dc1379c5bd8c3528a9f7d6942b3c6fcb282e3239b38c I0702 11:47:02.394850 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-ghsb8 with fingerprint=a23fc19638e950211182c70b0d8449ed288c9310514fdd88cf08aba225bd2ac9 I0702 11:47:02.394903 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-v6w9j with fingerprint=1a69e188fb8f18257e37f3f9f9ce06c3f3999129bbf98a6b1b70befd6bf5089d I0702 11:47:02.394911 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.852291054s to process 31 records I0702 11:47:08.589565 1 tasks_processing.go:74] worker 29 stopped. I0702 11:47:08.589621 1 recorder.go:70] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0702 11:47:08.589635 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.048527701s to process 1 records I0702 11:47:09.362531 1 tasks_processing.go:74] worker 30 stopped. I0702 11:47:09.362842 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=bc495a7b9680b945cbf764552ab96dea7b445a47c6f80188881acd3d207a865a I0702 11:47:09.362859 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.805386684s to process 1 records E0702 11:47:09.362944 1 periodic.go:250] "Unhandled Error" err="clusterconfig failed after 13.823s with: function \"silenced_alerts\" failed with an error, function \"metrics\" failed with an error, function \"active_alerts\" failed with an error, function \"machine_configs\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"tsdb_status\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json\"" I0702 11:47:09.364048 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "silenced_alerts" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "machine_configs" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "tsdb_status" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-85b84f584d-mn8sm.json" I0702 11:47:09.364063 1 periodic.go:212] Running workloads gatherer I0702 11:47:09.364079 1 tasks_processing.go:45] number of workers: 2 I0702 11:47:09.364090 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 11:47:09.364095 1 tasks_processing.go:71] worker 1 working on workload_info task. I0702 11:47:09.364111 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 11:47:09.364200 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0702 11:47:09.369915 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-122.ec2.internal... I0702 11:47:09.369921 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-159.ec2.internal... I0702 11:47:09.369932 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-220.ec2.internal... I0702 11:47:09.387351 1 tasks_processing.go:74] worker 0 stopped. I0702 11:47:09.387369 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 23.123328ms to process 0 records I0702 11:47:13.593828 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0702 11:47:13.616133 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0702 11:47:13.626344 1 gather_workloads_info.go:387] No image sha256:a7c71f3c9714cf1717c45e12ec817be6d6209f00989e6a31d634e527f0c8147d (11ms) I0702 11:47:13.636289 1 gather_workloads_info.go:387] No image sha256:bae7a33f8db8a3d4b3c4c05498aba85a0ce463f85322067c48e79663710e616e (10ms) I0702 11:47:13.645955 1 gather_workloads_info.go:387] No image sha256:9697bc2258bdfa9ae8c1866cc7eb0b3b46851998db827b121bdf77417a881eb3 (10ms) I0702 11:47:13.655677 1 gather_workloads_info.go:387] No image sha256:a9e885d6f0456a2cc10f9e5da71fe9403f5e4c639b9b7ea15bd403b272ccb824 (10ms) I0702 11:47:13.665572 1 gather_workloads_info.go:387] No image sha256:7941b5d9a758b8667e99cd9236f7f96ec036af61df5cbc96e16077d36700d7c7 (10ms) I0702 11:47:13.675759 1 gather_workloads_info.go:387] No image sha256:87efe06afd1f04426fca7f86c0f74c4ee75c311ba199ddabd5b849b877bc59fa (10ms) I0702 11:47:13.685029 1 gather_workloads_info.go:387] No image sha256:944d9261ba7a143131fe8267c172defc5f37acc1cea3d4d373ec6fc5d8bfcc31 (9ms) I0702 11:47:13.694295 1 gather_workloads_info.go:387] No image sha256:7854174b4b98e9b10d9c8e9b43113df1217ab21db4b2395d9399af0c3d7911a3 (9ms) I0702 11:47:13.703635 1 gather_workloads_info.go:387] No image sha256:14703f73bb1ca69ea03b726e340b3b68f8e294e1f22c80d1a16666ea3d4a88a3 (9ms) I0702 11:47:13.713141 1 gather_workloads_info.go:387] No image sha256:23c4fe84047a4e6cbe7f75f470e7d6fe0e61e7910dd17945b1b61bc4b72f3f2a (9ms) I0702 11:47:13.724463 1 gather_workloads_info.go:387] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (11ms) I0702 11:47:13.826882 1 gather_workloads_info.go:387] No image sha256:3d0634fe58641d5242649c44ddba70ed67fdd6d2dcb5c2261df5cee8b33de9fd (102ms) I0702 11:47:13.925335 1 gather_workloads_info.go:387] No image sha256:4f175ee49f51dc4379e9993fecd1657c7a9c4c3dc096772cb198d0212b9eea47 (98ms) I0702 11:47:14.026449 1 gather_workloads_info.go:387] No image sha256:420c12f61c53e54eab2d99476759c0de339fca98ce0a3a782bc6545cc0e97a9c (101ms) I0702 11:47:14.125885 1 gather_workloads_info.go:387] No image sha256:527ed405bf66234718321996dacf51f91c64c746de9f1509416ef6ce89c90a50 (99ms) I0702 11:47:14.225434 1 gather_workloads_info.go:387] No image sha256:6277dac65decba0ea094146bb99600f479eaf804696842e724de1f1b436f9a38 (100ms) I0702 11:47:14.325792 1 gather_workloads_info.go:387] No image sha256:a3d34d8f38a540ce79d34a5248b1965346155497db57b72cb4758d659418059c (100ms) I0702 11:47:14.425011 1 gather_workloads_info.go:387] No image sha256:f2c46054a8f64a0e949cbf30295f31b4a35a0203c6ad03fa7ec922b4101dcbf8 (99ms) I0702 11:47:14.526161 1 gather_workloads_info.go:387] No image sha256:ceab8a8c340801fcebf800620d3fef5493d8a443077a21c6e56984051bd3abde (101ms) I0702 11:47:14.626295 1 gather_workloads_info.go:387] No image sha256:752be69b2262be713df12c47f4bac8c2dafed272c401e9a89f8060f053d68054 (100ms) I0702 11:47:14.726317 1 gather_workloads_info.go:387] No image sha256:6c3c7f77d7a3815bbb69cad5f6f1742c17d9597bc135ba3d116c9736d98fae33 (100ms) I0702 11:47:14.825727 1 gather_workloads_info.go:387] No image sha256:824a3b8f78e19aa21a9f6444aefe8e0b624886cca18fe828b13acbab55e6e868 (99ms) I0702 11:47:14.927303 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (102ms) I0702 11:47:15.025910 1 gather_workloads_info.go:387] No image sha256:ccdda82b4adfa8ee4a84b3bad19693be6a42a9881a7b83788d835912129c49a4 (99ms) I0702 11:47:15.125488 1 gather_workloads_info.go:387] No image sha256:373614619b9420b110d1508c5f17e066ce69c4c226fd04d02fbb959d9ba41eb6 (100ms) I0702 11:47:15.226889 1 gather_workloads_info.go:387] No image sha256:bcca5c17371caa2dce7e28da4c9fe30163c340fa172187f94db90ea1bd6ab80d (101ms) I0702 11:47:15.325729 1 gather_workloads_info.go:387] No image sha256:929a4a9c96a08b57fe4c315deca55c30d1990c83e34b8b5692f7646d29f715c8 (99ms) I0702 11:47:15.426489 1 gather_workloads_info.go:387] No image sha256:38dd36da63b1f01316a02daf6c18b821318c6ec313712147292d74abf955e0da (101ms) I0702 11:47:15.525684 1 gather_workloads_info.go:387] No image sha256:6d6ebff54b8adac74f4d1b12ac8aa0f16cd7b28370e0d6aa847d1c457e03a5b6 (99ms) I0702 11:47:15.625362 1 gather_workloads_info.go:387] No image sha256:18956d5fbc8d0369a26cb1e00eb5c3aff1973ae8cbd3983920a0cf2851f05358 (100ms) I0702 11:47:15.725655 1 gather_workloads_info.go:387] No image sha256:2d077e73dd76873ee2a1583aa3b3e76a1b408737f0af2e22b2aa055604d89e81 (100ms) I0702 11:47:15.825706 1 gather_workloads_info.go:387] No image sha256:1d471ad866b8564b209214d7dd3a7363c47e9535211d660b63b46abb278af7ab (100ms) I0702 11:47:15.926638 1 gather_workloads_info.go:387] No image sha256:46c2c396ca651699738c3f26b79b10af2922123dd27c2e32382d2a31a87e92c7 (101ms) I0702 11:47:16.026068 1 gather_workloads_info.go:387] No image sha256:0a8a0473029ace3adbb66f490dbe560f0e7782a38cdf32c1f7dd3e092e1d191e (99ms) I0702 11:47:16.127600 1 gather_workloads_info.go:387] No image sha256:039479f355d6903e1018abf8427fece3465c4348d4de2144ec38114b51bec27e (102ms) I0702 11:47:16.225885 1 gather_workloads_info.go:387] No image sha256:1cc2f3c83068c8c67d9070f67677e1bb6b4feff7ac5744cad08711893e56b55f (98ms) I0702 11:47:16.326428 1 gather_workloads_info.go:387] No image sha256:e40bbab1e52f580ab471bd7ab19ae23a4bcbaf1893243a05238af6bede7ba9cc (101ms) I0702 11:47:16.425889 1 gather_workloads_info.go:387] No image sha256:8b1e32ce43eb1247b0558230b6e0baa85ac62f1a5e2a2089d40b4fea90538529 (99ms) I0702 11:47:16.525796 1 gather_workloads_info.go:387] No image sha256:68d8e92046969bcfcbca2d2b3dcf4dcad5f6edc5fcb44861eb22f8079a43ea4d (100ms) I0702 11:47:16.625079 1 gather_workloads_info.go:387] No image sha256:5cba745f09f3bd6b8028c6773597cbb30bf3639a84f2b4b6493fe20642339a2c (99ms) I0702 11:47:16.725611 1 gather_workloads_info.go:387] No image sha256:2fac754deaeade3456361eed52e344318ff16d04819384432759f0ea35530114 (101ms) I0702 11:47:16.826196 1 gather_workloads_info.go:387] No image sha256:0a92dd43975f972e3dc707fb37854e046813531553038147e9c90d54a8d9df73 (101ms) I0702 11:47:16.925809 1 gather_workloads_info.go:387] No image sha256:00f977c2275f5bf03d423911159a39ca44fabc1862ecb7ae492198ec8c46c6bb (100ms) I0702 11:47:17.025646 1 gather_workloads_info.go:387] No image sha256:48404bd61d05dc738ccce4d22e36b30dbcdc6015b06b4afb604fd1baeee35bf2 (100ms) I0702 11:47:17.126441 1 gather_workloads_info.go:387] No image sha256:1f2aa1e1026570162984f7fbb634635bcf319e408cafa8f9034ee842a2f193ab (101ms) I0702 11:47:17.226114 1 gather_workloads_info.go:387] No image sha256:76f820bd9bd138d29305d545d7d49bfe63e2923f1fb1ec2e8eac81a388359024 (100ms) I0702 11:47:17.330403 1 gather_workloads_info.go:387] No image sha256:8ef4bdf07dda423fab73484dbc66d527ce8ba8cc2d4f99210bc1bc24ea08c0cb (104ms) I0702 11:47:17.425578 1 gather_workloads_info.go:387] No image sha256:59db91eadf1e1e5343777be43ff25fb712f300f9341be48ba05f45270944536f (95ms) I0702 11:47:17.425618 1 tasks_processing.go:74] worker 1 stopped. I0702 11:47:17.425926 1 recorder.go:70] Recording config/workload_info with fingerprint=898822cbe3315b4e94fe837530cdd386a8ccf0e595bd88ddb8bb04a4bc633e3e I0702 11:47:17.425941 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.061515737s to process 1 records I0702 11:47:17.425955 1 periodic.go:259] Periodic gather workloads completed in 8.061s I0702 11:47:17.425963 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0702 11:47:17.425967 1 periodic.go:212] Running conditional gatherer I0702 11:47:17.435405 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules I0702 11:47:17.442338 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.10:37537->172.30.0.10:53: read: connection refused I0702 11:47:17.442631 1 conditional_gatherer.go:336] updating alerts cache for conditional gatherer E0702 11:47:17.445697 1 conditional_gatherer.go:322] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:49473->172.30.0.10:53: read: connection refused I0702 11:47:17.445769 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0702 11:47:17.455883 1 conditional_gatherer.go:390] cluster version is '4.19.9' E0702 11:47:17.455900 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455906 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455910 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455914 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455916 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455920 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455923 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455926 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 11:47:17.455929 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0702 11:47:17.455943 1 tasks_processing.go:45] number of workers: 3 I0702 11:47:17.455950 1 tasks_processing.go:69] worker 2 listening for tasks. I0702 11:47:17.455954 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0702 11:47:17.456018 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 11:47:17.456025 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 11:47:17.456036 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0702 11:47:17.456057 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0702 11:47:17.456059 1 tasks_processing.go:71] worker 1 working on remote_configuration task. I0702 11:47:17.456073 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 979ns to process 1 records I0702 11:47:17.456053 1 tasks_processing.go:74] worker 0 stopped. I0702 11:47:17.456103 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0702 11:47:17.456112 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.698µs to process 1 records I0702 11:47:17.456117 1 tasks_processing.go:74] worker 1 stopped. I0702 11:47:17.456329 1 tasks_processing.go:74] worker 2 stopped. I0702 11:47:17.456342 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 283.845µs to process 0 records I0702 11:47:17.456361 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.10:37537->172.30.0.10:53: read: connection refused I0702 11:47:17.456380 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0702 11:47:17.482836 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=09e9ca7829ca7b1cdd84fb61cb23d208e65518c7a267c6ddc7a682b14567e4d7 I0702 11:47:17.482958 1 diskrecorder.go:70] Writing 136 records to /var/lib/insights-operator/insights-2026-07-02-114717.tar.gz I0702 11:47:17.493399 1 diskrecorder.go:51] Wrote 136 records to disk in 10ms I0702 11:47:17.493427 1 periodic.go:281] Gathering cluster info every 2h0m0s I0702 11:47:17.493443 1 periodic.go:282] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0702 11:48:25.539728 1 diskrecorder.go:223] Found files to send: insights-2026-07-02-114717.tar.gz I0702 11:48:25.539760 1 insightsuploader.go:150] Checking archives to upload periodically every 15m52.311869772s I0702 11:48:25.539771 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0702 11:48:25.552340 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0702 11:48:25.829614 1 requests.go:87] Successfully reported id=2026-07-02T11:48:25Z x-rh-insights-request-id=418227e4aa0a496ea86e9d187e5304c6, wrote=79094 I0702 11:48:25.829746 1 insightsuploader.go:187] Uploaded report successfully in 289.968372ms I0702 11:48:25.829772 1 controller.go:119] Initializing last reported time to 2026-07-02T11:48:25Z I0702 11:48:25.829796 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0702 11:48:25.829829 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0702 11:48:25.829845 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0702 11:48:25.835303 1 controller.go:481] The operator is healthy I0702 11:48:55.561861 1 controller.go:481] The operator is healthy I0702 11:49:26.801291 1 insightsreport.go:137] Pulling report from smart-proxy I0702 11:49:26.801319 1 insightsreport.go:149] Retrieving report I0702 11:49:26.810689 1 requests.go:110] Retrieving report for cluster: db0d3d3f-418f-4886-b820-640f093a78f8 I0702 11:49:26.810700 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/db0d3d3f-418f-4886-b820-640f093a78f8/reports I0702 11:49:26.813629 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/db0d3d3f-418f-4886-b820-640f093a78f8/reports I0702 11:49:27.171314 1 insightsreport.go:184] Report retrieved I0702 11:49:27.213808 1 insightsreport.go:239] Report retrieved correctly I0702 11:50:55.560111 1 controller.go:481] The operator is healthy I0702 11:51:55.531659 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 11:51:55.537254 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 11:51:55.537339 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 11:51:55.541487 1 secretconfigobserver.go:119] support secret does not exist I0702 11:52:55.560930 1 controller.go:481] The operator is healthy I0702 11:54:55.560513 1 controller.go:481] The operator is healthy I0702 11:56:55.530243 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 11:56:55.534140 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 11:56:55.542205 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 11:56:55.546385 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 11:56:55.546405 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 11:56:55.550101 1 secretconfigobserver.go:119] support secret does not exist I0702 11:56:55.559408 1 controller.go:481] The operator is healthy I0702 11:58:55.561012 1 controller.go:481] The operator is healthy I0702 12:00:55.564451 1 controller.go:481] The operator is healthy I0702 12:01:55.550613 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 12:01:55.559405 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 12:01:55.559428 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 12:01:55.563332 1 secretconfigobserver.go:119] support secret does not exist I0702 12:02:55.563906 1 controller.go:481] The operator is healthy I0702 12:04:23.143290 1 insightsuploader.go:146] Nothing to report since 2026-07-02T11:48:25Z I0702 12:04:55.560920 1 controller.go:481] The operator is healthy I0702 12:06:55.531078 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 12:06:55.559554 1 controller.go:481] The operator is healthy I0702 12:06:55.564392 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 12:06:55.568994 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 12:06:55.569014 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 12:06:55.573099 1 secretconfigobserver.go:119] support secret does not exist I0702 12:08:55.560613 1 controller.go:481] The operator is healthy I0702 12:10:55.563881 1 controller.go:481] The operator is healthy I0702 12:11:55.573448 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 12:11:55.581080 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 12:11:55.581107 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 12:11:55.584968 1 secretconfigobserver.go:119] support secret does not exist I0702 12:12:55.560094 1 controller.go:481] The operator is healthy I0702 12:14:55.563735 1 controller.go:481] The operator is healthy I0702 12:16:55.531095 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 12:16:55.535224 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 12:16:55.565511 1 controller.go:481] The operator is healthy I0702 12:16:55.585370 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 12:16:55.592357 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 12:16:55.592374 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 12:16:55.596830 1 secretconfigobserver.go:119] support secret does not exist I0702 12:18:55.560864 1 controller.go:481] The operator is healthy I0702 12:20:51.472277 1 insightsuploader.go:146] Nothing to report since 2026-07-02T11:48:25Z I0702 12:20:55.563386 1 controller.go:481] The operator is healthy I0702 12:21:55.597106 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 12:21:55.602158 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 12:21:55.602277 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 12:21:55.605935 1 secretconfigobserver.go:119] support secret does not exist I0702 12:22:55.564309 1 controller.go:481] The operator is healthy I0702 12:24:55.564360 1 controller.go:481] The operator is healthy