I0701 19:18:36.178348 1 cmd.go:253] Using service-serving-cert provided certificates I0701 19:18:36.178847 1 observer_polling.go:159] Starting file observer I0701 19:18:36.772866 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0701 19:18:36.773062 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0701 19:18:36.773748 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0701 19:18:36.774227 1 secure_serving.go:57] Forcing use of http/1.1 only W0701 19:18:36.774307 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0701 19:18:36.774319 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0701 19:18:36.774326 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0701 19:18:36.774331 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0701 19:18:36.774335 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0701 19:18:36.774339 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0701 19:18:36.778556 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0701 19:18:36.778583 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"63a01164-ed55-4dc0-9451-2e9ec897d064", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0701 19:18:36.783658 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0701 19:18:36.783737 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0701 19:18:36.783762 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0701 19:18:36.783691 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0701 19:18:36.783843 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0701 19:18:36.784173 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0701 19:18:36.784334 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0701 19:18:36.784483 1 secure_serving.go:213] Serving securely on [::]:8443 I0701 19:18:36.784501 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0701 19:18:36.788503 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0701 19:18:36.788526 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0701 19:18:36.788628 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0701 19:18:36.796876 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:18:36.796894 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0701 19:18:36.802921 1 secretconfigobserver.go:119] support secret does not exist I0701 19:18:36.809485 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:18:36.815601 1 secretconfigobserver.go:119] support secret does not exist I0701 19:18:36.820008 1 recorder.go:161] Pruning old reports every 8h7m42s, max age is 288h0m0s I0701 19:18:36.828102 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0701 19:18:36.828128 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0701 19:18:36.828139 1 insightsreport.go:296] Starting report retriever I0701 19:18:36.828145 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0701 19:18:36.828127 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0701 19:18:36.828108 1 periodic.go:209] Running clusterconfig gatherer I0701 19:18:36.828235 1 tasks_processing.go:45] number of workers: 32 I0701 19:18:36.828346 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:18:36.828373 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:18:36.828421 1 tasks_processing.go:69] worker 2 listening for tasks. I0701 19:18:36.828694 1 tasks_processing.go:69] worker 3 listening for tasks. I0701 19:18:36.828702 1 tasks_processing.go:69] worker 10 listening for tasks. I0701 19:18:36.828708 1 tasks_processing.go:71] worker 10 working on monitoring_persistent_volumes task. I0701 19:18:36.828716 1 tasks_processing.go:69] worker 22 listening for tasks. I0701 19:18:36.828716 1 tasks_processing.go:69] worker 16 listening for tasks. I0701 19:18:36.828719 1 tasks_processing.go:69] worker 15 listening for tasks. I0701 19:18:36.828708 1 tasks_processing.go:71] worker 3 working on nodes task. I0701 19:18:36.828731 1 tasks_processing.go:69] worker 18 listening for tasks. I0701 19:18:36.828730 1 tasks_processing.go:69] worker 28 listening for tasks. I0701 19:18:36.828736 1 tasks_processing.go:69] worker 30 listening for tasks. I0701 19:18:36.828739 1 tasks_processing.go:69] worker 19 listening for tasks. I0701 19:18:36.828744 1 tasks_processing.go:71] worker 1 working on infrastructures task. I0701 19:18:36.828747 1 tasks_processing.go:69] worker 20 listening for tasks. I0701 19:18:36.828752 1 tasks_processing.go:69] worker 24 listening for tasks. I0701 19:18:36.828755 1 tasks_processing.go:69] worker 7 listening for tasks. I0701 19:18:36.828761 1 tasks_processing.go:69] worker 25 listening for tasks. I0701 19:18:36.828763 1 tasks_processing.go:69] worker 4 listening for tasks. I0701 19:18:36.828770 1 tasks_processing.go:69] worker 26 listening for tasks. I0701 19:18:36.828771 1 tasks_processing.go:69] worker 5 listening for tasks. I0701 19:18:36.828729 1 tasks_processing.go:69] worker 29 listening for tasks. I0701 19:18:36.828781 1 tasks_processing.go:69] worker 6 listening for tasks. I0701 19:18:36.828787 1 tasks_processing.go:69] worker 27 listening for tasks. I0701 19:18:36.828788 1 tasks_processing.go:71] worker 2 working on validating_webhook_configurations task. I0701 19:18:36.829019 1 tasks_processing.go:69] worker 31 listening for tasks. I0701 19:18:36.829036 1 tasks_processing.go:71] worker 31 working on openshift_logging task. I0701 19:18:36.829045 1 tasks_processing.go:71] worker 27 working on machine_healthchecks task. I0701 19:18:36.829104 1 tasks_processing.go:69] worker 12 listening for tasks. I0701 19:18:36.829207 1 tasks_processing.go:69] worker 11 listening for tasks. I0701 19:18:36.829237 1 tasks_processing.go:71] worker 19 working on pdbs task. I0701 19:18:36.829526 1 tasks_processing.go:71] worker 11 working on metrics task. I0701 19:18:36.828708 1 tasks_processing.go:69] worker 21 listening for tasks. I0701 19:18:36.829599 1 tasks_processing.go:71] worker 12 working on machine_sets task. I0701 19:18:36.828745 1 tasks_processing.go:69] worker 23 listening for tasks. I0701 19:18:36.829784 1 tasks_processing.go:69] worker 8 listening for tasks. I0701 19:18:36.829824 1 tasks_processing.go:71] worker 8 working on schedulers task. I0701 19:18:36.829865 1 tasks_processing.go:71] worker 22 working on mutating_webhook_configurations task. I0701 19:18:36.829975 1 tasks_processing.go:71] worker 16 working on install_plans task. I0701 19:18:36.830028 1 tasks_processing.go:71] worker 18 working on dvo_metrics task. I0701 19:18:36.830093 1 tasks_processing.go:71] worker 30 working on storage_classes task. I0701 19:18:36.829785 1 tasks_processing.go:71] worker 15 working on openstack_dataplanenodesets task. I0701 19:18:36.830183 1 tasks_processing.go:71] worker 4 working on image_registries task. I0701 19:18:36.830246 1 tasks_processing.go:69] worker 13 listening for tasks. I0701 19:18:36.830312 1 tasks_processing.go:71] worker 13 working on silenced_alerts task. I0701 19:18:36.828757 1 tasks_processing.go:71] worker 0 working on olm_operators task. I0701 19:18:36.830245 1 tasks_processing.go:71] worker 7 working on nodenetworkconfigurationpolicies task. I0701 19:18:36.830339 1 tasks_processing.go:71] worker 25 working on cluster_apiserver task. I0701 19:18:36.829988 1 tasks_processing.go:71] worker 28 working on sap_pods task. I0701 19:18:36.829798 1 tasks_processing.go:71] worker 23 working on openstack_controlplanes task. I0701 19:18:36.829685 1 tasks_processing.go:71] worker 21 working on machine_configs task. I0701 19:18:36.828723 1 tasks_processing.go:69] worker 17 listening for tasks. I0701 19:18:36.830112 1 tasks_processing.go:69] worker 14 listening for tasks. I0701 19:18:36.831345 1 tasks_processing.go:71] worker 14 working on authentication task. I0701 19:18:36.830168 1 tasks_processing.go:71] worker 20 working on lokistack task. I0701 19:18:36.830284 1 tasks_processing.go:71] worker 26 working on crds task. I0701 19:18:36.830348 1 tasks_processing.go:71] worker 29 working on machine_config_pools task. I0701 19:18:36.830384 1 tasks_processing.go:71] worker 5 working on machines task. I0701 19:18:36.830388 1 tasks_processing.go:71] worker 6 working on active_alerts task. I0701 19:18:36.830392 1 tasks_processing.go:71] worker 24 working on pod_network_connectivity_checks task. I0701 19:18:36.829799 1 tasks_processing.go:69] worker 9 listening for tasks. I0701 19:18:36.831777 1 tasks_processing.go:71] worker 9 working on version task. I0701 19:18:36.832342 1 tasks_processing.go:71] worker 17 working on storage_cluster task. I0701 19:18:36.834623 1 tasks_processing.go:71] worker 27 working on overlapping_namespace_uids task. E0701 19:18:36.834649 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0701 19:18:36.834798 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 5.558946ms to process 0 records I0701 19:18:36.835528 1 tasks_processing.go:71] worker 7 working on image_pruners task. I0701 19:18:36.835541 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 5.109703ms to process 0 records I0701 19:18:36.835981 1 tasks_processing.go:71] worker 31 working on proxies task. I0701 19:18:36.836011 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 6.934515ms to process 0 records I0701 19:18:36.837535 1 tasks_processing.go:71] worker 12 working on config_maps task. I0701 19:18:36.837547 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 7.912574ms to process 0 records I0701 19:18:36.839049 1 tasks_processing.go:71] worker 17 working on ceph_cluster task. I0701 19:18:36.839074 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 6.68214ms to process 0 records E0701 19:18:36.841698 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.841722 1 tasks_processing.go:71] worker 11 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. E0701 19:18:36.841733 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.841765 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 12.173103ms to process 0 records I0701 19:18:36.842625 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0701 19:18:36.842644 1 controller.go:326] The initial operator extension status is healthy I0701 19:18:36.842655 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0701 19:18:36.842661 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0701 19:18:36.842665 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0701 19:18:36.842686 1 controller.go:489] The operator is still being initialized I0701 19:18:36.842699 1 controller.go:512] The operator is healthy I0701 19:18:36.842791 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0701 19:18:36.842851 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0701 19:18:36.842943 1 operator.go:288] started I0701 19:18:36.842969 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0701 19:18:36.844458 1 tasks_processing.go:71] worker 23 working on cost_management_metrics_configs task. I0701 19:18:36.844466 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 13.215806ms to process 0 records I0701 19:18:36.846696 1 tasks_processing.go:71] worker 1 working on oauths task. I0701 19:18:36.847383 1 recorder.go:75] Recording config/infrastructure with fingerprint=9b03b284fdb9bab842ed0e440dc34fdaf3dfd5563c162c308ed91e2be6c1b61f I0701 19:18:36.847404 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 17.939852ms to process 1 records I0701 19:18:36.847679 1 tasks_processing.go:71] worker 20 working on container_images task. I0701 19:18:36.847692 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 16.251594ms to process 0 records I0701 19:18:36.847883 1 tasks_processing.go:71] worker 15 working on support_secret task. I0701 19:18:36.847905 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 17.697518ms to process 0 records I0701 19:18:36.850659 1 tasks_processing.go:71] worker 28 working on feature_gates task. I0701 19:18:36.850692 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 20.070453ms to process 0 records I0701 19:18:36.852376 1 tasks_processing.go:71] worker 8 working on machine_autoscalers task. I0701 19:18:36.852548 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=4aa1c9521121d6685f3ee68aedd1454e607d7474bf8a5671d552eab2c16ec029 I0701 19:18:36.852857 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 22.503989ms to process 1 records I0701 19:18:36.853300 1 tasks_processing.go:71] worker 17 working on sap_config task. I0701 19:18:36.853336 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 14.235807ms to process 0 records I0701 19:18:36.853348 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 24.594754ms to process 0 records I0701 19:18:36.853356 1 tasks_processing.go:71] worker 10 working on openshift_machine_api_events task. I0701 19:18:36.853725 1 tasks_processing.go:71] worker 2 working on nodenetworkstates task. I0701 19:18:36.854204 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=1725931436df6ea90dce8bc4d6e7cd12b03d2c94df2632cd89e2d3bd28d13782 I0701 19:18:36.854369 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=638e59dc708bbdab74849219da90f4cc9b2a0f85f3d2b72ca7fe708bcb1fd176 I0701 19:18:36.854408 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=5bc2ef528b723bb689dfb90a18bb1049c1b68b13e2f02f16da91e60a42e1eb6c I0701 19:18:36.854447 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=e1c302cc64beedf5b4a6296ea0cb630826f7c8f186f41f098202f07ef3b3b2a6 I0701 19:18:36.854491 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=51406cc1b9be83e2c672bfbc4a726f89959765cdf5f8f54685c2cc128624dbdf I0701 19:18:36.854539 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=b577c1a6d42c7496ea90acbe4e2b7ff27066cdf31a070337819afaa459be7a15 I0701 19:18:36.854574 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=053db3450d55dbbace21c8a0a2322922ecb9d5c577db8d30ae02d6a3cdc7aa0f I0701 19:18:36.854630 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=64abc8c98411a5e45112e46b36947a9a3e93653f1cbe7c1c52db7b963fc44992 I0701 19:18:36.854667 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=739b43919f9b0a20b7b111748c14b9c9d56143b54c2348d1fb75bd3efdc24b44 I0701 19:18:36.854705 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=0363e924e5c9c5d5a639eff1a6e4517016da88f83006031dc3a7343b247f9941 I0701 19:18:36.854740 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=70ee2ca5f6e1e934235195c1ddcc689b00901a5e602d8840bdf23f55a3cbeedb I0701 19:18:36.854749 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 24.800989ms to process 11 records I0701 19:18:36.855208 1 tasks_processing.go:71] worker 23 working on ingress task. I0701 19:18:36.855490 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 10.735534ms to process 0 records I0701 19:18:36.859666 1 tasks_processing.go:71] worker 31 working on operators_pods_and_events task. I0701 19:18:36.859778 1 recorder.go:75] Recording config/proxy with fingerprint=f6e6677707920568b0b16d8b1cc86f7be80ae6d97b158724f9b48dd9feafcef8 I0701 19:18:36.859800 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 23.627294ms to process 1 records I0701 19:18:36.859962 1 tasks_processing.go:71] worker 14 working on networks task. I0701 19:18:36.860284 1 recorder.go:75] Recording config/authentication with fingerprint=94aaa020dde828e9e3fe48ad120d34449a94188ee923cbd84e3449bd65ef3925 I0701 19:18:36.860361 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 28.585923ms to process 1 records I0701 19:18:36.860965 1 tasks_processing.go:71] worker 24 working on operators task. E0701 19:18:36.860980 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0701 19:18:36.860992 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 29.23581ms to process 0 records I0701 19:18:36.861036 1 tasks_processing.go:71] worker 3 working on aggregated_monitoring_cr_names task. I0701 19:18:36.861461 1 recorder.go:75] Recording config/node/ip-10-0-2-226.ec2.internal with fingerprint=02b1ed05dfd1ef3c44ffa41ec653d73cd705e411067320d5ebbbb785eaafe71c I0701 19:18:36.861479 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 32.295083ms to process 1 records I0701 19:18:36.861527 1 tasks_processing.go:71] worker 5 working on container_runtime_configs task. E0701 19:18:36.861555 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0701 19:18:36.861645 1 gather.go:177] gatherer "clusterconfig" function "machines" took 29.941041ms to process 0 records I0701 19:18:36.862073 1 tasks_processing.go:71] worker 22 working on node_logs task. I0701 19:18:36.862507 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=d3816b9b27c6b169acd7b06695f2c721ba30b0e68c4c3b79734c8dadf0ba8789 I0701 19:18:36.862591 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=7343ce090321e598e72fd6c210199ccc921cd9b77e4c59758d63e1c73e9c1aac I0701 19:18:36.862659 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=8e2b1129e51dbdff430f58fa1758cff616339cd0547a56badd69d27f99cefa91 I0701 19:18:36.862687 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 32.181184ms to process 3 records I0701 19:18:36.862985 1 recorder.go:75] Recording config/apiserver with fingerprint=967fd59b393a4d64c2da25ef17a072eb22777d7d0330d4811a92cf4633123be9 I0701 19:18:36.863037 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 31.64538ms to process 1 records I0701 19:18:36.863055 1 tasks_processing.go:71] worker 25 working on openstack_version task. I0701 19:18:36.864882 1 tasks_processing.go:71] worker 29 working on qemu_kubevirt_launcher_logs task. I0701 19:18:36.865087 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 33.337221ms to process 0 records I0701 19:18:36.884405 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0701 19:18:36.884405 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0701 19:18:36.884580 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0701 19:18:36.886383 1 tasks_processing.go:71] worker 15 working on service_accounts task. E0701 19:18:36.886422 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0701 19:18:36.886439 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 38.48404ms to process 0 records I0701 19:18:36.886587 1 tasks_processing.go:71] worker 30 working on sap_datahubs task. I0701 19:18:36.886609 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=8fb01fce3d49cce11d9b8e558704e65cb1e01625a2068c09d794e62c6ce0252a I0701 19:18:36.886958 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=f91e6659dad06a3eb895036e3f615cb6154ed032aef3ef56734ea4912075b4bb I0701 19:18:36.886978 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 56.362319ms to process 2 records I0701 19:18:36.887072 1 tasks_processing.go:71] worker 1 working on tsdb_status task. I0701 19:18:36.887337 1 recorder.go:75] Recording config/oauth with fingerprint=aef7bc268a7e7b560c4957ad00c96a5a49811f18d48d06058f9e431aba5ab972 I0701 19:18:36.887363 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 39.935592ms to process 1 records I0701 19:18:36.887568 1 recorder.go:75] Recording config/network with fingerprint=446e4401a1d99901afc49a07aaa9085f3411c93ce6a0816abfcb8657fa35da7b I0701 19:18:36.887583 1 gather.go:177] gatherer "clusterconfig" function "networks" took 26.918197ms to process 1 records I0701 19:18:36.887583 1 tasks_processing.go:71] worker 14 working on jaegers task. I0701 19:18:36.887840 1 tasks_processing.go:71] worker 7 working on clusterroles task. I0701 19:18:36.887858 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=8862c4c58952740abb484703fea478ecffb1e1cacfacf6b2bae20081a96f4a89 I0701 19:18:36.887975 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 51.668309ms to process 1 records E0701 19:18:36.888324 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.888349 1 tasks_processing.go:71] worker 6 working on certificate_signing_requests task. E0701 19:18:36.888364 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: no such host E0701 19:18:36.888455 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.888478 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 56.731946ms to process 0 records E0701 19:18:36.888490 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.888503 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 58.042063ms to process 0 records I0701 19:18:36.888516 1 tasks_processing.go:71] worker 13 working on openstack_dataplanedeployments task. I0701 19:18:36.888665 1 base_controller.go:82] Caches are synced for ConfigController I0701 19:18:36.888681 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0701 19:18:36.888869 1 tasks_processing.go:71] worker 10 working on image task. I0701 19:18:36.889135 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 35.505609ms to process 0 records I0701 19:18:36.889199 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 36.424283ms to process 0 records I0701 19:18:36.889212 1 tasks_processing.go:71] worker 8 working on ingress_certificates task. I0701 19:18:36.889336 1 tasks_processing.go:74] worker 23 stopped. I0701 19:18:36.889429 1 recorder.go:75] Recording config/ingress with fingerprint=4302169f2c6a5a268a200b1147bd404a60eaba094c7e2b6b7560b28f762dcc1b I0701 19:18:36.889445 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 33.901313ms to process 1 records I0701 19:18:36.889598 1 tasks_processing.go:74] worker 2 stopped. I0701 19:18:36.889762 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 35.811362ms to process 0 records I0701 19:18:36.889794 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 26.625869ms to process 0 records I0701 19:18:36.889889 1 tasks_processing.go:74] worker 25 stopped. I0701 19:18:36.890614 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=1a3cb75017828b9b7a529b1b6f807a58fbe167b3a39e66999c8050edd5ab2c6c I0701 19:18:36.890802 1 tasks_processing.go:74] worker 26 stopped. I0701 19:18:36.890999 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=6a2083507a05c3037089ecb8ee7b05d84c80b5c546200bb46fc129139a6a1953 I0701 19:18:36.891044 1 gather.go:177] gatherer "clusterconfig" function "crds" took 58.214142ms to process 2 records I0701 19:18:36.891106 1 controller.go:212] Source scaController *sca.Controller is not ready I0701 19:18:36.891126 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready E0701 19:18:36.890628 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.891132 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0701 19:18:36.891137 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0701 19:18:36.891141 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0701 19:18:36.891166 1 controller.go:489] The operator is still being initialized I0701 19:18:36.891180 1 controller.go:512] The operator is healthy I0701 19:18:36.891278 1 tasks_processing.go:74] worker 4 stopped. I0701 19:18:36.891637 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=1b85eaf85c92833bc3c7e36498458d9d683c691a5e51c49b45e17df5ad8c42ac I0701 19:18:36.893778 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 60.164195ms to process 1 records I0701 19:18:36.893802 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 37.338561ms to process 0 records I0701 19:18:36.894100 1 recorder.go:75] Recording config/featuregate with fingerprint=1a3e46fa14ca506af1169e8b6919202ce74041ae41e9d1164564a61b120241fd I0701 19:18:36.894114 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 40.111395ms to process 1 records I0701 19:18:36.894152 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0701 19:18:36.894163 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 56.164745ms to process 1 records I0701 19:18:36.894179 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 29.516982ms to process 0 records E0701 19:18:36.894187 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:36.894219 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.019157ms to process 0 records I0701 19:18:36.894345 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 5.348866ms to process 0 records I0701 19:18:36.894554 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=84119c165af8c4139a7ee0c99cb648ed9ee17c8c9a9c04f009e010e87e538433 I0701 19:18:36.894624 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=db5f8206c5b7870ae8b3f19468bacc943f036f0156bba7b1cf71acc8113e8654 I0701 19:18:36.894678 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=cb3b9f34510c88534586ac1d027c19c671a9ab5e9493a2b1d29ff129814a9272 I0701 19:18:36.894704 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=f419ae378470926433e79495837b809e62c6447404cef9ba6899bb22562d7bc9 I0701 19:18:36.894755 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=4185c2792726ecad71ac3b5b62920f6ae3b4dd3e81d74529c0b9f5d2f6d9b7f3 I0701 19:18:36.894786 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=d98b69df99cb8f95344ccc0c344511d6c9e39752f7e752bbdb2075eebc1261a8 I0701 19:18:36.894796 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 62.758812ms to process 6 records I0701 19:18:36.894825 1 tasks_processing.go:74] worker 19 stopped. I0701 19:18:36.894839 1 tasks_processing.go:74] worker 17 stopped. I0701 19:18:36.894845 1 tasks_processing.go:74] worker 28 stopped. I0701 19:18:36.894851 1 tasks_processing.go:74] worker 27 stopped. I0701 19:18:36.894857 1 tasks_processing.go:74] worker 5 stopped. I0701 19:18:36.894863 1 tasks_processing.go:74] worker 1 stopped. I0701 19:18:36.894870 1 tasks_processing.go:74] worker 30 stopped. I0701 19:18:36.895100 1 tasks_processing.go:74] worker 13 stopped. I0701 19:18:36.895113 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 6.576752ms to process 0 records I0701 19:18:36.895174 1 tasks_processing.go:74] worker 14 stopped. I0701 19:18:36.895186 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 7.577925ms to process 0 records I0701 19:18:36.899112 1 tasks_processing.go:74] worker 18 stopped. I0701 19:18:36.899134 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0701 19:18:36.899222 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 69.029114ms to process 1 records I0701 19:18:36.899382 1 tasks_processing.go:74] worker 10 stopped. I0701 19:18:36.899400 1 recorder.go:75] Recording config/image with fingerprint=ac6fe97eec3c33a058af480e5b7fd0956f2d4ec271e75209d1482fb8da9e8f33 I0701 19:18:36.899415 1 gather.go:177] gatherer "clusterconfig" function "image" took 10.296203ms to process 1 records I0701 19:18:36.899496 1 recorder.go:75] Recording config/olm_operators with fingerprint=bd65712f29cbcf4089d58eecef82dc351751a15fc3618c4b8795598a0d599cf0 I0701 19:18:36.899510 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 68.860348ms to process 1 records I0701 19:18:36.899522 1 tasks_processing.go:74] worker 0 stopped. I0701 19:18:36.905604 1 tasks_processing.go:74] worker 6 stopped. I0701 19:18:36.905624 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 17.240307ms to process 0 records I0701 19:18:36.905828 1 tasks_processing.go:74] worker 11 stopped. I0701 19:18:36.905852 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 64.091931ms to process 0 records I0701 19:18:36.907237 1 tasks_processing.go:74] worker 20 stopped. I0701 19:18:36.908836 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:18:36.909672 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-qjgcw with fingerprint=f53cf82aced4f3fa9819c43405e9587aafd70142a6f0993c34b526662d49c7d8 I0701 19:18:36.909918 1 recorder.go:75] Recording config/running_containers with fingerprint=2120e5fa84c49d28d3b64e210f14d9ec09c2a21e093c74c5da622c10f4882b16 I0701 19:18:36.909967 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 59.540307ms to process 2 records I0701 19:18:36.910298 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=1e37f60d348f2c1af7e516791b4621c47e87c07ccf11f0943bb066344c8a9347 I0701 19:18:36.910488 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=66b4c2fdedc86f8fe64135830205f1826255d276d734a41a47b1a41b44fad8be I0701 19:18:36.910543 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 20.822208ms to process 2 records I0701 19:18:36.910576 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 48.301483ms to process 0 records I0701 19:18:36.910610 1 tasks_processing.go:74] worker 3 stopped. I0701 19:18:36.910640 1 tasks_processing.go:74] worker 7 stopped. I0701 19:18:36.928473 1 tasks_processing.go:74] worker 12 stopped. E0701 19:18:36.928492 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0701 19:18:36.928500 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0701 19:18:36.928506 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0701 19:18:36.928519 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0701 19:18:36.928554 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0701 19:18:36.928559 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0701 19:18:36.928563 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0701 19:18:36.928572 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0701 19:18:36.928610 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0701 19:18:36.928619 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0701 19:18:36.928624 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 90.916843ms to process 7 records I0701 19:18:36.941110 1 tasks_processing.go:74] worker 8 stopped. E0701 19:18:36.941136 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0701 19:18:36.941152 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2r9h8n05fjgmel2d3av4qf8julb1tirv-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2r9h8n05fjgmel2d3av4qf8julb1tirv-primary-cert-bundle-secret" not found I0701 19:18:36.941244 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=6b2578e5b9159a49b2e2ab5a27efdbcc8f4d9921f5bc40b8fed455311649ef7a I0701 19:18:36.941285 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 51.849858ms to process 1 records I0701 19:18:36.943197 1 base_controller.go:82] Caches are synced for LoggingSyncer I0701 19:18:36.943214 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0701 19:18:36.951214 1 tasks_processing.go:74] worker 22 stopped. I0701 19:18:36.951239 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 89.073573ms to process 0 records I0701 19:18:36.954867 1 tasks_processing.go:74] worker 21 stopped. I0701 19:18:36.954900 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0701 19:18:36.954911 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 123.566791ms to process 1 records I0701 19:18:36.962348 1 tasks_processing.go:74] worker 9 stopped. I0701 19:18:36.962353 1 gather_logs.go:145] no pods in namespace were found I0701 19:18:36.962831 1 recorder.go:75] Recording config/version with fingerprint=70792d6f54f548c6a92361a07eef5ab5670ce020fbb05e64bc616002f8ed1664 I0701 19:18:36.962853 1 recorder.go:75] Recording config/id with fingerprint=0c37466f55bb04ce6e6ff59dd6d2c0a4373bd3261a26e9cf08dda64d6d129d48 I0701 19:18:36.962862 1 gather.go:177] gatherer "clusterconfig" function "version" took 130.554901ms to process 2 records I0701 19:18:36.962879 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 97.472279ms to process 0 records I0701 19:18:36.962889 1 tasks_processing.go:74] worker 29 stopped. I0701 19:18:36.963345 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0701 19:18:37.001190 1 gather_cluster_operator_pods_and_events.go:121] Found 4 pods with 4 containers I0701 19:18:37.001205 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 6291456 bytes I0701 19:18:37.002775 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5d7cc55746-l4m7v pod in namespace openshift-ingress (previous: false). I0701 19:18:37.009119 1 cluster_transfer.go:102] no available accepted cluster transfer I0701 19:18:37.009143 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0701 19:18:37.014456 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:18:37.014474 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5d7cc55746-vv2k5 pod in namespace openshift-ingress (previous: false). I0701 19:18:37.041820 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-68b654865c-bdrkr pod in namespace openshift-ingress (previous: false). I0701 19:18:37.078226 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-5v9sd pod in namespace openshift-ingress-canary (previous: false). I0701 19:18:37.100198 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0701 19:18:37.106604 1 tasks_processing.go:74] worker 31 stopped. I0701 19:18:37.106728 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=bd634b26f21284c7c01f72a2f89877b8b483c76c2bb269858187f23e33c87f98 I0701 19:18:37.106795 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=93bc4b3bdf2c54e1f3de516deacf6d53c53495167779feb12ad3ff2623c1d29f I0701 19:18:37.106833 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=e86bcba43d4028158201b9ff21e8dd14aba5faefecce6c7a669991e79de5a459 I0701 19:18:37.106854 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5d7cc55746-vv2k5/router_current.log with fingerprint=a6860ac7a1a51cb577e4daecf07b4f233adfd7f757cfe2a0c218642764b9248a I0701 19:18:37.106874 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-68b654865c-bdrkr/router_current.log with fingerprint=0c518860f2cba335da91b6a21b15a1551241849953296a6eb4347b969358e180 I0701 19:18:37.106880 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-5v9sd/serve-healthcheck-canary_current.log with fingerprint=c5e2efaaab9340ddc0bf681d51d04c912a9ef1573e8f63c2458da4470a86727a I0701 19:18:37.106887 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 246.915187ms to process 6 records I0701 19:18:37.117267 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0701 19:18:37.198548 1 sca.go:266] etc-pki-entitlement secret successfully created I0701 19:18:37.198592 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0701 19:18:39.410863 1 tasks_processing.go:74] worker 24 stopped. I0701 19:18:39.411002 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=946024c654f5f4626c073832bde270592de9c47ecd15fb4dcfa139365c81e90c I0701 19:18:39.411051 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=b1d3a57201f472fba248e6e22732d03f4ebac77f3d7204455295e9c6f31f376a I0701 19:18:39.411076 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=035416e492c40e0cfa773f65f9312567164592492a902293470bdcad3931882b I0701 19:18:39.411086 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0701 19:18:39.411108 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=ce7dee7cabd48d203ac18d94a22e01c4dfd6b66b3e711331ef59ba7512e7ee5c I0701 19:18:39.411124 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0701 19:18:39.411146 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=8b2012a6210d9da73f3e2728b5066d96e721a60c191fa1018db61026d04de5cd I0701 19:18:39.411175 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=4fc7275272b2935d169f2bdc1db0d2236c933039f48d418bc7461bd6318d57cd I0701 19:18:39.411199 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=b243bd54394edcdc6df08bac1749a169151084d8b41a6ef3699d3cf79333931b I0701 19:18:39.411230 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=27a08f365545b4177e471e3114af7e42c604d64d4a665cc2054d9b76a90757e0 I0701 19:18:39.411525 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=8f43809a86f7b9d9be53844dbea52837fbfff9e0f8932663bd3e1fa3acf8d8bd I0701 19:18:39.411547 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=7f22c7ca973feb650e9204480400710372499d949b3c08f873521dfa6d8561b1 I0701 19:18:39.411557 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0701 19:18:39.411573 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=9823208887159679630180811fe486ed29b9488a8a304b0584f37f65d7ad808a I0701 19:18:39.411583 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0701 19:18:39.411598 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=ed0c75c86a09e5d007662e2d9f3410db27547db9e982ce393c0f5133dce2c7bb I0701 19:18:39.411608 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0701 19:18:39.411628 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=4abe9eb1d9a72537ef953ee2ef09b6aa63affec09d798c7ad2df068fe9e9b1c2 I0701 19:18:39.411637 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0701 19:18:39.411658 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=7ac9e3949b7de690a315a78f2e55c499c734c659eec2017e23ac32efc33b92b4 I0701 19:18:39.411760 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=0cdb63db8d01d040230dbff9f35c28a51929b6e654fe37183fcb1fa479a3e0e0 I0701 19:18:39.411770 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0701 19:18:39.411778 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0701 19:18:39.411800 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0701 19:18:39.411820 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=04858be4ce2b7ab671150511e5b00dab66b57c7d08c83752acddf6e9d818f65d I0701 19:18:39.411844 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=395dcd102ffde6d013fcb276d80ec3ffb567246bb9319c8e53f3587b77b1e480 I0701 19:18:39.411853 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0701 19:18:39.411869 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=f56f1df89d3687f8f40aae79ab31f7e87090c6739ff1700d098f189055f7e1d8 I0701 19:18:39.411879 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0701 19:18:39.411896 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=889be5ffe452e5908f02e0aa021564838985f358596d44c6743ffa4cbdc35a05 I0701 19:18:39.411905 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0701 19:18:39.411917 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=204957587fae4d4493bd490b83b8099444a60ae69ce7683abf0955fd5e6aa816 I0701 19:18:39.411933 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=6b092a61dd38b29da321d8b7cd60f7e4918ecee6096de87f82478bfaf1fab598 I0701 19:18:39.411949 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=42694c5d28ac6f777910f17a12d89e37658a24718e314a1b8ce1144ad5c5a239 I0701 19:18:39.411970 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=a709fcabc86f5c1edf656ba1c910bf23f834d6f72e5859dbdafa76f81bd3d0c8 I0701 19:18:39.411979 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0701 19:18:39.412007 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=361367f6c32fa916cbe0c7fe0edd7bf8eb939b1cf6f7a907232905b2d1715376 I0701 19:18:39.412024 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0701 19:18:39.412033 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0701 19:18:39.412040 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.549874992s to process 39 records I0701 19:18:49.873506 1 tasks_processing.go:74] worker 16 stopped. I0701 19:18:49.873559 1 recorder.go:75] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0701 19:18:49.873572 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.043474506s to process 1 records I0701 19:18:50.695181 1 tasks_processing.go:74] worker 15 stopped. I0701 19:18:50.695550 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=bae76059ad12f92c03c06b36414d832a0dbbce27d94c635a80399044861a2efd I0701 19:18:50.695568 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.808770904s to process 1 records E0701 19:18:50.695643 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.867s with: function \"machine_healthchecks\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"tsdb_status\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error" I0701 19:18:50.696752 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error I0701 19:18:50.696769 1 periodic.go:209] Running workloads gatherer I0701 19:18:50.696784 1 tasks_processing.go:45] number of workers: 2 I0701 19:18:50.696799 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:18:50.696803 1 tasks_processing.go:71] worker 1 working on workload_info task. I0701 19:18:50.696816 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:18:50.696834 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0701 19:18:50.706353 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-226.ec2.internal... I0701 19:18:50.731423 1 tasks_processing.go:74] worker 0 stopped. I0701 19:18:50.731438 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 34.572954ms to process 0 records I0701 19:18:54.447022 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0701 19:18:54.463748 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 20s for image data I0701 19:18:54.475452 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (12ms) I0701 19:18:54.486824 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (11ms) I0701 19:18:54.498940 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (12ms) I0701 19:18:54.509685 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (11ms) I0701 19:18:54.520231 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (11ms) I0701 19:18:54.531352 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (11ms) I0701 19:18:54.541919 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (11ms) I0701 19:18:54.552768 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (11ms) I0701 19:18:54.563475 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (11ms) I0701 19:18:54.576161 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (13ms) I0701 19:18:54.587392 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (11ms) I0701 19:18:54.675603 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (88ms) I0701 19:18:54.778790 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (103ms) I0701 19:18:54.876798 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (98ms) I0701 19:18:54.976317 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (99ms) I0701 19:18:55.076160 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (100ms) I0701 19:18:55.178397 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (102ms) I0701 19:18:55.276638 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (98ms) I0701 19:18:55.376898 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0701 19:18:55.477130 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0701 19:18:55.577391 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (100ms) I0701 19:18:55.680689 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (103ms) I0701 19:18:55.777317 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (97ms) I0701 19:18:55.876496 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0701 19:18:55.976414 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0701 19:18:56.079815 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (103ms) I0701 19:18:56.179797 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0701 19:18:56.277886 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (98ms) I0701 19:18:56.377674 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0701 19:18:56.476094 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (98ms) I0701 19:18:56.577732 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (102ms) I0701 19:18:56.676712 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (99ms) I0701 19:18:56.776668 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0701 19:18:56.879603 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (103ms) I0701 19:18:56.977512 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (98ms) I0701 19:18:57.076846 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0701 19:18:57.181310 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (104ms) I0701 19:18:57.276622 1 gather_workloads_info.go:387] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (95ms) I0701 19:18:57.276657 1 tasks_processing.go:74] worker 1 stopped. I0701 19:18:57.276883 1 recorder.go:75] Recording config/workload_info with fingerprint=3fe8540aeece24734982bdf7c1a9896f1d85bea2163cae0bc67d68444aebfeab I0701 19:18:57.276898 1 gather.go:177] gatherer "workloads" function "workload_info" took 6.579846662s to process 1 records I0701 19:18:57.276912 1 periodic.go:256] Periodic gather workloads completed in 6.58s I0701 19:18:57.276919 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0701 19:18:57.276924 1 periodic.go:209] Running conditional gatherer I0701 19:18:57.285571 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0701 19:18:57.421891 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0701 19:18:57.424680 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0701 19:18:57.424767 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0701 19:18:57.437922 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0701 19:18:57.437938 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437944 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437948 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437951 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437954 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437957 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437961 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437965 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437970 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:18:57.437974 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0701 19:18:57.437992 1 tasks_processing.go:45] number of workers: 3 I0701 19:18:57.438006 1 tasks_processing.go:69] worker 2 listening for tasks. I0701 19:18:57.438015 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0701 19:18:57.438015 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:18:57.438024 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:18:57.438031 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0701 19:18:57.438034 1 tasks_processing.go:74] worker 1 stopped. I0701 19:18:57.438043 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0701 19:18:57.438102 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0701 19:18:57.438116 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.136µs to process 1 records I0701 19:18:57.438148 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0701 19:18:57.438156 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.297µs to process 1 records I0701 19:18:57.438161 1 tasks_processing.go:74] worker 0 stopped. I0701 19:18:57.438379 1 tasks_processing.go:74] worker 2 stopped. I0701 19:18:57.438397 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 324.676µs to process 0 records I0701 19:18:57.438417 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0701 19:18:57.438442 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0701 19:18:57.468140 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=d6006c1676f4980dd1e518e1ae2b397be0bca1cc25c2005cf5d429a6320e3516 I0701 19:18:57.468312 1 diskrecorder.go:70] Writing 106 records to /var/lib/insights-operator/insights-2026-07-01-191857.tar.gz I0701 19:18:57.475305 1 diskrecorder.go:51] Wrote 106 records to disk in 7ms I0701 19:18:57.475333 1 periodic.go:278] Gathering cluster info every 2h0m0s I0701 19:18:57.475347 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0701 19:20:06.828297 1 diskrecorder.go:223] Found files to send: insights-2026-07-01-191857.tar.gz I0701 19:20:06.828327 1 insightsuploader.go:150] Checking archives to upload periodically every 15m39.610754506s I0701 19:20:06.828337 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0701 19:20:06.848388 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0701 19:20:06.982596 1 requests.go:87] Successfully reported id=2026-07-01T19:20:06Z x-rh-insights-request-id=2cf1ed8eaeca4a1097c6b8e9a37e9af7, wrote=50353 I0701 19:20:06.982672 1 insightsuploader.go:187] Uploaded report successfully in 154.321747ms I0701 19:20:06.982704 1 controller.go:128] Initializing last reported time to 2026-07-01T19:20:06Z I0701 19:20:06.982769 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0701 19:20:06.982785 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0701 19:20:06.982795 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0701 19:20:06.995416 1 controller.go:512] The operator is healthy I0701 19:20:36.853703 1 controller.go:512] The operator is healthy I0701 19:21:11.181352 1 insightsreport.go:137] Pulling report from smart-proxy I0701 19:21:11.181373 1 insightsreport.go:149] Retrieving report I0701 19:21:11.191001 1 requests.go:110] Retrieving report for cluster: 11beda1b-203a-4cce-9a9e-1896a20428f2 I0701 19:21:11.191013 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/11beda1b-203a-4cce-9a9e-1896a20428f2/reports I0701 19:21:11.194326 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/11beda1b-203a-4cce-9a9e-1896a20428f2/reports I0701 19:21:11.329566 1 insightsreport.go:184] Report retrieved I0701 19:21:11.377697 1 insightsreport.go:239] Report retrieved correctly I0701 19:22:36.855364 1 controller.go:512] The operator is healthy I0701 19:23:36.816153 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:23:36.824105 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:23:36.824127 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:23:36.830243 1 secretconfigobserver.go:119] support secret does not exist I0701 19:24:36.852418 1 controller.go:512] The operator is healthy I0701 19:26:36.854912 1 controller.go:512] The operator is healthy I0701 19:28:36.809286 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:28:36.831349 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:28:36.837807 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:28:36.837828 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:28:36.843473 1 secretconfigobserver.go:119] support secret does not exist I0701 19:28:36.850272 1 controller.go:512] The operator is healthy I0701 19:30:36.851166 1 controller.go:512] The operator is healthy I0701 19:32:36.852046 1 controller.go:512] The operator is healthy I0701 19:33:36.844281 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:33:36.852056 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:33:36.852076 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:33:36.858219 1 secretconfigobserver.go:119] support secret does not exist I0701 19:34:36.852671 1 controller.go:512] The operator is healthy I0701 19:35:51.595134 1 insightsuploader.go:146] Nothing to report since 2026-07-01T19:20:06Z I0701 19:36:36.851555 1 controller.go:512] The operator is healthy I0701 19:38:36.808201 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:38:36.814643 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:38:36.850840 1 controller.go:512] The operator is healthy I0701 19:38:36.858686 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:38:36.865962 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:38:36.865978 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:38:36.874480 1 secretconfigobserver.go:119] support secret does not exist I0701 19:40:36.852607 1 controller.go:512] The operator is healthy I0701 19:42:36.852225 1 controller.go:512] The operator is healthy I0701 19:43:36.875139 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:43:36.883010 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:43:36.883034 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:43:36.891822 1 secretconfigobserver.go:119] support secret does not exist I0701 19:44:36.851520 1 controller.go:512] The operator is healthy I0701 19:46:36.851906 1 controller.go:512] The operator is healthy I0701 19:48:36.811403 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:48:36.850922 1 controller.go:512] The operator is healthy I0701 19:48:36.892074 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:48:36.899763 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:48:36.899785 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:48:36.907729 1 secretconfigobserver.go:119] support secret does not exist I0701 19:50:36.856917 1 controller.go:512] The operator is healthy I0701 19:51:08.548764 1 insightsuploader.go:146] Nothing to report since 2026-07-01T19:20:06Z I0701 19:52:36.853361 1 controller.go:512] The operator is healthy I0701 19:53:36.908362 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:53:36.917987 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:53:36.918018 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:53:36.924747 1 secretconfigobserver.go:119] support secret does not exist I0701 19:54:36.853699 1 controller.go:512] The operator is healthy I0701 19:56:36.851563 1 controller.go:512] The operator is healthy I0701 19:58:36.813121 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:58:36.850446 1 controller.go:512] The operator is healthy I0701 19:58:36.925520 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:58:36.933559 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:58:36.933582 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:58:36.940943 1 secretconfigobserver.go:119] support secret does not exist I0701 20:00:36.854692 1 controller.go:512] The operator is healthy I0701 20:02:36.855171 1 controller.go:512] The operator is healthy I0701 20:03:36.941569 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:03:36.950569 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:03:36.950591 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:03:36.957397 1 secretconfigobserver.go:119] support secret does not exist I0701 20:04:36.851055 1 controller.go:512] The operator is healthy I0701 20:06:36.854894 1 controller.go:512] The operator is healthy I0701 20:07:37.787919 1 insightsuploader.go:146] Nothing to report since 2026-07-01T19:20:06Z I0701 20:08:36.811412 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:08:36.818000 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:08:36.850831 1 controller.go:512] The operator is healthy I0701 20:08:36.958067 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:08:36.965842 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:08:36.965871 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:08:36.973423 1 secretconfigobserver.go:119] support secret does not exist