W0701 19:44:54.983026 1 cmd.go:257] Using insecure, self-signed certificates I0701 19:44:55.395600 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0701 19:44:55.395958 1 observer_polling.go:159] Starting file observer I0701 19:44:56.004599 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0701 19:44:56.004802 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0701 19:44:56.005371 1 secure_serving.go:57] Forcing use of http/1.1 only W0701 19:44:56.005396 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0701 19:44:56.005403 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0701 19:44:56.005409 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0701 19:44:56.005414 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0701 19:44:56.005419 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0701 19:44:56.005423 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0701 19:44:56.005443 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0701 19:44:56.008953 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0701 19:44:56.008995 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"3cfa0c51-13c2-4ffa-8086-a545175e7779", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0701 19:44:56.011747 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0701 19:44:56.011763 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-334031974/tls.crt::/tmp/serving-cert-334031974/tls.key" I0701 19:44:56.011783 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0701 19:44:56.011801 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0701 19:44:56.011803 1 secure_serving.go:213] Serving securely on [::]:8443 I0701 19:44:56.011875 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" I0701 19:44:56.011767 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0701 19:44:56.011891 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0701 19:44:56.011772 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController W0701 19:44:56.013958 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0701 19:44:56.014016 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0701 19:44:56.014089 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0701 19:44:56.021160 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:44:56.021178 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0701 19:44:56.031319 1 secretconfigobserver.go:119] support secret does not exist I0701 19:44:56.036748 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:44:56.040637 1 secretconfigobserver.go:119] support secret does not exist I0701 19:44:56.042577 1 recorder.go:161] Pruning old reports every 4h56m52s, max age is 288h0m0s I0701 19:44:56.047448 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0701 19:44:56.047457 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0701 19:44:56.047470 1 insightsreport.go:296] Starting report retriever I0701 19:44:56.047474 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0701 19:44:56.047479 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0701 19:44:56.047490 1 periodic.go:209] Running clusterconfig gatherer I0701 19:44:56.047530 1 tasks_processing.go:45] number of workers: 32 I0701 19:44:56.047565 1 tasks_processing.go:69] worker 10 listening for tasks. I0701 19:44:56.047574 1 tasks_processing.go:69] worker 4 listening for tasks. I0701 19:44:56.047576 1 tasks_processing.go:71] worker 10 working on monitoring_persistent_volumes task. I0701 19:44:56.047584 1 tasks_processing.go:69] worker 12 listening for tasks. I0701 19:44:56.047585 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:44:56.047587 1 tasks_processing.go:69] worker 7 listening for tasks. I0701 19:44:56.047592 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:44:56.047596 1 tasks_processing.go:69] worker 13 listening for tasks. I0701 19:44:56.047600 1 tasks_processing.go:69] worker 6 listening for tasks. I0701 19:44:56.047599 1 tasks_processing.go:69] worker 5 listening for tasks. I0701 19:44:56.047603 1 tasks_processing.go:69] worker 2 listening for tasks. I0701 19:44:56.047608 1 tasks_processing.go:69] worker 23 listening for tasks. I0701 19:44:56.047611 1 tasks_processing.go:69] worker 22 listening for tasks. I0701 19:44:56.047603 1 tasks_processing.go:69] worker 14 listening for tasks. I0701 19:44:56.047614 1 tasks_processing.go:69] worker 18 listening for tasks. I0701 19:44:56.047617 1 tasks_processing.go:69] worker 24 listening for tasks. I0701 19:44:56.047621 1 tasks_processing.go:69] worker 19 listening for tasks. I0701 19:44:56.047622 1 tasks_processing.go:69] worker 15 listening for tasks. I0701 19:44:56.047619 1 tasks_processing.go:69] worker 3 listening for tasks. I0701 19:44:56.047629 1 tasks_processing.go:69] worker 26 listening for tasks. I0701 19:44:56.047623 1 tasks_processing.go:69] worker 25 listening for tasks. I0701 19:44:56.047633 1 tasks_processing.go:69] worker 8 listening for tasks. I0701 19:44:56.047638 1 tasks_processing.go:69] worker 27 listening for tasks. I0701 19:44:56.047640 1 tasks_processing.go:69] worker 17 listening for tasks. I0701 19:44:56.047627 1 tasks_processing.go:69] worker 9 listening for tasks. I0701 19:44:56.047644 1 tasks_processing.go:69] worker 28 listening for tasks. I0701 19:44:56.047648 1 tasks_processing.go:69] worker 29 listening for tasks. I0701 19:44:56.047651 1 tasks_processing.go:71] worker 13 working on pod_network_connectivity_checks task. I0701 19:44:56.047633 1 tasks_processing.go:69] worker 16 listening for tasks. I0701 19:44:56.047652 1 tasks_processing.go:69] worker 30 listening for tasks. I0701 19:44:56.047660 1 tasks_processing.go:71] worker 16 working on openstack_version task. I0701 19:44:56.047567 1 tasks_processing.go:69] worker 31 listening for tasks. I0701 19:44:56.047665 1 tasks_processing.go:71] worker 5 working on cluster_apiserver task. I0701 19:44:56.047663 1 tasks_processing.go:71] worker 12 working on lokistack task. I0701 19:44:56.047691 1 tasks_processing.go:71] worker 8 working on jaegers task. I0701 19:44:56.047694 1 tasks_processing.go:71] worker 15 working on container_images task. I0701 19:44:56.047660 1 tasks_processing.go:71] worker 6 working on tsdb_status task. I0701 19:44:56.047744 1 tasks_processing.go:71] worker 7 working on machine_healthchecks task. I0701 19:44:56.047746 1 tasks_processing.go:71] worker 23 working on operators_pods_and_events task. I0701 19:44:56.047760 1 tasks_processing.go:71] worker 3 working on machine_config_pools task. W0701 19:44:56.047763 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0701 19:44:56.047778 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 25.352µs to process 0 records I0701 19:44:56.047788 1 tasks_processing.go:71] worker 6 working on config_maps task. I0701 19:44:56.047654 1 tasks_processing.go:71] worker 0 working on authentication task. I0701 19:44:56.047634 1 tasks_processing.go:69] worker 20 listening for tasks. I0701 19:44:56.047892 1 tasks_processing.go:71] worker 20 working on image task. I0701 19:44:56.047920 1 tasks_processing.go:71] worker 26 working on machine_sets task. I0701 19:44:56.047938 1 tasks_processing.go:71] worker 1 working on install_plans task. I0701 19:44:56.047989 1 tasks_processing.go:71] worker 22 working on ceph_cluster task. I0701 19:44:56.047574 1 tasks_processing.go:69] worker 11 listening for tasks. I0701 19:44:56.048035 1 tasks_processing.go:71] worker 30 working on support_secret task. I0701 19:44:56.048044 1 tasks_processing.go:71] worker 14 working on sap_datahubs task. I0701 19:44:56.048063 1 tasks_processing.go:71] worker 18 working on version task. I0701 19:44:56.047654 1 tasks_processing.go:71] worker 29 working on node_logs task. I0701 19:44:56.048287 1 tasks_processing.go:71] worker 24 working on container_runtime_configs task. I0701 19:44:56.048400 1 tasks_processing.go:71] worker 25 working on mutating_webhook_configurations task. I0701 19:44:56.048713 1 tasks_processing.go:71] worker 19 working on openshift_machine_api_events task. I0701 19:44:56.047742 1 tasks_processing.go:71] worker 2 working on crds task. I0701 19:44:56.048970 1 tasks_processing.go:71] worker 11 working on ingress_certificates task. I0701 19:44:56.050381 1 tasks_processing.go:71] worker 31 working on openstack_dataplanedeployments task. I0701 19:44:56.050566 1 tasks_processing.go:69] worker 21 listening for tasks. I0701 19:44:56.050626 1 tasks_processing.go:71] worker 21 working on nodes task. I0701 19:44:56.050576 1 tasks_processing.go:71] worker 28 working on active_alerts task. I0701 19:44:56.050767 1 tasks_processing.go:71] worker 17 working on operators task. W0701 19:44:56.050784 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0701 19:44:56.051040 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 266.81µs to process 0 records I0701 19:44:56.047659 1 tasks_processing.go:71] worker 4 working on pdbs task. I0701 19:44:56.051100 1 tasks_processing.go:71] worker 28 working on certificate_signing_requests task. I0701 19:44:56.050606 1 tasks_processing.go:71] worker 9 working on openstack_dataplanenodesets task. I0701 19:44:56.050783 1 tasks_processing.go:71] worker 27 working on sap_pods task. I0701 19:44:56.052839 1 tasks_processing.go:71] worker 12 working on machines task. I0701 19:44:56.052861 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 5.143882ms to process 0 records I0701 19:44:56.052957 1 tasks_processing.go:71] worker 7 working on olm_operators task. E0701 19:44:56.052973 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0701 19:44:56.052988 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 5.199248ms to process 0 records I0701 19:44:56.053036 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0701 19:44:56.053057 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0701 19:44:56.053065 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0701 19:44:56.053070 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0701 19:44:56.053091 1 controller.go:489] The operator is still being initialized I0701 19:44:56.053103 1 controller.go:512] The operator is healthy I0701 19:44:56.055789 1 tasks_processing.go:71] worker 26 working on storage_classes task. I0701 19:44:56.055866 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 7.855152ms to process 0 records I0701 19:44:56.055882 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 7.507959ms to process 0 records I0701 19:44:56.055903 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 8.130492ms to process 0 records I0701 19:44:56.055943 1 tasks_processing.go:71] worker 3 working on overlapping_namespace_uids task. I0701 19:44:56.055956 1 tasks_processing.go:71] worker 24 working on feature_gates task. I0701 19:44:56.055948 1 tasks_processing.go:71] worker 5 working on silenced_alerts task. W0701 19:44:56.056084 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0701 19:44:56.056175 1 recorder.go:75] Recording config/apiserver with fingerprint=739036979325439281e75fe52889bf4d695d4a5aa949e5799a643373d69cfaf5 I0701 19:44:56.056196 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 8.256853ms to process 1 records I0701 19:44:56.056284 1 tasks_processing.go:71] worker 0 working on machine_configs task. I0701 19:44:56.056444 1 recorder.go:75] Recording config/authentication with fingerprint=5dfe12869df72cc8a2daafd364117461aa052cc111206afdfb0ddb93678da0da I0701 19:44:56.056460 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 8.24264ms to process 1 records I0701 19:44:56.056472 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 32.813µs to process 0 records I0701 19:44:56.056485 1 tasks_processing.go:71] worker 5 working on ingress task. I0701 19:44:56.059857 1 tasks_processing.go:71] worker 13 working on service_accounts task. E0701 19:44:56.059867 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0701 19:44:56.059878 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 12.195575ms to process 0 records I0701 19:44:56.062106 1 tasks_processing.go:71] worker 8 working on image_pruners task. I0701 19:44:56.062134 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 14.40579ms to process 0 records I0701 19:44:56.067693 1 tasks_processing.go:71] worker 30 working on clusterroles task. E0701 19:44:56.067716 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0701 19:44:56.067726 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 19.650288ms to process 0 records I0701 19:44:56.067754 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 20.082869ms to process 0 records I0701 19:44:56.067768 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 17.667414ms to process 0 records I0701 19:44:56.067818 1 tasks_processing.go:71] worker 19 working on dvo_metrics task. I0701 19:44:56.067833 1 tasks_processing.go:71] worker 16 working on schedulers task. I0701 19:44:56.067922 1 tasks_processing.go:71] worker 20 working on nodenetworkstates task. I0701 19:44:56.068040 1 recorder.go:75] Recording config/image with fingerprint=18cc3b3b0272fb23ea87516a610eaeacf03d9a4eba9e146011578a94dd5df0c7 I0701 19:44:56.068060 1 gather.go:177] gatherer "clusterconfig" function "image" took 20.013183ms to process 1 records I0701 19:44:56.068080 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 20.023695ms to process 0 records I0701 19:44:56.068092 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 20.493689ms to process 0 records I0701 19:44:56.068100 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 20.078095ms to process 0 records I0701 19:44:56.068107 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 16.839886ms to process 0 records I0701 19:44:56.068121 1 tasks_processing.go:71] worker 22 working on validating_webhook_configurations task. I0701 19:44:56.068127 1 tasks_processing.go:71] worker 10 working on machine_autoscalers task. I0701 19:44:56.068178 1 tasks_processing.go:71] worker 14 working on aggregated_monitoring_cr_names task. I0701 19:44:56.068184 1 tasks_processing.go:71] worker 9 working on sap_config task. I0701 19:44:56.068397 1 tasks_processing.go:71] worker 27 working on infrastructures task. I0701 19:44:56.068407 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 17.018933ms to process 0 records I0701 19:44:56.068574 1 tasks_processing.go:71] worker 25 working on qemu_kubevirt_launcher_logs task. I0701 19:44:56.068797 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=b18daeff97304f426f8711c89cd62f7a6976cf3dcb5d40ac5103bb0ee5c0d2d6 I0701 19:44:56.068834 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=6a0e05f6268c870465453cb05555422d010832fc4cbdfaf95ff38d2d67993fee I0701 19:44:56.068860 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=6b4524f09bdb81b9f4a24a596db94f98744bebc390c6cd68f4fe11fd3bae3b7d I0701 19:44:56.068872 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 20.116793ms to process 3 records I0701 19:44:56.071444 1 tasks_processing.go:71] worker 12 working on nodenetworkconfigurationpolicies task. E0701 19:44:56.071507 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0701 19:44:56.071521 1 gather.go:177] gatherer "clusterconfig" function "machines" took 18.592164ms to process 0 records I0701 19:44:56.071530 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 21.066724ms to process 0 records I0701 19:44:56.071573 1 tasks_processing.go:71] worker 31 working on storage_cluster task. I0701 19:44:56.071613 1 tasks_processing.go:71] worker 4 working on openstack_controlplanes task. I0701 19:44:56.071795 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=5f8820e0790f6369291dbcd97b0785701d278f47400aacd5a7ae5eabe7bb40d5 I0701 19:44:56.071835 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=cf92a4137caffff4537f88ab68b15569d05eb15c1a24cd6d6dc9f4e53d4c42b5 I0701 19:44:56.071872 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=050ff81c1a532494b7520a533cb47ced7b4ac5b0638960beb99c741f77429315 I0701 19:44:56.071888 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 20.549001ms to process 3 records I0701 19:44:56.071908 1 tasks_processing.go:71] worker 8 working on oauths task. I0701 19:44:56.072093 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0701 19:44:56.072157 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0701 19:44:56.072227 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=3e5803b7b22e1d697d9f5d9a8ed8ce17e7af63bce3a50118761382fd374039ce W0701 19:44:56.072294 1 operator.go:288] started I0701 19:44:56.072300 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 9.789577ms to process 1 records I0701 19:44:56.072324 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0701 19:44:56.083601 1 tasks_processing.go:71] worker 26 working on image_registries task. I0701 19:44:56.083716 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=dafcbe4da28039954663aa320efdc01aaeb8502d5bfb07b8fd26f09716b8c2be I0701 19:44:56.083741 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=dfb3948ebf07641d03773ce3aa63cb9592f3a7755de924ba1997e558f7d29d6b I0701 19:44:56.083754 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 27.790948ms to process 2 records I0701 19:44:56.083824 1 tasks_processing.go:71] worker 5 working on networks task. I0701 19:44:56.083969 1 recorder.go:75] Recording config/ingress with fingerprint=f72b3d7441970fdacc482191c109c8949940ea58536bb4e8551493413b051483 I0701 19:44:56.083983 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 27.326433ms to process 1 records I0701 19:44:56.087008 1 tasks_processing.go:71] worker 7 working on metrics task. I0701 19:44:56.087015 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 34.031541ms to process 0 records W0701 19:44:56.087040 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0701 19:44:56.087048 1 tasks_processing.go:71] worker 7 working on openshift_logging task. I0701 19:44:56.087088 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.473µs to process 0 records I0701 19:44:56.089290 1 tasks_processing.go:71] worker 16 working on cost_management_metrics_configs task. I0701 19:44:56.089486 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=694f21c45546b25b0ff6a2dfa1fa48d02c456708f807cf9ca3d0b4ddbcb40791 I0701 19:44:56.089506 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 21.446645ms to process 1 records I0701 19:44:56.089610 1 tasks_processing.go:71] worker 20 working on proxies task. I0701 19:44:56.089645 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 21.638324ms to process 0 records I0701 19:44:56.089687 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 21.515433ms to process 0 records I0701 19:44:56.089712 1 tasks_processing.go:71] worker 10 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0701 19:44:56.089767 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 21.546546ms to process 0 records I0701 19:44:56.089749 1 tasks_processing.go:74] worker 9 stopped. I0701 19:44:56.091037 1 tasks_processing.go:74] worker 12 stopped. I0701 19:44:56.091095 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 19.578572ms to process 0 records I0701 19:44:56.091204 1 tasks_processing.go:74] worker 24 stopped. I0701 19:44:56.091481 1 recorder.go:75] Recording config/featuregate with fingerprint=78dc1d661245e14b7f9679fbe66b7f1ce4ac3c8b7db24789674624a63dd4fa80 I0701 19:44:56.091549 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 35.23737ms to process 1 records I0701 19:44:56.092210 1 tasks_processing.go:74] worker 31 stopped. I0701 19:44:56.092226 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 20.621307ms to process 0 records I0701 19:44:56.092504 1 tasks_processing.go:74] worker 4 stopped. I0701 19:44:56.092521 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 20.839142ms to process 0 records I0701 19:44:56.092634 1 tasks_processing.go:74] worker 8 stopped. I0701 19:44:56.092906 1 recorder.go:75] Recording config/oauth with fingerprint=8c193bd8a2f4e6a1b9f370d3bdd133114da524dd7122902735a6a92d387d10eb I0701 19:44:56.092973 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 20.584504ms to process 1 records I0701 19:44:56.094432 1 tasks_processing.go:74] worker 7 stopped. I0701 19:44:56.094450 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 7.374337ms to process 0 records I0701 19:44:56.094724 1 tasks_processing.go:74] worker 27 stopped. I0701 19:44:56.094770 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0701 19:44:56.094785 1 controller.go:212] Source scaController *sca.Controller is not ready I0701 19:44:56.094791 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0701 19:44:56.094796 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0701 19:44:56.094800 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0701 19:44:56.094821 1 controller.go:489] The operator is still being initialized I0701 19:44:56.094831 1 controller.go:512] The operator is healthy I0701 19:44:56.095448 1 recorder.go:75] Recording config/infrastructure with fingerprint=54a3ab1616b8bf36ec2dfcf43d08ff629024004076211468521ec7fc63f6f1d5 I0701 19:44:56.095465 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 26.289024ms to process 1 records I0701 19:44:56.095495 1 tasks_processing.go:74] worker 28 stopped. I0701 19:44:56.095513 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 44.372419ms to process 0 records I0701 19:44:56.096331 1 tasks_processing.go:74] worker 16 stopped. I0701 19:44:56.096388 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 6.972428ms to process 0 records I0701 19:44:56.096658 1 recorder.go:75] Recording config/network with fingerprint=307cbe0f5435b79f5e23eb4152a62c37c239333bb5160c292340e3ad0ff16cf7 I0701 19:44:56.096730 1 gather.go:177] gatherer "clusterconfig" function "networks" took 12.576305ms to process 1 records I0701 19:44:56.096916 1 tasks_processing.go:74] worker 5 stopped. I0701 19:44:56.097138 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=b6fa8730ad6365e4c716fe7541de23f9179d38a87f7c2cc2a814e8f6c43d7a24 I0701 19:44:56.097385 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=7f8b6df96526127f62d3199e1c6de0e3b74f2953a4f1db1b3a7af55c5e0ec546 I0701 19:44:56.097427 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=0b4a51f0268e6a18e20db8c881dcfe406e0a5a5bf5e489704eb0e5c9d610ace0 I0701 19:44:56.097480 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=d25a932e96a57b893321846817079aab9c75898c18bec87a20542b856c77087e I0701 19:44:56.097524 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=06f8f664bef45a36392edc951628cc704f0ec68b01d5e82f5ae9d9274cfb6da8 I0701 19:44:56.097525 1 prometheus_rules.go:88] Prometheus rules successfully created I0701 19:44:56.097813 1 tasks_processing.go:74] worker 22 stopped. I0701 19:44:56.098583 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=77e72562dcf618c69f0b8bd0b5346234518f99a91f39d4a7c5164b9298aed29e I0701 19:44:56.098639 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=3af199153ef33ac9e8a80a9dfe44c847e28c7bf68ce53ead3bc86b50e9f1e491 I0701 19:44:56.098718 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=bcda980323ea089ef148565a1d71276683a1da0efcdd6728b2ad9cb147b72271 I0701 19:44:56.098812 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=88325f8473293c692cebf8d4658e004a239e736c30908ac1c7c73ff5d3488f7a I0701 19:44:56.098872 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=947db6500ec474b15cebfde95db25d78038fb5a729bc20ea05087b917498993b I0701 19:44:56.098938 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=b93958f5e53cf6136591048335af1fd98c5da5b5b52e683a5493375fa552928e I0701 19:44:56.098987 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 28.59494ms to process 11 records I0701 19:44:56.099030 1 tasks_processing.go:74] worker 3 stopped. I0701 19:44:56.099094 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0701 19:44:56.099127 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 43.069655ms to process 1 records W0701 19:44:56.099737 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:44:56.102509 1 tasks_processing.go:74] worker 20 stopped. I0701 19:44:56.102597 1 recorder.go:75] Recording config/proxy with fingerprint=d6a62e8f21418b92247216b5bcce9143d178ef69e5412853361d04daa3126c8f I0701 19:44:56.102666 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 12.879708ms to process 1 records I0701 19:44:56.102779 1 tasks_processing.go:74] worker 2 stopped. I0701 19:44:56.103496 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=31207cadfd983176498b9263711956e9c7f9e6fedf79ab09da171567ddc79ac7 I0701 19:44:56.103810 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=df0ec1848ef6c3531f1e3bcf810283eb395cbb87b3fb5c214e257143bcc13bc9 I0701 19:44:56.103824 1 gather.go:177] gatherer "clusterconfig" function "crds" took 52.443594ms to process 2 records I0701 19:44:56.103918 1 tasks_processing.go:74] worker 30 stopped. I0701 19:44:56.104080 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=4fc1b0829c7de8e84335d40c5c5441932485de4e6ac794a08d8819521aab3410 I0701 19:44:56.104201 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=2257e3680e9b411edee995174f97ce7c427a67a1e3fb7c0f89dd6baf22a68479 I0701 19:44:56.104210 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 35.096967ms to process 2 records I0701 19:44:56.104298 1 tasks_processing.go:74] worker 26 stopped. I0701 19:44:56.104580 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=a2b014963681a8d48bc199c68dd2c01aba19081224dc2ba470f1eb736368f5ad I0701 19:44:56.104591 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 19.406129ms to process 1 records I0701 19:44:56.106721 1 tasks_processing.go:74] worker 14 stopped. I0701 19:44:56.106739 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 38.520767ms to process 0 records E0701 19:44:56.106914 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27b03129ce-c613-4c8d-aff3-0e949bf6aea2%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.13:46104->172.30.0.10:53: read: connection refused I0701 19:44:56.106926 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27b03129ce-c613-4c8d-aff3-0e949bf6aea2%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.13:46104->172.30.0.10:53: read: connection refused I0701 19:44:56.108096 1 tasks_processing.go:74] worker 15 stopped. I0701 19:44:56.109222 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-hnzxj with fingerprint=28621b2257933f39cfa9dd28a3397ba9a1921983d9d207ffbd9195dde4347bb3 I0701 19:44:56.109332 1 recorder.go:75] Recording config/pod/openshift-multus/multus-8qn89 with fingerprint=7a5c0e349eef512ca4e2f3fdb73369df6e3b469e54732dff720331c61a0790e5 I0701 19:44:56.109559 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-ljlcq with fingerprint=2ca3d97a2d151f4d308091ff03cd7e86c69237fd635322867c46aa805628bf81 I0701 19:44:56.109599 1 recorder.go:75] Recording config/running_containers with fingerprint=042022c72c5a13b0abd71c505b4abd067b6e87aa4bb93176a56290a542662f2a I0701 19:44:56.109608 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 60.384619ms to process 4 records I0701 19:44:56.112011 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0701 19:44:56.112032 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0701 19:44:56.112056 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0701 19:44:56.114247 1 base_controller.go:82] Caches are synced for ConfigController I0701 19:44:56.114260 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0701 19:44:56.117636 1 tasks_processing.go:74] worker 6 stopped. E0701 19:44:56.117648 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0701 19:44:56.117654 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0701 19:44:56.117658 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0701 19:44:56.117667 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0701 19:44:56.117719 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0701 19:44:56.117726 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0701 19:44:56.117731 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0701 19:44:56.117735 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0701 19:44:56.117771 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0701 19:44:56.117779 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0701 19:44:56.117784 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 69.833475ms to process 7 records I0701 19:44:56.119857 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:44:56.137651 1 tasks_processing.go:74] worker 29 stopped. I0701 19:44:56.137662 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 89.409158ms to process 0 records I0701 19:44:56.138421 1 tasks_processing.go:74] worker 21 stopped. I0701 19:44:56.138711 1 recorder.go:75] Recording config/node/ip-10-0-0-56.ec2.internal with fingerprint=de40b40a327f156cbaeef773135d47aac0f3ca330b7a11b4f1308d1910b2b0d4 I0701 19:44:56.138772 1 recorder.go:75] Recording config/node/ip-10-0-1-54.ec2.internal with fingerprint=c249dc0b837b2935405645c96cbdebf174c9a06dc01aa56e834cac336eb860dc I0701 19:44:56.138822 1 recorder.go:75] Recording config/node/ip-10-0-2-238.ec2.internal with fingerprint=c213877ada6cdae770d9bf011d32d5e3feb003e9224d1f9e95f28cf136d3c4b0 I0701 19:44:56.138830 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 87.759837ms to process 3 records I0701 19:44:56.148492 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0701 19:44:56.151923 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.13:52251->172.30.0.10:53: read: connection refused I0701 19:44:56.151935 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.13:52251->172.30.0.10:53: read: connection refused I0701 19:44:56.163782 1 tasks_processing.go:74] worker 18 stopped. I0701 19:44:56.163818 1 gather_logs.go:145] no pods in namespace were found I0701 19:44:56.164132 1 recorder.go:75] Recording config/version with fingerprint=a0ff4b4dee2c17164524e46808a810a9515b8f8e30b29aec918cea29b56ceecc I0701 19:44:56.164149 1 recorder.go:75] Recording config/id with fingerprint=403a989d24a4c30ae891481a69408dd2aebc2ff01ba2b0b07cb4dfa12e4d1305 I0701 19:44:56.164158 1 gather.go:177] gatherer "clusterconfig" function "version" took 115.710797ms to process 2 records I0701 19:44:56.164172 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 95.241286ms to process 0 records I0701 19:44:56.164182 1 tasks_processing.go:74] worker 25 stopped. I0701 19:44:56.172650 1 base_controller.go:82] Caches are synced for LoggingSyncer I0701 19:44:56.172661 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0701 19:44:56.180396 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0701 19:44:56.183053 1 tasks_processing.go:74] worker 10 stopped. I0701 19:44:56.183066 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 93.285457ms to process 0 records I0701 19:44:56.200513 1 tasks_processing.go:74] worker 11 stopped. E0701 19:44:56.200533 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0701 19:44:56.200543 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2r9hocmc24gutgt57e2bmcpr09hu1mnp-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2r9hocmc24gutgt57e2bmcpr09hu1mnp-primary-cert-bundle-secret" not found I0701 19:44:56.200618 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=7e47a55485f4b6d45c391aec58add9df6fbed5b0cd4b3eb4fbbfaf31b5bd1d15 I0701 19:44:56.200637 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 151.170855ms to process 1 records I0701 19:44:56.234518 1 tasks_processing.go:74] worker 0 stopped. I0701 19:44:56.234552 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0701 19:44:56.234564 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 178.21261ms to process 1 records I0701 19:44:56.503253 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0701 19:44:56.503268 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0701 19:44:56.503476 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-856wm pod in namespace openshift-dns (previous: false). I0701 19:44:56.733106 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-856wm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-856wm\" is waiting to start: ContainerCreating" I0701 19:44:56.733128 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-856wm\" is waiting to start: ContainerCreating" I0701 19:44:56.733138 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-856wm pod in namespace openshift-dns (previous: false). I0701 19:44:56.908766 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-856wm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-856wm\" is waiting to start: ContainerCreating" I0701 19:44:56.908784 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-856wm\" is waiting to start: ContainerCreating" I0701 19:44:56.908812 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-s9mjt pod in namespace openshift-dns (previous: false). W0701 19:44:57.096507 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:44:57.130443 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-s9mjt pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-s9mjt\" is waiting to start: ContainerCreating" I0701 19:44:57.130460 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-s9mjt\" is waiting to start: ContainerCreating" I0701 19:44:57.130468 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-s9mjt pod in namespace openshift-dns (previous: false). I0701 19:44:57.309045 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-s9mjt pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-s9mjt\" is waiting to start: ContainerCreating" I0701 19:44:57.309065 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-s9mjt\" is waiting to start: ContainerCreating" I0701 19:44:57.309078 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-vmqv4 pod in namespace openshift-dns (previous: false). I0701 19:44:57.531201 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-vmqv4 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-vmqv4\" is waiting to start: ContainerCreating" I0701 19:44:57.531221 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-vmqv4\" is waiting to start: ContainerCreating" I0701 19:44:57.531233 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-vmqv4 pod in namespace openshift-dns (previous: false). I0701 19:44:57.707976 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-vmqv4 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-vmqv4\" is waiting to start: ContainerCreating" I0701 19:44:57.707993 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-vmqv4\" is waiting to start: ContainerCreating" I0701 19:44:57.708006 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-dqzw5 pod in namespace openshift-dns (previous: false). I0701 19:44:57.721081 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0701 19:44:57.908095 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:44:57.908114 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-p72rb pod in namespace openshift-dns (previous: false). W0701 19:44:58.097106 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:44:58.108833 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:44:58.108856 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-qc6px pod in namespace openshift-dns (previous: false). I0701 19:44:58.308460 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:44:58.308483 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5f589495cd-4wqkf pod in namespace openshift-image-registry (previous: false). I0701 19:44:58.324114 1 tasks_processing.go:74] worker 17 stopped. I0701 19:44:58.324167 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=5cd1fd112aa10bef8d8941651867f4def9001d5b4b4f9e7f327ea53ca393068c I0701 19:44:58.324199 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=7f288ea96226c2945d3ac61b98407f7f1e54db271d28408cc21579c141f46c5e I0701 19:44:58.324228 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0701 19:44:58.324270 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=96dbc0afad5588b11e52fa0e2a44fe195bd2b7d00f89aca5a848458a32fedcb4 I0701 19:44:58.324286 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0701 19:44:58.324310 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=0f89f0413f05dfafbc3e2288c120938b70f77262ce5b467a60437dd7ad5a3296 I0701 19:44:58.324343 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=36e132ee7b7ad2234035c0f74cb0fae14dbf092ce035a811d78f04b2e80d5dd8 I0701 19:44:58.324365 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=e72d82e502e67017aee8ab160ed433d4bdaf177447c08a33ad979df9d0fb6b58 I0701 19:44:58.324395 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=7e09b7ebc7bcc4a21497fbb27e76c2fceea7bf94a6b4fbf08568e2379abcf382 I0701 19:44:58.324404 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e5ff11d57817f84a678f6fa9565af55bd1120227c16a21933637ab62675a6d70 I0701 19:44:58.324421 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=ba6e4d32e33905c570e66d3848849657cbd0b01e86c375f225c6faf739db78b6 I0701 19:44:58.324431 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0701 19:44:58.324447 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=c6e40124afe0a8279231d6588535c49342f11658df345fd12be6d93d85b78b78 I0701 19:44:58.324456 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0701 19:44:58.324472 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=44595e2e2b62e18de59680dd108a1bf9a24a141f2801783873bd622d75c9fc47 I0701 19:44:58.324481 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0701 19:44:58.324500 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=a1d98762a1e2bf7362242d1495fd42d0f02b41dc4cc6f588caebcc1c438ecb20 I0701 19:44:58.324509 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0701 19:44:58.324524 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=e0f7cfb3efbc29482924494a3640b2f79d01f67117f09115a18bebd12e31ad9e I0701 19:44:58.324644 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=1d99fe26fc955546db86d230f3d4107d82ade6826faf5f452150403f225e67ca I0701 19:44:58.324653 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0701 19:44:58.324661 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0701 19:44:58.324695 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0701 19:44:58.324716 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=63423d4b45862fafe026ed715928393159f6ecaea3c2d8500ed33f9f9644ef31 I0701 19:44:58.324740 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=a9c1bbb0fee6ca35a4727d43039cc7ba5b4782f8832c110d7df4c019648d031c I0701 19:44:58.324749 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0701 19:44:58.324764 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=aaf532a449d28d58d2a8488410efdb5b2ab571cf5b7467b095159c44ed283bc3 I0701 19:44:58.324773 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0701 19:44:58.324786 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=50e2fb74d2c01bed2f0d26f6f9e6b2386aaeda96cc247177db18e0ea0be33aa9 I0701 19:44:58.324801 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=3e8f5e05b403446a8e4a4d36bd14714317e9aa81e132a4d5d789894291c1dba9 I0701 19:44:58.324814 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=a2e25d7c58fa7161b831c5ff5b031593334bd7689ca03d56e986f7d0b69dccc8 I0701 19:44:58.324829 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=cbd990fca00907a68f354658a7a1f73029af45d78d65f94c47116c673143c6db I0701 19:44:58.324850 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=6d29a816e20b3249ec0d49dcd9fb96e6bee481572f058b8ab753fff05597f14b I0701 19:44:58.324859 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0701 19:44:58.324886 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=df2d50da1e1c20554b54b05880dfca482519ba459559bb970bfd61813aac32ad I0701 19:44:58.324902 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0701 19:44:58.324911 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0701 19:44:58.324918 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.273282845s to process 37 records I0701 19:44:58.507907 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5f589495cd-4wqkf pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5f589495cd-4wqkf\" is waiting to start: ContainerCreating" I0701 19:44:58.507925 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5f589495cd-4wqkf\" is waiting to start: ContainerCreating" I0701 19:44:58.507936 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5f589495cd-dslqq pod in namespace openshift-image-registry (previous: false). I0701 19:44:58.708171 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5f589495cd-dslqq pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5f589495cd-dslqq\" is waiting to start: ContainerCreating" I0701 19:44:58.708199 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5f589495cd-dslqq\" is waiting to start: ContainerCreating" I0701 19:44:58.708208 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-78c8878d5-2l276 pod in namespace openshift-image-registry (previous: false). I0701 19:44:58.909001 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-78c8878d5-2l276 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-78c8878d5-2l276\" is waiting to start: ContainerCreating" I0701 19:44:58.909018 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-78c8878d5-2l276\" is waiting to start: ContainerCreating" I0701 19:44:58.909027 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-cfj7q pod in namespace openshift-image-registry (previous: false). W0701 19:44:59.096986 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:44:59.107582 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:44:59.107602 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-dznkm pod in namespace openshift-image-registry (previous: false). I0701 19:44:59.307712 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:44:59.307730 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-v8mrq pod in namespace openshift-image-registry (previous: false). I0701 19:44:59.508158 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0701 19:44:59.508178 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6f579d4fdd-d6dx5 pod in namespace openshift-ingress (previous: false). I0701 19:44:59.708155 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6f579d4fdd-d6dx5 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6f579d4fdd-d6dx5\" is waiting to start: ContainerCreating" I0701 19:44:59.708172 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6f579d4fdd-d6dx5\" is waiting to start: ContainerCreating" I0701 19:44:59.708181 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6f579d4fdd-khfnd pod in namespace openshift-ingress (previous: false). I0701 19:44:59.907356 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6f579d4fdd-khfnd pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6f579d4fdd-khfnd\" is waiting to start: ContainerCreating" I0701 19:44:59.907376 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6f579d4fdd-khfnd\" is waiting to start: ContainerCreating" I0701 19:44:59.907389 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-84fcdf4669-v92ng pod in namespace openshift-ingress (previous: false). W0701 19:45:00.096319 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:45:00.108605 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-84fcdf4669-v92ng pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-84fcdf4669-v92ng\" is waiting to start: ContainerCreating" I0701 19:45:00.108621 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-84fcdf4669-v92ng\" is waiting to start: ContainerCreating" I0701 19:45:00.108635 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-g2wbn pod in namespace openshift-ingress-canary (previous: false). I0701 19:45:00.308965 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-g2wbn pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-g2wbn\" is waiting to start: ContainerCreating" I0701 19:45:00.308982 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-g2wbn\" is waiting to start: ContainerCreating" I0701 19:45:00.308993 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-ls24p pod in namespace openshift-ingress-canary (previous: false). I0701 19:45:00.507977 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-ls24p pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-ls24p\" is waiting to start: ContainerCreating" I0701 19:45:00.507995 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-ls24p\" is waiting to start: ContainerCreating" I0701 19:45:00.508005 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-wj2gk pod in namespace openshift-ingress-canary (previous: false). I0701 19:45:00.709226 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-wj2gk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-wj2gk\" is waiting to start: ContainerCreating" I0701 19:45:00.709244 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-wj2gk\" is waiting to start: ContainerCreating" I0701 19:45:00.709260 1 tasks_processing.go:74] worker 23 stopped. I0701 19:45:00.709343 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=3a669db989ce6dcff4cddbc84d54313171b4d6654e9e369ec1d242cb9f85e492 I0701 19:45:00.709407 1 recorder.go:75] Recording events/openshift-dns with fingerprint=f8b22008ed3a6bca43a5e64785da63708739a6487bc87aad8f75b3c385f1916d I0701 19:45:00.709491 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=302d18db9daff9dde590bad9862198999bf5d34728bb777f0e771138fff3c736 I0701 19:45:00.709517 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=3c103dab73217cb8fc5f80dc56678f2a4d0298818dcb4e05ea887cdbd74529d0 I0701 19:45:00.709565 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=7622645d8010dad4872717b212785cbf9ef79805349bb6f81cf3c071434b8c5e I0701 19:45:00.709583 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=a915fefbbbe6e6ddc9bba8ad02924283445a952563ab8c73f7744decfd1e0975 I0701 19:45:00.709590 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.661500939s to process 6 records W0701 19:45:01.096445 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0701 19:45:01.096477 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0701 19:45:01.096493 1 tasks_processing.go:74] worker 19 stopped. E0701 19:45:01.096503 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0701 19:45:01.096513 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0701 19:45:01.096529 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0701 19:45:01.096539 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.028657104s to process 1 records I0701 19:45:07.078771 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:45:09.103523 1 tasks_processing.go:74] worker 1 stopped. I0701 19:45:09.103560 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0701 19:45:09.103572 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.05556756s to process 1 records I0701 19:45:09.869636 1 tasks_processing.go:74] worker 13 stopped. I0701 19:45:09.869916 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=3a23a80037b37537e6bce19267a0686640cc3d2d84bcc6f7bc973979a947d17d I0701 19:45:09.869933 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.80975641s to process 1 records E0701 19:45:09.869989 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.822s with: function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"machines\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0701 19:45:09.871098 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machines" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0701 19:45:09.871112 1 periodic.go:209] Running workloads gatherer I0701 19:45:09.871126 1 tasks_processing.go:45] number of workers: 2 I0701 19:45:09.871134 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:45:09.871137 1 tasks_processing.go:71] worker 1 working on workload_info task. I0701 19:45:09.871150 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:45:09.871165 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0701 19:45:09.895761 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0701 19:45:09.901291 1 tasks_processing.go:74] worker 0 stopped. I0701 19:45:09.901308 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 30.111557ms to process 0 records I0701 19:45:09.903206 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (8ms) I0701 19:45:09.911015 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0701 19:45:09.918427 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (7ms) I0701 19:45:09.925924 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (7ms) I0701 19:45:09.933395 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (7ms) I0701 19:45:09.940808 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (7ms) I0701 19:45:09.948220 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (7ms) I0701 19:45:09.955725 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (7ms) I0701 19:45:09.963385 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (8ms) I0701 19:45:09.970575 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (7ms) I0701 19:45:10.004156 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (34ms) I0701 19:45:10.104493 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0701 19:45:10.205485 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0701 19:45:10.304416 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (99ms) I0701 19:45:10.404174 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (100ms) I0701 19:45:10.474043 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:45:10.503874 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0701 19:45:10.605162 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0701 19:45:10.674936 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:45:10.705275 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0701 19:45:10.804064 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0701 19:45:10.904071 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0701 19:45:11.005289 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (101ms) I0701 19:45:11.104594 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (99ms) I0701 19:45:11.204220 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0701 19:45:11.304378 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0701 19:45:11.405089 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (101ms) I0701 19:45:11.503861 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (99ms) I0701 19:45:11.604029 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0701 19:45:11.714629 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (111ms) I0701 19:45:11.806350 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (92ms) I0701 19:45:11.904127 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (98ms) I0701 19:45:12.004000 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0701 19:45:12.103840 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0701 19:45:12.204989 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0701 19:45:12.205021 1 tasks_processing.go:74] worker 1 stopped. E0701 19:45:12.205031 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0701 19:45:12.205333 1 recorder.go:75] Recording config/workload_info with fingerprint=44e700d7b59427ff777c81027b487724692552335229c559cce951ce045b94ac I0701 19:45:12.205347 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.333876803s to process 1 records E0701 19:45:12.205372 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.334s with: function \"workload_info\" failed with an error" I0701 19:45:12.206480 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0701 19:45:12.206493 1 periodic.go:209] Running conditional gatherer I0701 19:45:12.212435 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0701 19:45:12.218956 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.13:56647->172.30.0.10:53: read: connection refused E0701 19:45:12.219195 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0701 19:45:12.219259 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0701 19:45:12.225452 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0701 19:45:12.225470 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225477 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225484 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225488 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225493 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225496 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225501 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225506 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0701 19:45:12.225510 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0701 19:45:12.225530 1 tasks_processing.go:45] number of workers: 3 I0701 19:45:12.225547 1 tasks_processing.go:69] worker 2 listening for tasks. I0701 19:45:12.225552 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0701 19:45:12.225554 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:45:12.225570 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:45:12.225580 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0701 19:45:12.225584 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0701 19:45:12.225597 1 tasks_processing.go:74] worker 1 stopped. I0701 19:45:12.225660 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0701 19:45:12.225693 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.209µs to process 1 records I0701 19:45:12.225727 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0701 19:45:12.225736 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.29µs to process 1 records I0701 19:45:12.225743 1 tasks_processing.go:74] worker 0 stopped. I0701 19:45:12.225859 1 tasks_processing.go:74] worker 2 stopped. I0701 19:45:12.225874 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 300.031µs to process 0 records I0701 19:45:12.225908 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.13:56647->172.30.0.10:53: read: connection refused I0701 19:45:12.225935 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0701 19:45:12.247390 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=aa82f3c249e2ffbcb4bb62df71bef0a838082fc1d3bf28dbb8101121d6aea6c4 I0701 19:45:12.247561 1 diskrecorder.go:70] Writing 104 records to /var/lib/insights-operator/insights-2026-07-01-194512.tar.gz I0701 19:45:12.254201 1 diskrecorder.go:51] Wrote 104 records to disk in 6ms I0701 19:45:12.254230 1 periodic.go:278] Gathering cluster info every 2h0m0s I0701 19:45:12.254252 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0701 19:45:21.104118 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:46:15.396897 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="f0d943700fb8b5446ac9ddd544ad2b03945c7581cc0fb01f57516fff2828b44c") W0701 19:46:15.396932 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0701 19:46:15.396997 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="c95bd33fb3ea2c05ebb8648065521cd5f772dd4689da160e078a590d2172af5e") I0701 19:46:15.397008 1 base_controller.go:181] Shutting down ConfigController ... I0701 19:46:15.397046 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0701 19:46:15.397068 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0701 19:46:15.397092 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="18705143dcf9fc3df607a304b42aa80419edef725ef3d427c5ad8df29454f092") I0701 19:46:15.397098 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated"