May 25 13:03:23.331239 ip-10-0-142-251 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 25 13:03:23.331251 ip-10-0-142-251 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 25 13:03:23.331259 ip-10-0-142-251 systemd[1]: kubelet.service: Failed with result 'resources'. May 25 13:03:23.331482 ip-10-0-142-251 systemd[1]: Failed to start Kubernetes Kubelet. May 25 13:03:33.540624 ip-10-0-142-251 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 25 13:03:33.540643 ip-10-0-142-251 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 70ec825b5e434ee3ba8732e3d72c5ab8 -- May 25 13:05:46.303281 ip-10-0-142-251 systemd[1]: Starting Kubernetes Kubelet... May 25 13:05:46.723044 ip-10-0-142-251 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 25 13:05:46.723044 ip-10-0-142-251 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 25 13:05:46.723044 ip-10-0-142-251 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 25 13:05:46.723044 ip-10-0-142-251 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 25 13:05:46.723044 ip-10-0-142-251 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 25 13:05:46.727756 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.727650 2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 25 13:05:46.733863 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733838 2577 feature_gate.go:328] unrecognized feature gate: NewOLM May 25 13:05:46.733863 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733861 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 25 13:05:46.733863 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733866 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733880 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733884 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733887 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733890 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733893 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733902 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733905 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733908 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733911 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733913 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733916 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733919 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733922 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733924 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733927 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733929 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733932 2577 feature_gate.go:328] unrecognized feature gate: Example2 May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733935 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 25 13:05:46.733979 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733937 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733940 2577 feature_gate.go:328] unrecognized feature gate: DualReplica May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733943 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733945 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733948 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733951 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733953 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733956 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733958 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733961 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733963 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733966 2577 feature_gate.go:328] unrecognized feature gate: Example May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733968 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733971 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733973 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733979 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733991 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733994 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.733997 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 25 13:05:46.734434 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734007 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734013 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734015 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734018 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734021 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734024 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734026 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734029 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734031 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734034 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734037 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734039 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734042 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734044 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734047 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734049 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734052 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734054 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734057 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 25 13:05:46.734925 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734059 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734062 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734064 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734067 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734069 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734072 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734074 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734077 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734081 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734084 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734087 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734090 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734094 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734096 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734099 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734101 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734104 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734106 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734109 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734112 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 25 13:05:46.735392 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734114 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734117 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734120 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734122 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734125 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734127 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734130 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734563 2577 feature_gate.go:328] unrecognized feature gate: NewOLM May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734568 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734571 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734574 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734576 2577 feature_gate.go:328] unrecognized feature gate: Example2 May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734579 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734581 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734584 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734587 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734589 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734592 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734595 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734597 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 25 13:05:46.735905 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734600 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734605 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734608 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734611 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734616 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734620 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734623 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734626 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734629 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734631 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734634 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734636 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734639 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734642 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734644 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734647 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734650 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734652 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734654 2577 feature_gate.go:328] unrecognized feature gate: DualReplica May 25 13:05:46.736402 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734657 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734659 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734662 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734664 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734667 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734669 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734671 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734674 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734676 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734679 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734681 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734684 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734686 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734689 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734692 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734695 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734697 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734703 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734705 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 25 13:05:46.736916 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734708 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734710 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734716 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734720 2577 feature_gate.go:328] unrecognized feature gate: Example May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734722 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734726 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734728 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734731 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734734 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734736 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734739 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734741 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734744 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734746 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734748 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734751 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734754 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734756 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734759 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734761 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 25 13:05:46.737386 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734764 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734767 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734769 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734772 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734774 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734777 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734779 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734783 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734785 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734788 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734790 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734793 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734797 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734799 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.734801 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736075 2577 flags.go:64] FLAG: --address="0.0.0.0" May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736087 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736093 2577 flags.go:64] FLAG: --anonymous-auth="true" May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736098 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736104 2577 flags.go:64] FLAG: --authentication-token-webhook="false" May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736107 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 25 13:05:46.737888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736111 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736117 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736121 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736124 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736128 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736131 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736134 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736137 2577 flags.go:64] FLAG: --cgroup-root="" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736141 2577 flags.go:64] FLAG: --cgroups-per-qos="true" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736144 2577 flags.go:64] FLAG: --client-ca-file="" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736147 2577 flags.go:64] FLAG: --cloud-config="" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736150 2577 flags.go:64] FLAG: --cloud-provider="external" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736153 2577 flags.go:64] FLAG: --cluster-dns="[]" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736158 2577 flags.go:64] FLAG: --cluster-domain="" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736161 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736164 2577 flags.go:64] FLAG: --config-dir="" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736166 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736170 2577 flags.go:64] FLAG: --container-log-max-files="5" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736174 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736177 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736180 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736184 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736187 2577 flags.go:64] FLAG: --contention-profiling="false" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736191 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" May 25 13:05:46.738418 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736194 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736198 2577 flags.go:64] FLAG: --cpu-manager-policy="none" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736201 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736205 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736208 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736211 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736214 2577 flags.go:64] FLAG: --enable-load-reader="false" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736217 2577 flags.go:64] FLAG: --enable-server="true" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736221 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736225 2577 flags.go:64] FLAG: --event-burst="100" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736228 2577 flags.go:64] FLAG: --event-qps="50" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736231 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736235 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736237 2577 flags.go:64] FLAG: --eviction-hard="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736242 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736245 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736248 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736251 2577 flags.go:64] FLAG: --eviction-soft="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736254 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736257 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736260 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736263 2577 flags.go:64] FLAG: --experimental-mounter-path="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736266 2577 flags.go:64] FLAG: --fail-cgroupv1="false" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736269 2577 flags.go:64] FLAG: --fail-swap-on="true" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736272 2577 flags.go:64] FLAG: --feature-gates="" May 25 13:05:46.739020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736276 2577 flags.go:64] FLAG: --file-check-frequency="20s" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736279 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736283 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736286 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736289 2577 flags.go:64] FLAG: --healthz-port="10248" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736292 2577 flags.go:64] FLAG: --help="false" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736295 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-251.ec2.internal" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736299 2577 flags.go:64] FLAG: --housekeeping-interval="10s" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736302 2577 flags.go:64] FLAG: --http-check-frequency="20s" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736306 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736309 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736312 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736315 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736318 2577 flags.go:64] FLAG: --image-service-endpoint="" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736321 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736324 2577 flags.go:64] FLAG: --kube-api-burst="100" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736327 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736331 2577 flags.go:64] FLAG: --kube-api-qps="50" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736334 2577 flags.go:64] FLAG: --kube-reserved="" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736337 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736340 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736343 2577 flags.go:64] FLAG: --kubelet-cgroups="" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736345 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736348 2577 flags.go:64] FLAG: --lock-file="" May 25 13:05:46.739682 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736351 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736354 2577 flags.go:64] FLAG: --log-flush-frequency="5s" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736357 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736363 2577 flags.go:64] FLAG: --log-json-split-stream="false" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736366 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736369 2577 flags.go:64] FLAG: --log-text-split-stream="false" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736372 2577 flags.go:64] FLAG: --logging-format="text" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736375 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736378 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736381 2577 flags.go:64] FLAG: --manifest-url="" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736385 2577 flags.go:64] FLAG: --manifest-url-header="" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736389 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736392 2577 flags.go:64] FLAG: --max-open-files="1000000" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736397 2577 flags.go:64] FLAG: --max-pods="110" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736400 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736403 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736406 2577 flags.go:64] FLAG: --memory-manager-policy="None" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736410 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736413 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736416 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736419 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736427 2577 flags.go:64] FLAG: --node-status-max-images="50" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736432 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736435 2577 flags.go:64] FLAG: --oom-score-adj="-999" May 25 13:05:46.740301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736438 2577 flags.go:64] FLAG: --pod-cidr="" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736441 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736447 2577 flags.go:64] FLAG: --pod-manifest-path="" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736450 2577 flags.go:64] FLAG: --pod-max-pids="-1" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736453 2577 flags.go:64] FLAG: --pods-per-core="0" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736456 2577 flags.go:64] FLAG: --port="10250" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736459 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736462 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fcebbec7af454466" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736466 2577 flags.go:64] FLAG: --qos-reserved="" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736469 2577 flags.go:64] FLAG: --read-only-port="10255" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736472 2577 flags.go:64] FLAG: --register-node="true" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736475 2577 flags.go:64] FLAG: --register-schedulable="true" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736477 2577 flags.go:64] FLAG: --register-with-taints="" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736481 2577 flags.go:64] FLAG: --registry-burst="10" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736484 2577 flags.go:64] FLAG: --registry-qps="5" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736486 2577 flags.go:64] FLAG: --reserved-cpus="" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736489 2577 flags.go:64] FLAG: --reserved-memory="" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736494 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736497 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736500 2577 flags.go:64] FLAG: --rotate-certificates="false" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736503 2577 flags.go:64] FLAG: --rotate-server-certificates="false" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736506 2577 flags.go:64] FLAG: --runonce="false" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736509 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736512 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736515 2577 flags.go:64] FLAG: --seccomp-default="false" May 25 13:05:46.740951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736518 2577 flags.go:64] FLAG: --serialize-image-pulls="true" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736521 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736524 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736527 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736530 2577 flags.go:64] FLAG: --storage-driver-password="root" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736533 2577 flags.go:64] FLAG: --storage-driver-secure="false" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736536 2577 flags.go:64] FLAG: --storage-driver-table="stats" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736539 2577 flags.go:64] FLAG: --storage-driver-user="root" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736542 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736545 2577 flags.go:64] FLAG: --sync-frequency="1m0s" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736548 2577 flags.go:64] FLAG: --system-cgroups="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736551 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736557 2577 flags.go:64] FLAG: --system-reserved-cgroup="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736560 2577 flags.go:64] FLAG: --tls-cert-file="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736563 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736572 2577 flags.go:64] FLAG: --tls-min-version="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736575 2577 flags.go:64] FLAG: --tls-private-key-file="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736578 2577 flags.go:64] FLAG: --topology-manager-policy="none" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736582 2577 flags.go:64] FLAG: --topology-manager-policy-options="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736585 2577 flags.go:64] FLAG: --topology-manager-scope="container" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736588 2577 flags.go:64] FLAG: --v="2" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736642 2577 flags.go:64] FLAG: --version="false" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736651 2577 flags.go:64] FLAG: --vmodule="" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736657 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.736663 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 25 13:05:46.741583 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736786 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736790 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736794 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736797 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736800 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736803 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736806 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736809 2577 feature_gate.go:328] unrecognized feature gate: Example2 May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736812 2577 feature_gate.go:328] unrecognized feature gate: Example May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736815 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736819 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736823 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736826 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736829 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736832 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736835 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736837 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736840 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736843 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 25 13:05:46.742205 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736846 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736848 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736851 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736853 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736856 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736859 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736862 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736865 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736868 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736883 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736887 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736890 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736893 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736896 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736898 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736901 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736904 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736906 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736909 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736912 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 25 13:05:46.742660 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736914 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736917 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736920 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736922 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736925 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736928 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736931 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736939 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736943 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736947 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736950 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736952 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736955 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736958 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736960 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736963 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736966 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736969 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736971 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736974 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 25 13:05:46.743416 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736977 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736980 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736983 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736987 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736989 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736992 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736995 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.736998 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737001 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737003 2577 feature_gate.go:328] unrecognized feature gate: NewOLM May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737006 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737008 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737011 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737014 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737016 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737019 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737021 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737024 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737027 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 25 13:05:46.744209 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737029 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737033 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737037 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737040 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737043 2577 feature_gate.go:328] unrecognized feature gate: DualReplica May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737046 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737049 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.737052 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 25 13:05:46.744823 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.737527 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 25 13:05:46.745642 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.745618 2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 25 13:05:46.745678 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.745644 2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 25 13:05:46.745706 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745699 2577 feature_gate.go:328] unrecognized feature gate: NewOLM May 25 13:05:46.745706 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745705 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745708 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745712 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745715 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745718 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745720 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745725 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745730 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745733 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745736 2577 feature_gate.go:328] unrecognized feature gate: Example May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745739 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745742 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745745 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745748 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745751 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745754 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745757 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745760 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 25 13:05:46.745754 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745763 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745767 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745769 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745772 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745775 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745778 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745781 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745784 2577 feature_gate.go:328] unrecognized feature gate: DualReplica May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745787 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745789 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745792 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745795 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745797 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745800 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745803 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745805 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745808 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745811 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745813 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 25 13:05:46.746234 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745815 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745818 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745820 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745823 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745825 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745828 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745831 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745833 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745836 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745838 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745841 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745844 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745846 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745849 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745851 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745854 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745857 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745860 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745863 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 25 13:05:46.746693 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745867 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745891 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745894 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745897 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745900 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745903 2577 feature_gate.go:328] unrecognized feature gate: Example2 May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745906 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745908 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745911 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745913 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745916 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745918 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745921 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745924 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745926 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745929 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745931 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745934 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745937 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745941 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 25 13:05:46.747268 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745943 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745946 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745948 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745951 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745953 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745955 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745958 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745961 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.745964 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.745970 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746070 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746075 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746078 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746081 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746084 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746087 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 25 13:05:46.747775 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746090 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746092 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746095 2577 feature_gate.go:328] unrecognized feature gate: Example2 May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746098 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746100 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746103 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746106 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746108 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746111 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746113 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746116 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746118 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746121 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746123 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746127 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746131 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746135 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746138 2577 feature_gate.go:328] unrecognized feature gate: Example May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746142 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 25 13:05:46.748182 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746145 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746148 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746150 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746153 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746155 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746158 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746162 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746164 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746167 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746170 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746172 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746175 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746178 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746180 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746183 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746186 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746188 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746191 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746194 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746196 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 25 13:05:46.748655 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746199 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746201 2577 feature_gate.go:328] unrecognized feature gate: NewOLM May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746204 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746206 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746209 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746211 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746214 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746216 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746219 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746222 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746225 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746227 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746230 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746232 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746235 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746237 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746240 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746243 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746247 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746250 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 25 13:05:46.749218 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746252 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746255 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746257 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746260 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746263 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746265 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746268 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746270 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746273 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746275 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746278 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746280 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746282 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746285 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746287 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746290 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746292 2577 feature_gate.go:328] unrecognized feature gate: DualReplica May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746295 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746298 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746300 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 25 13:05:46.749707 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:46.746303 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 25 13:05:46.750238 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.746309 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 25 13:05:46.750238 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.747767 2577 server.go:962] "Client rotation is on, will bootstrap in background" May 25 13:05:46.751713 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.751697 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 25 13:05:46.752520 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.752507 2577 server.go:1019] "Starting client certificate rotation" May 25 13:05:46.752640 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.752619 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 25 13:05:46.752700 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.752675 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 25 13:05:46.781118 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.781081 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 25 13:05:46.782320 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.782297 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 25 13:05:46.794063 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.794034 2577 log.go:25] "Validated CRI v1 runtime API" May 25 13:05:46.798720 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.798696 2577 log.go:25] "Validated CRI v1 image API" May 25 13:05:46.800629 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.800610 2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 25 13:05:46.805229 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.805194 2577 fs.go:135] Filesystem UUIDs: map[47e98bf7-d334-41c2-8bf4-0abe0615de62:/dev/nvme0n1p4 7694dffd-c3b5-4bdd-ad7f-58b153b0abca:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] May 25 13:05:46.805316 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.805225 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 25 13:05:46.810339 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.810212 2577 manager.go:217] Machine: {Timestamp:2026-05-25 13:05:46.809641279 +0000 UTC m=+0.387151654 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097896 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec224d6c9d250cc7f019e5c55951cd22 SystemUUID:ec224d6c-9d25-0cc7-f019-e5c55951cd22 BootID:70ec825b-5e43-4ee3-ba87-32e3d72c5ab8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:24:49:87:c7:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:24:49:87:c7:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:b3:1b:85:9b:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 25 13:05:46.810339 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.810329 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 25 13:05:46.810476 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.810427 2577 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 25 13:05:46.811649 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.811621 2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 25 13:05:46.811828 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.811650 2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-251.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 25 13:05:46.811898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.811837 2577 topology_manager.go:138] "Creating topology manager with none policy" May 25 13:05:46.811898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.811846 2577 container_manager_linux.go:306] "Creating device plugin manager" May 25 13:05:46.811898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.811854 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 25 13:05:46.812596 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.812584 2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 25 13:05:46.815464 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.815451 2577 state_mem.go:36] "Initialized new in-memory state store" May 25 13:05:46.815611 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.815602 2577 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 25 13:05:46.817684 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.817674 2577 kubelet.go:491] "Attempting to sync node with API server" May 25 13:05:46.817739 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.817688 2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 25 13:05:46.817739 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.817700 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 25 13:05:46.817739 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.817710 2577 kubelet.go:397] "Adding apiserver pod source" May 25 13:05:46.817739 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.817720 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 25 13:05:46.818178 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.818155 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 25 13:05:46.818935 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.818919 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 25 13:05:46.819013 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.818941 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 25 13:05:46.821471 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.821453 2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 25 13:05:46.824555 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.824538 2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 25 13:05:46.825267 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825256 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 25 13:05:46.825305 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825272 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 25 13:05:46.825305 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825279 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 25 13:05:46.825305 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825286 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 25 13:05:46.825305 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825295 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 25 13:05:46.825305 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825302 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 25 13:05:46.825441 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825308 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 25 13:05:46.825441 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825315 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 25 13:05:46.825441 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825321 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 25 13:05:46.825441 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825328 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 25 13:05:46.825441 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825348 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 25 13:05:46.825578 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.825567 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 25 13:05:46.826981 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.826969 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 25 13:05:46.826981 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.826983 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 25 13:05:46.830464 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.830450 2577 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 25 13:05:46.830547 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.830484 2577 server.go:1295] "Started kubelet" May 25 13:05:46.830626 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.830573 2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 25 13:05:46.830689 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.830648 2577 server_v1.go:47] "podresources" method="list" useActivePods=true May 25 13:05:46.831464 ip-10-0-142-251 systemd[1]: Started Kubernetes Kubelet. May 25 13:05:46.832085 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.831712 2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 25 13:05:46.832085 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.831673 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 25 13:05:46.832798 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.832778 2577 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-251.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 25 13:05:46.832907 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.832812 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-251.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 25 13:05:46.832907 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.832848 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 25 13:05:46.835562 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.835542 2577 server.go:317] "Adding debug handlers to kubelet server" May 25 13:05:46.837960 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.837940 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-22fhc" May 25 13:05:46.838592 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.838571 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 25 13:05:46.839194 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.839175 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 25 13:05:46.839389 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.839375 2577 volume_manager.go:295] "The desired_state_of_world populator starts" May 25 13:05:46.839389 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.839390 2577 volume_manager.go:297] "Starting Kubelet Volume Manager" May 25 13:05:46.839479 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.839471 2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 25 13:05:46.839516 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.839509 2577 reconstruct.go:97] "Volume reconstruction finished" May 25 13:05:46.839516 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.839516 2577 reconciler.go:26] "Reconciler: start to sync state" May 25 13:05:46.840050 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.839108 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-251.ec2.internal.18b2d0c4e47efc31 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-251.ec2.internal,UID:ip-10-0-142-251.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-251.ec2.internal,},FirstTimestamp:2026-05-25 13:05:46.830462001 +0000 UTC m=+0.407972376,LastTimestamp:2026-05-25 13:05:46.830462001 +0000 UTC m=+0.407972376,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-251.ec2.internal,}" May 25 13:05:46.840197 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840176 2577 factory.go:55] Registering systemd factory May 25 13:05:46.840197 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840197 2577 factory.go:223] Registration of the systemd container factory successfully May 25 13:05:46.840348 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.840299 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:46.840428 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840414 2577 factory.go:153] Registering CRI-O factory May 25 13:05:46.840474 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840432 2577 factory.go:223] Registration of the crio container factory successfully May 25 13:05:46.840515 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840488 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 25 13:05:46.840515 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840511 2577 factory.go:103] Registering Raw factory May 25 13:05:46.840602 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.840514 2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 25 13:05:46.840602 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.840531 2577 manager.go:1196] Started watching for new ooms in manager May 25 13:05:46.841026 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.841010 2577 manager.go:319] Starting recovery of all containers May 25 13:05:46.843036 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.843008 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 25 13:05:46.843127 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.843029 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-251.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 25 13:05:46.846258 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.846237 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-22fhc" May 25 13:05:46.850947 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.850805 2577 manager.go:324] Recovery completed May 25 13:05:46.855667 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.855654 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 25 13:05:46.860254 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.860232 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 25 13:05:46.860340 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.860274 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 25 13:05:46.860340 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.860289 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 25 13:05:46.860893 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.860852 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" May 25 13:05:46.860893 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.860886 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 25 13:05:46.860995 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.860904 2577 state_mem.go:36] "Initialized new in-memory state store" May 25 13:05:46.863806 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.863794 2577 policy_none.go:49] "None policy: Start" May 25 13:05:46.863854 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.863811 2577 memory_manager.go:186] "Starting memorymanager" policy="None" May 25 13:05:46.863854 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.863828 2577 state_mem.go:35] "Initializing new in-memory state store" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.904525 2577 manager.go:341] "Starting Device Plugin manager" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.904559 2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.904570 2577 server.go:85] "Starting device plugin registration server" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.904846 2577 eviction_manager.go:189] "Eviction manager: starting control loop" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.904857 2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.904991 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.905168 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 25 13:05:46.905261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.905178 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 25 13:05:46.905732 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.905713 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 25 13:05:46.905787 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.905758 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:46.926509 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.926465 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 25 13:05:46.927735 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.927708 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 25 13:05:46.927735 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.927728 2577 status_manager.go:230] "Starting to sync pod status with apiserver" May 25 13:05:46.927913 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.927748 2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 25 13:05:46.927913 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.927755 2577 kubelet.go:2451] "Starting kubelet main sync loop" May 25 13:05:46.927913 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:46.927797 2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 25 13:05:46.930154 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:46.930138 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 25 13:05:47.005585 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.005501 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 25 13:05:47.006747 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.006723 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 25 13:05:47.006866 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.006755 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 25 13:05:47.006866 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.006767 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 25 13:05:47.006866 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.006791 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-251.ec2.internal" May 25 13:05:47.018061 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.018031 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-251.ec2.internal" May 25 13:05:47.018061 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.018064 2577 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-251.ec2.internal\": node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.027903 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.027858 2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal"] May 25 13:05:47.028008 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.027944 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 25 13:05:47.030379 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.030363 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 25 13:05:47.030458 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.030392 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 25 13:05:47.030458 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.030401 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 25 13:05:47.031885 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.031863 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 25 13:05:47.032064 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.032051 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.032101 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.032083 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 25 13:05:47.033068 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.033048 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 25 13:05:47.033068 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.033060 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 25 13:05:47.033194 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.033075 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 25 13:05:47.033194 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.033082 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 25 13:05:47.033194 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.033084 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 25 13:05:47.033194 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.033093 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 25 13:05:47.034139 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.034126 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 25 13:05:47.034177 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.034151 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 25 13:05:47.034639 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.034622 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.034927 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.034913 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 25 13:05:47.034988 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.034939 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 25 13:05:47.034988 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.034953 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 25 13:05:47.040301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.040281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.040394 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.040324 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.040394 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.040343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8c054cb9a59ce3874e2199aacfd5039-config\") pod \"kube-apiserver-proxy-ip-10-0-142-251.ec2.internal\" (UID: \"e8c054cb9a59ce3874e2199aacfd5039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 25 13:05:47.053641 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.053612 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-251.ec2.internal\" not found" node="ip-10-0-142-251.ec2.internal" May 25 13:05:47.058043 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.058021 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-251.ec2.internal\" not found" node="ip-10-0-142-251.ec2.internal" May 25 13:05:47.135140 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.135107 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.141441 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.141414 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8c054cb9a59ce3874e2199aacfd5039-config\") pod \"kube-apiserver-proxy-ip-10-0-142-251.ec2.internal\" (UID: \"e8c054cb9a59ce3874e2199aacfd5039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 25 13:05:47.141501 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.141447 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.141501 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.141471 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.141600 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.141506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8c054cb9a59ce3874e2199aacfd5039-config\") pod \"kube-apiserver-proxy-ip-10-0-142-251.ec2.internal\" (UID: \"e8c054cb9a59ce3874e2199aacfd5039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 25 13:05:47.141600 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.141539 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.141600 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.141506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.235214 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.235173 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.336042 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.335972 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.356457 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.356432 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:47.360229 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.360209 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 25 13:05:47.436894 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.436841 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.537399 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.537360 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.637970 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.637895 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.738425 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.738392 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.753897 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.753856 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 25 13:05:47.754037 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.754022 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 25 13:05:47.838483 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.838445 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:47.839172 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.839159 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 25 13:05:47.848468 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.848434 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-24 13:00:46 +0000 UTC" deadline="2027-10-23 08:07:00.669061541 +0000 UTC" May 25 13:05:47.848468 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.848464 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12379h1m12.820601511s" May 25 13:05:47.850727 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.850703 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 25 13:05:47.871536 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.871501 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qp464" May 25 13:05:47.871926 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:47.871899 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c054cb9a59ce3874e2199aacfd5039.slice/crio-c175c48817b71c6ffb7280c58f95aa5b41d9cc9b0a9c5a31ef7593e2dc23d715 WatchSource:0}: Error finding container c175c48817b71c6ffb7280c58f95aa5b41d9cc9b0a9c5a31ef7593e2dc23d715: Status 404 returned error can't find the container with id c175c48817b71c6ffb7280c58f95aa5b41d9cc9b0a9c5a31ef7593e2dc23d715 May 25 13:05:47.872107 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:47.872085 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5541ecea1d6385d1bbfe3fb4b8556b5e.slice/crio-e011d467dd4531f8c2d9ce4ec5a93894ec667c3d8570a67e3769a9d3da21fc97 WatchSource:0}: Error finding container e011d467dd4531f8c2d9ce4ec5a93894ec667c3d8570a67e3769a9d3da21fc97: Status 404 returned error can't find the container with id e011d467dd4531f8c2d9ce4ec5a93894ec667c3d8570a67e3769a9d3da21fc97 May 25 13:05:47.874931 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.874914 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 25 13:05:47.879031 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.879011 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qp464" May 25 13:05:47.931026 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.930902 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" event={"ID":"5541ecea1d6385d1bbfe3fb4b8556b5e","Type":"ContainerStarted","Data":"e011d467dd4531f8c2d9ce4ec5a93894ec667c3d8570a67e3769a9d3da21fc97"} May 25 13:05:47.931848 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:47.931821 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" event={"ID":"e8c054cb9a59ce3874e2199aacfd5039","Type":"ContainerStarted","Data":"c175c48817b71c6ffb7280c58f95aa5b41d9cc9b0a9c5a31ef7593e2dc23d715"} May 25 13:05:47.939024 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:47.938996 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:48.014780 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.014743 2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 25 13:05:48.039611 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.039586 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:48.140049 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.140010 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:48.201527 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.201449 2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 25 13:05:48.240686 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.240643 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 25 13:05:48.266964 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.266896 2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 25 13:05:48.339900 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.339856 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 25 13:05:48.352387 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.352348 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 25 13:05:48.353144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.353122 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 25 13:05:48.360433 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.360409 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 25 13:05:48.818778 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.818738 2577 apiserver.go:52] "Watching apiserver" May 25 13:05:48.825382 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.825356 2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 25 13:05:48.826550 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.826526 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-ddjsl","kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal","openshift-multus/multus-additional-cni-plugins-plnkx","openshift-multus/network-metrics-daemon-44458","openshift-network-operator/iptables-alerter-lm9jv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57","openshift-cluster-node-tuning-operator/tuned-zfm5s","openshift-dns/node-resolver-sls6w","openshift-image-registry/node-ca-mrcbj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal","openshift-multus/multus-4pd99","openshift-network-diagnostics/network-check-target-hb8pp","openshift-ovn-kubernetes/ovnkube-node-4rg2g"] May 25 13:05:48.828471 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.828452 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.829495 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.829472 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.830585 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.830526 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:48.830715 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.830633 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:05:48.831053 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.831006 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 25 13:05:48.831053 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.831035 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 25 13:05:48.831219 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.831065 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zwzfv\"" May 25 13:05:48.831663 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.831640 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.831830 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.831808 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 25 13:05:48.831955 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.831930 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 25 13:05:48.832246 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.832231 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 25 13:05:48.832334 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.832263 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4r4qt\"" May 25 13:05:48.832334 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.832313 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 25 13:05:48.832439 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.832316 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 25 13:05:48.833062 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.832626 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.833632 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.833614 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.833807 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.833787 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 25 13:05:48.834042 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834025 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 25 13:05:48.834108 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834089 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wrvjj\"" May 25 13:05:48.834154 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834117 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 25 13:05:48.834572 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834553 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 25 13:05:48.834673 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834587 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 25 13:05:48.834762 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834694 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 25 13:05:48.834987 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.834966 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.835086 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.835011 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t6b65\"" May 25 13:05:48.835803 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.835783 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 25 13:05:48.836022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.836003 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hht9v\"" May 25 13:05:48.836139 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.836035 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 25 13:05:48.836294 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.836262 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.837300 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.837042 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 25 13:05:48.837300 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.837056 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9pmml\"" May 25 13:05:48.837300 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.837192 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 25 13:05:48.838071 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.838051 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4pd99" May 25 13:05:48.838435 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.838416 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 25 13:05:48.838966 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.838948 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 25 13:05:48.838966 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.838962 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qzxcp\"" May 25 13:05:48.839105 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.838953 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 25 13:05:48.839802 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.839781 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:48.840444 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.840414 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:05:48.841116 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.841095 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 25 13:05:48.841439 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.841423 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ggcdh\"" May 25 13:05:48.843013 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.842853 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.845147 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.845121 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 25 13:05:48.845246 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.845145 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 25 13:05:48.846273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.845479 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 25 13:05:48.846273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.845547 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 25 13:05:48.846273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.845578 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 25 13:05:48.846273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.845826 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 25 13:05:48.846273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.846038 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cn9g4\"" May 25 13:05:48.849564 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849539 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cnibin\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.849645 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-socket-dir-parent\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.849645 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-hostroot\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.849645 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849603 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-multus-certs\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.849783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-slash\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.849783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849690 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-run-netns\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.849783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-kubernetes\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.849783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849751 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-ovn\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.849969 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-cni-bin\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.849969 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.849969 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05a827a6-4e72-4283-8967-2c44c405e0c9-agent-certs\") pod \"konnectivity-agent-ddjsl\" (UID: \"05a827a6-4e72-4283-8967-2c44c405e0c9\") " pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.849969 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849867 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-cni-binary-copy\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.849969 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-k8s-cni-cncf-io\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.849969 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849953 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-cni-multus\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.849976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-etc-kubernetes\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gg4q\" (UniqueName: \"kubernetes.io/projected/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-kube-api-access-5gg4q\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rd2b\" (UniqueName: \"kubernetes.io/projected/bcaff31e-f7bd-4028-8e50-38894ee8c24f-kube-api-access-8rd2b\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-env-overrides\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850111 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19706064-0ad5-4245-93cf-960a91ff5e6d-host\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-host\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-lib-modules\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-cni-bin\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.850223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-node-log\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850259 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsz9s\" (UniqueName: \"kubernetes.io/projected/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-kube-api-access-nsz9s\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5kwb\" (UniqueName: \"kubernetes.io/projected/36393f8d-8a92-4e03-9340-4623cf2d2177-kube-api-access-d5kwb\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-ovnkube-config\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-hosts-file\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbvfx\" (UniqueName: \"kubernetes.io/projected/7088f44e-edb0-4b89-89dc-da82bc7e466a-kube-api-access-lbvfx\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850495 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-netns\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-systemd\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.850590 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cni-binary-copy\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850637 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36393f8d-8a92-4e03-9340-4623cf2d2177-iptables-alerter-script\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850661 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36393f8d-8a92-4e03-9340-4623cf2d2177-host-slash\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-kubelet\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-kubelet\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-var-lib-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850756 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-tmp-dir\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850781 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-run\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-cni-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-sys\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-os-release\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.850984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdg7f\" (UniqueName: \"kubernetes.io/projected/916f48b4-46b7-4562-b64c-f62903759bd3-kube-api-access-mdg7f\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-sys-fs\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl75l\" (UniqueName: \"kubernetes.io/projected/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-kube-api-access-wl75l\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cc33bae-9021-427e-a689-a8fffd1378bf-tmp\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-system-cni-dir\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-cnibin\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-systemd-units\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-os-release\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-cni-netd\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-device-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851333 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c54lr\" (UniqueName: \"kubernetes.io/projected/19706064-0ad5-4245-93cf-960a91ff5e6d-kube-api-access-c54lr\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-var-lib-kubelet\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-etc-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851396 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-log-socket\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851420 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-socket-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851445 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysconfig\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.851844 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs44h\" (UniqueName: \"kubernetes.io/projected/6cc33bae-9021-427e-a689-a8fffd1378bf-kube-api-access-bs44h\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-daemon-config\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-ovnkube-script-lib\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-registration-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05a827a6-4e72-4283-8967-2c44c405e0c9-konnectivity-ca\") pod \"konnectivity-agent-ddjsl\" (UID: \"05a827a6-4e72-4283-8967-2c44c405e0c9\") " pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851631 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysctl-d\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851655 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysctl-conf\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-systemd\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-system-cni-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-conf-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-modprobe-d\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-tuned\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/916f48b4-46b7-4562-b64c-f62903759bd3-ovn-node-metrics-cert\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.852584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.851899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19706064-0ad5-4245-93cf-960a91ff5e6d-serviceca\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.879582 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.879551 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 13:00:47 +0000 UTC" deadline="2028-02-01 00:53:05.184027402 +0000 UTC" May 25 13:05:48.879582 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.879581 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14795h47m16.304451529s" May 25 13:05:48.940724 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.940693 2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 25 13:05:48.952251 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952221 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-sys\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.952251 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952249 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-os-release\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952270 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mdg7f\" (UniqueName: \"kubernetes.io/projected/916f48b4-46b7-4562-b64c-f62903759bd3-kube-api-access-mdg7f\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952342 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-os-release\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952341 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-sys\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952395 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-sys-fs\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952421 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wl75l\" (UniqueName: \"kubernetes.io/projected/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-kube-api-access-wl75l\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952445 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cc33bae-9021-427e-a689-a8fffd1378bf-tmp\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.952522 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952494 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-sys-fs\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952527 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-system-cni-dir\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952563 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-cnibin\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952587 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-systemd-units\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952607 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952623 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-os-release\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952640 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-cni-netd\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952655 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-device-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952679 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c54lr\" (UniqueName: \"kubernetes.io/projected/19706064-0ad5-4245-93cf-960a91ff5e6d-kube-api-access-c54lr\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952702 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-var-lib-kubelet\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952727 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-etc-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952753 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-log-socket\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952736 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952784 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-socket-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952808 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysconfig\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952833 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bs44h\" (UniqueName: \"kubernetes.io/projected/6cc33bae-9021-427e-a689-a8fffd1378bf-kube-api-access-bs44h\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952891 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-daemon-config\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.952923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952918 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-ovnkube-script-lib\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952948 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-registration-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.952976 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05a827a6-4e72-4283-8967-2c44c405e0c9-konnectivity-ca\") pod \"konnectivity-agent-ddjsl\" (UID: \"05a827a6-4e72-4283-8967-2c44c405e0c9\") " pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953004 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysctl-d\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953017 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-var-lib-kubelet\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953027 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysctl-conf\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953051 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-systemd\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953065 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-system-cni-dir\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953073 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-system-cni-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953096 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-conf-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953102 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-cni-netd\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953101 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-os-release\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953119 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-modprobe-d\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953142 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-tuned\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953152 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-device-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953143 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-cnibin\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953179 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-systemd-units\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953194 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-etc-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.953687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953214 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953249 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/916f48b4-46b7-4562-b64c-f62903759bd3-ovn-node-metrics-cert\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953278 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19706064-0ad5-4245-93cf-960a91ff5e6d-serviceca\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953302 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cnibin\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953319 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-socket-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953327 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-socket-dir-parent\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953352 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-hostroot\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953358 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-log-socket\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953385 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-multus-certs\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953394 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-hostroot\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953406 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-slash\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953422 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-run-netns\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953431 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cnibin\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953462 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-kubernetes\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953477 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-socket-dir-parent\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953479 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-ovn\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953502 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-ovn\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953513 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-cni-bin\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.954508 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953544 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953543 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysconfig\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953572 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05a827a6-4e72-4283-8967-2c44c405e0c9-agent-certs\") pod \"konnectivity-agent-ddjsl\" (UID: \"05a827a6-4e72-4283-8967-2c44c405e0c9\") " pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953601 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-cni-binary-copy\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953629 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-k8s-cni-cncf-io\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953664 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-cni-multus\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953690 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-etc-kubernetes\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953714 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5gg4q\" (UniqueName: \"kubernetes.io/projected/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-kube-api-access-5gg4q\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953741 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rd2b\" (UniqueName: \"kubernetes.io/projected/bcaff31e-f7bd-4028-8e50-38894ee8c24f-kube-api-access-8rd2b\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953751 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19706064-0ad5-4245-93cf-960a91ff5e6d-serviceca\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953768 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-env-overrides\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953804 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19706064-0ad5-4245-93cf-960a91ff5e6d-host\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953817 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-cni-bin\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953829 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-host\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953855 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953985 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-lib-modules\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.954175 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-daemon-config\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.954747 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-k8s-cni-cncf-io\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.955360 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955218 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-cni-multus\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955268 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-etc-kubernetes\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955497 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-registration-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.953856 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-lib-modules\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955648 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-cni-bin\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955719 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-node-log\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955755 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955789 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nsz9s\" (UniqueName: \"kubernetes.io/projected/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-kube-api-access-nsz9s\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955856 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955903 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d5kwb\" (UniqueName: \"kubernetes.io/projected/36393f8d-8a92-4e03-9340-4623cf2d2177-kube-api-access-d5kwb\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955939 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.955972 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-ovnkube-config\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.956203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956103 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.956632 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956134 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-hosts-file\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.956632 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956242 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lbvfx\" (UniqueName: \"kubernetes.io/projected/7088f44e-edb0-4b89-89dc-da82bc7e466a-kube-api-access-lbvfx\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:48.956632 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956350 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-conf-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956719 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956650 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-netns\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956719 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956677 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-systemd\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.956719 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956696 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/916f48b4-46b7-4562-b64c-f62903759bd3-ovn-node-metrics-cert\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.956800 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956717 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cni-binary-copy\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.956800 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956753 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.956800 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956786 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36393f8d-8a92-4e03-9340-4623cf2d2177-iptables-alerter-script\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.956898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956817 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36393f8d-8a92-4e03-9340-4623cf2d2177-host-slash\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.956898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956844 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-kubelet\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956857 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-cni-bin\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.956898 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956893 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-kubelet\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.957022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956922 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-node-log\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.957022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956926 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-var-lib-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.957022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956920 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05a827a6-4e72-4283-8967-2c44c405e0c9-konnectivity-ca\") pod \"konnectivity-agent-ddjsl\" (UID: \"05a827a6-4e72-4283-8967-2c44c405e0c9\") " pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.957022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956973 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-tmp-dir\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.957135 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957062 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-var-lib-openvswitch\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.957135 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957072 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysctl-d\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.957135 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957117 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-run\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.957225 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957158 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.957225 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957189 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.957225 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957206 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-tmp-dir\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.957309 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957249 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-cni-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.957309 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957291 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:48.957309 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957299 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.957491 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957473 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-sysctl-conf\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.957564 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957550 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-systemd\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.957645 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957625 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-ovnkube-config\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.957685 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957629 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-system-cni-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.957760 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957746 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.957791 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.957769 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:48.957791 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957775 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-run\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.957843 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957833 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-hosts-file\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.957891 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.957843 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:05:49.457808611 +0000 UTC m=+3.035318973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:48.957958 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.957940 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcaff31e-f7bd-4028-8e50-38894ee8c24f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.958110 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958094 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-netns\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.958170 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-run-systemd\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.958323 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958306 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36393f8d-8a92-4e03-9340-4623cf2d2177-host-slash\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.958453 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958435 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.958536 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958522 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-multus-cni-dir\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.958587 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958568 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-cni-binary-copy\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.958739 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958722 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcaff31e-f7bd-4028-8e50-38894ee8c24f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.958786 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958768 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-var-lib-kubelet\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.958825 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.956818 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.958854 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958836 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-kubelet\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.958937 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958908 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19706064-0ad5-4245-93cf-960a91ff5e6d-host\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.959014 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.958942 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-host-run-multus-certs\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.959100 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959081 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-modprobe-d\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.959156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959109 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-slash\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.959156 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959125 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-kubernetes\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.959243 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959202 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36393f8d-8a92-4e03-9340-4623cf2d2177-iptables-alerter-script\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.959302 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959284 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cc33bae-9021-427e-a689-a8fffd1378bf-host\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.959357 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959321 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-env-overrides\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.959357 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cc33bae-9021-427e-a689-a8fffd1378bf-tmp\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.959496 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959376 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/916f48b4-46b7-4562-b64c-f62903759bd3-host-run-netns\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.959918 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959725 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-cni-binary-copy\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:48.959918 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.959783 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/916f48b4-46b7-4562-b64c-f62903759bd3-ovnkube-script-lib\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.961470 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.961321 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cc33bae-9021-427e-a689-a8fffd1378bf-etc-tuned\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.963511 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.963484 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05a827a6-4e72-4283-8967-2c44c405e0c9-agent-certs\") pod \"konnectivity-agent-ddjsl\" (UID: \"05a827a6-4e72-4283-8967-2c44c405e0c9\") " pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:48.967445 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.967416 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:05:48.967445 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.967440 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:05:48.967602 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.967453 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:48.967602 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.967465 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl75l\" (UniqueName: \"kubernetes.io/projected/dc5fd3fa-97ed-4118-b25b-fe2d0332652b-kube-api-access-wl75l\") pod \"node-resolver-sls6w\" (UID: \"dc5fd3fa-97ed-4118-b25b-fe2d0332652b\") " pod="openshift-dns/node-resolver-sls6w" May 25 13:05:48.967602 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:48.967506 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:05:49.467489345 +0000 UTC m=+3.044999711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:48.968402 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.968377 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs44h\" (UniqueName: \"kubernetes.io/projected/6cc33bae-9021-427e-a689-a8fffd1378bf-kube-api-access-bs44h\") pod \"tuned-zfm5s\" (UID: \"6cc33bae-9021-427e-a689-a8fffd1378bf\") " pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:48.968497 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.968479 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdg7f\" (UniqueName: \"kubernetes.io/projected/916f48b4-46b7-4562-b64c-f62903759bd3-kube-api-access-mdg7f\") pod \"ovnkube-node-4rg2g\" (UID: \"916f48b4-46b7-4562-b64c-f62903759bd3\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:48.969217 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.969060 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c54lr\" (UniqueName: \"kubernetes.io/projected/19706064-0ad5-4245-93cf-960a91ff5e6d-kube-api-access-c54lr\") pod \"node-ca-mrcbj\" (UID: \"19706064-0ad5-4245-93cf-960a91ff5e6d\") " pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:48.970372 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.970333 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbvfx\" (UniqueName: \"kubernetes.io/projected/7088f44e-edb0-4b89-89dc-da82bc7e466a-kube-api-access-lbvfx\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:48.971012 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.970988 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsz9s\" (UniqueName: \"kubernetes.io/projected/eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3-kube-api-access-nsz9s\") pod \"aws-ebs-csi-driver-node-s2x57\" (UID: \"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:48.971419 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.971394 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rd2b\" (UniqueName: \"kubernetes.io/projected/bcaff31e-f7bd-4028-8e50-38894ee8c24f-kube-api-access-8rd2b\") pod \"multus-additional-cni-plugins-plnkx\" (UID: \"bcaff31e-f7bd-4028-8e50-38894ee8c24f\") " pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:48.972010 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.971985 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5kwb\" (UniqueName: \"kubernetes.io/projected/36393f8d-8a92-4e03-9340-4623cf2d2177-kube-api-access-d5kwb\") pod \"iptables-alerter-lm9jv\" (UID: \"36393f8d-8a92-4e03-9340-4623cf2d2177\") " pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:48.972099 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:48.972085 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gg4q\" (UniqueName: \"kubernetes.io/projected/3dcb6605-7c8a-4f0e-81d0-92c9db492cb4-kube-api-access-5gg4q\") pod \"multus-4pd99\" (UID: \"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4\") " pod="openshift-multus/multus-4pd99" May 25 13:05:49.045246 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.045217 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 25 13:05:49.141548 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.141468 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" May 25 13:05:49.149281 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.149252 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-plnkx" May 25 13:05:49.157927 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.157901 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lm9jv" May 25 13:05:49.164563 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.164543 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" May 25 13:05:49.173180 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.173142 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ddjsl" May 25 13:05:49.177797 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.177776 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sls6w" May 25 13:05:49.184380 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.184356 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mrcbj" May 25 13:05:49.189941 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.189919 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4pd99" May 25 13:05:49.195615 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.195593 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:05:49.460994 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.460902 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:49.461134 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:49.461045 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:49.461134 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:49.461111 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:05:50.461091485 +0000 UTC m=+4.038601859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:49.476750 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.476720 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916f48b4_46b7_4562_b64c_f62903759bd3.slice/crio-3fe925723ba4ee63ac980edeaa4566a3829ea0d2b9a6ff31eb8d5fcfd57dce46 WatchSource:0}: Error finding container 3fe925723ba4ee63ac980edeaa4566a3829ea0d2b9a6ff31eb8d5fcfd57dce46: Status 404 returned error can't find the container with id 3fe925723ba4ee63ac980edeaa4566a3829ea0d2b9a6ff31eb8d5fcfd57dce46 May 25 13:05:49.480341 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.480309 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36393f8d_8a92_4e03_9340_4623cf2d2177.slice/crio-5cfea26f54c5426803d175a9f95a78d4fb9e53071da9be6d88ff1034d4728677 WatchSource:0}: Error finding container 5cfea26f54c5426803d175a9f95a78d4fb9e53071da9be6d88ff1034d4728677: Status 404 returned error can't find the container with id 5cfea26f54c5426803d175a9f95a78d4fb9e53071da9be6d88ff1034d4728677 May 25 13:05:49.481011 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.480955 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc33bae_9021_427e_a689_a8fffd1378bf.slice/crio-de76e20130b0a712826bd7d7cd7a520f434383eaec583bbfb23bb052b152daf2 WatchSource:0}: Error finding container de76e20130b0a712826bd7d7cd7a520f434383eaec583bbfb23bb052b152daf2: Status 404 returned error can't find the container with id de76e20130b0a712826bd7d7cd7a520f434383eaec583bbfb23bb052b152daf2 May 25 13:05:49.484020 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.483953 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9f20b9_0f8f_4b40_b47c_8bc9dee2b6d3.slice/crio-58a1b341c79b4191a7aaf54fca88ec4a32ed18b1583f075c1bbb010492c9e774 WatchSource:0}: Error finding container 58a1b341c79b4191a7aaf54fca88ec4a32ed18b1583f075c1bbb010492c9e774: Status 404 returned error can't find the container with id 58a1b341c79b4191a7aaf54fca88ec4a32ed18b1583f075c1bbb010492c9e774 May 25 13:05:49.484542 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.484503 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcaff31e_f7bd_4028_8e50_38894ee8c24f.slice/crio-a14dd963e2d7fdc7df4d8d319e2e9481b0bbbb27342c9fe056f7d16cdf6af6e3 WatchSource:0}: Error finding container a14dd963e2d7fdc7df4d8d319e2e9481b0bbbb27342c9fe056f7d16cdf6af6e3: Status 404 returned error can't find the container with id a14dd963e2d7fdc7df4d8d319e2e9481b0bbbb27342c9fe056f7d16cdf6af6e3 May 25 13:05:49.485453 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.485405 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5fd3fa_97ed_4118_b25b_fe2d0332652b.slice/crio-190638fdfc41f71456f251216cd8597faea365d601e442474065f010df0e61b2 WatchSource:0}: Error finding container 190638fdfc41f71456f251216cd8597faea365d601e442474065f010df0e61b2: Status 404 returned error can't find the container with id 190638fdfc41f71456f251216cd8597faea365d601e442474065f010df0e61b2 May 25 13:05:49.487352 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.487065 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a827a6_4e72_4283_8967_2c44c405e0c9.slice/crio-a3d715c8c124210a35363a2ee7186899b59772551d0a1b779a00bdc1d3b73c11 WatchSource:0}: Error finding container a3d715c8c124210a35363a2ee7186899b59772551d0a1b779a00bdc1d3b73c11: Status 404 returned error can't find the container with id a3d715c8c124210a35363a2ee7186899b59772551d0a1b779a00bdc1d3b73c11 May 25 13:05:49.487943 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:05:49.487917 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcb6605_7c8a_4f0e_81d0_92c9db492cb4.slice/crio-dcf72facf5555a30119dee0e7479571f1695a9685d5a017e3aa56d02a3d7484c WatchSource:0}: Error finding container dcf72facf5555a30119dee0e7479571f1695a9685d5a017e3aa56d02a3d7484c: Status 404 returned error can't find the container with id dcf72facf5555a30119dee0e7479571f1695a9685d5a017e3aa56d02a3d7484c May 25 13:05:49.561930 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.561709 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:49.562065 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:49.561906 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:05:49.562065 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:49.562016 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:05:49.562065 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:49.562029 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:49.562173 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:49.562081 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:05:50.56206647 +0000 UTC m=+4.139576832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:49.880290 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.880246 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 13:00:47 +0000 UTC" deadline="2027-12-22 02:28:39.157831485 +0000 UTC" May 25 13:05:49.880290 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.880287 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13813h22m49.277548426s" May 25 13:05:49.943853 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.943783 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lm9jv" event={"ID":"36393f8d-8a92-4e03-9340-4623cf2d2177","Type":"ContainerStarted","Data":"5cfea26f54c5426803d175a9f95a78d4fb9e53071da9be6d88ff1034d4728677"} May 25 13:05:49.948635 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.948596 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"3fe925723ba4ee63ac980edeaa4566a3829ea0d2b9a6ff31eb8d5fcfd57dce46"} May 25 13:05:49.954748 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.954702 2577 generic.go:358] "Generic (PLEG): container finished" podID="5541ecea1d6385d1bbfe3fb4b8556b5e" containerID="90b2ad5282a75e539535348c30d49431f0d9091d6f0cb8aa0d6faaa07bdeb6f1" exitCode=0 May 25 13:05:49.954930 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.954787 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" event={"ID":"5541ecea1d6385d1bbfe3fb4b8556b5e","Type":"ContainerDied","Data":"90b2ad5282a75e539535348c30d49431f0d9091d6f0cb8aa0d6faaa07bdeb6f1"} May 25 13:05:49.963190 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.963106 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sls6w" event={"ID":"dc5fd3fa-97ed-4118-b25b-fe2d0332652b","Type":"ContainerStarted","Data":"190638fdfc41f71456f251216cd8597faea365d601e442474065f010df0e61b2"} May 25 13:05:49.975942 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.964408 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" event={"ID":"6cc33bae-9021-427e-a689-a8fffd1378bf","Type":"ContainerStarted","Data":"de76e20130b0a712826bd7d7cd7a520f434383eaec583bbfb23bb052b152daf2"} May 25 13:05:49.979931 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.979897 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" event={"ID":"e8c054cb9a59ce3874e2199aacfd5039","Type":"ContainerStarted","Data":"a242a884e24e7815cffaca6fdf1138c0da711409b9c62df3be0c4028133231ae"} May 25 13:05:49.999319 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.999152 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" podStartSLOduration=1.9991320190000001 podStartE2EDuration="1.999132019s" podCreationTimestamp="2026-05-25 13:05:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:05:49.998025235 +0000 UTC m=+3.575535623" watchObservedRunningTime="2026-05-25 13:05:49.999132019 +0000 UTC m=+3.576642408" May 25 13:05:49.999485 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:49.999442 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mrcbj" event={"ID":"19706064-0ad5-4245-93cf-960a91ff5e6d","Type":"ContainerStarted","Data":"c2dc6dd9893c68edbe027840aed71a0aba28abf8a1372b0b0edc9b353ae362f7"} May 25 13:05:50.006536 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.006473 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerStarted","Data":"a14dd963e2d7fdc7df4d8d319e2e9481b0bbbb27342c9fe056f7d16cdf6af6e3"} May 25 13:05:50.009328 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.009293 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4pd99" event={"ID":"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4","Type":"ContainerStarted","Data":"dcf72facf5555a30119dee0e7479571f1695a9685d5a017e3aa56d02a3d7484c"} May 25 13:05:50.016420 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.016334 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ddjsl" event={"ID":"05a827a6-4e72-4283-8967-2c44c405e0c9","Type":"ContainerStarted","Data":"a3d715c8c124210a35363a2ee7186899b59772551d0a1b779a00bdc1d3b73c11"} May 25 13:05:50.020630 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.020332 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" event={"ID":"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3","Type":"ContainerStarted","Data":"58a1b341c79b4191a7aaf54fca88ec4a32ed18b1583f075c1bbb010492c9e774"} May 25 13:05:50.468616 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.468577 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:50.468801 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.468734 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:50.469330 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.468861 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:05:52.468841608 +0000 UTC m=+6.046351988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:50.572243 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.572139 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:50.572409 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.572317 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:05:50.572409 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.572335 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:05:50.572409 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.572348 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:50.572409 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.572405 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:05:52.572387868 +0000 UTC m=+6.149898235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:50.930713 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.930634 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:50.931170 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.930781 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:05:50.931285 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:50.931267 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:50.931392 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:50.931373 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:05:51.032015 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:51.031981 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" event={"ID":"5541ecea1d6385d1bbfe3fb4b8556b5e","Type":"ContainerStarted","Data":"66574a9ff4403e021dcb382031b72d7eb65e9a5942ef3ce52a96f00d31720645"} May 25 13:05:51.045377 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:51.045327 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" podStartSLOduration=3.045309527 podStartE2EDuration="3.045309527s" podCreationTimestamp="2026-05-25 13:05:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:05:51.044677377 +0000 UTC m=+4.622187763" watchObservedRunningTime="2026-05-25 13:05:51.045309527 +0000 UTC m=+4.622819912" May 25 13:05:52.489711 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:52.489169 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:52.489711 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.489309 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:52.489711 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.489374 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:05:56.489354134 +0000 UTC m=+10.066864515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:52.590227 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:52.590178 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:52.590410 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.590344 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:05:52.590410 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.590362 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:05:52.590410 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.590377 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:52.590575 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.590435 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:05:56.590416858 +0000 UTC m=+10.167927227 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:52.928233 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:52.928192 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:52.928399 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.928340 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:05:52.928701 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:52.928192 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:52.928810 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:52.928785 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:05:54.928927 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:54.928891 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:54.929445 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:54.929048 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:05:54.929513 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:54.929445 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:54.929568 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:54.929534 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:05:56.524888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:56.524833 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:56.525362 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.525015 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:56.525362 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.525078 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:04.525059662 +0000 UTC m=+18.102570031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:05:56.625771 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:56.625188 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:56.625771 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.625347 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:05:56.625771 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.625363 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:05:56.625771 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.625377 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:56.625771 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.625428 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:06:04.625412231 +0000 UTC m=+18.202922604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:05:56.929329 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:56.929246 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:56.929488 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.929357 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:05:56.929753 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:56.929730 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:56.929858 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:56.929834 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:05:58.928221 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:58.928180 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:05:58.928689 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:58.928297 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:05:58.928689 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:05:58.928359 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:05:58.928689 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:05:58.928474 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:00.928944 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:00.928900 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:00.929424 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:00.928966 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:00.929424 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:00.929034 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:00.929424 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:00.929153 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:02.928506 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:02.928477 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:02.928931 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:02.928613 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:02.928931 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:02.928674 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:02.928931 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:02.928780 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:04.588472 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:04.588431 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:04.588953 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.588553 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:06:04.588953 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.588620 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:20.588603544 +0000 UTC m=+34.166113926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:06:04.689531 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:04.689494 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:04.689701 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.689675 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:06:04.689701 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.689700 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:06:04.689787 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.689714 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:06:04.689787 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.689780 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:06:20.689762094 +0000 UTC m=+34.267272469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:06:04.928581 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:04.928486 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:04.928740 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.928617 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:04.928740 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:04.928635 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:04.928740 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:04.928722 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:06.928957 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:06.928918 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:06.928957 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:06.928947 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:06.929472 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:06.929048 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:06.929472 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:06.929184 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:08.071764 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.071422 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mrcbj" event={"ID":"19706064-0ad5-4245-93cf-960a91ff5e6d","Type":"ContainerStarted","Data":"d107f2d796dbc3ad9b59a0a3ff32fc1270e4502aab0b9a2982a7850046a3e936"} May 25 13:06:08.073204 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.073182 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerStarted","Data":"1e2896d386960cb503d7ad1c25db23322eb3c081d03c098acac2217b29049041"} May 25 13:06:08.075009 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.074574 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4pd99" event={"ID":"3dcb6605-7c8a-4f0e-81d0-92c9db492cb4","Type":"ContainerStarted","Data":"755b317826f17981a0882113d9562f21b5380564c0b78b48dcd51dba70c2c996"} May 25 13:06:08.076799 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.076738 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ddjsl" event={"ID":"05a827a6-4e72-4283-8967-2c44c405e0c9","Type":"ContainerStarted","Data":"9c4a21e31b2022d581521aa0764cf53e7915ca9a0068b34a6c5f9e3885dbfbe7"} May 25 13:06:08.078841 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.078815 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" event={"ID":"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3","Type":"ContainerStarted","Data":"27dfc92fe5c4e54b7756425d886459a5081a88a1fd14f6078e24c38decc6c840"} May 25 13:06:08.080307 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.080281 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"ac394d85525b336e741a28d4d34fdf0b078696da6eaa3efa657e6440c08bfad4"} May 25 13:06:08.082085 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.081915 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sls6w" event={"ID":"dc5fd3fa-97ed-4118-b25b-fe2d0332652b","Type":"ContainerStarted","Data":"ce2d3d9d8a16d27120d6ca772d8f995dc9a03084a6af70f7827fe9872dc9b273"} May 25 13:06:08.083314 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.083289 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" event={"ID":"6cc33bae-9021-427e-a689-a8fffd1378bf","Type":"ContainerStarted","Data":"3e7f56ab0715d937a470cdd08d48aacecfe00f1eb46912881b6d3d7eac6849ae"} May 25 13:06:08.089992 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.089927 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mrcbj" podStartSLOduration=10.600789232 podStartE2EDuration="21.089909766s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.490985971 +0000 UTC m=+3.068496334" lastFinishedPulling="2026-05-25 13:05:59.98010649 +0000 UTC m=+13.557616868" observedRunningTime="2026-05-25 13:06:08.088799795 +0000 UTC m=+21.666310180" watchObservedRunningTime="2026-05-25 13:06:08.089909766 +0000 UTC m=+21.667420152" May 25 13:06:08.103518 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.103472 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-sls6w" podStartSLOduration=2.7718019529999998 podStartE2EDuration="21.103458882s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.487360884 +0000 UTC m=+3.064871245" lastFinishedPulling="2026-05-25 13:06:07.819017801 +0000 UTC m=+21.396528174" observedRunningTime="2026-05-25 13:06:08.103434614 +0000 UTC m=+21.680944999" watchObservedRunningTime="2026-05-25 13:06:08.103458882 +0000 UTC m=+21.680969265" May 25 13:06:08.125519 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.123485 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4pd99" podStartSLOduration=2.736797276 podStartE2EDuration="21.123467696s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.489844452 +0000 UTC m=+3.067354816" lastFinishedPulling="2026-05-25 13:06:07.876514867 +0000 UTC m=+21.454025236" observedRunningTime="2026-05-25 13:06:08.122977153 +0000 UTC m=+21.700487538" watchObservedRunningTime="2026-05-25 13:06:08.123467696 +0000 UTC m=+21.700978081" May 25 13:06:08.145520 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.145468 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ddjsl" podStartSLOduration=2.817112486 podStartE2EDuration="21.145451257s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.488859395 +0000 UTC m=+3.066369770" lastFinishedPulling="2026-05-25 13:06:07.817198165 +0000 UTC m=+21.394708541" observedRunningTime="2026-05-25 13:06:08.144990061 +0000 UTC m=+21.722500445" watchObservedRunningTime="2026-05-25 13:06:08.145451257 +0000 UTC m=+21.722961643" May 25 13:06:08.217905 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.217841 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zfm5s" podStartSLOduration=2.880355951 podStartE2EDuration="21.217826136s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.48352351 +0000 UTC m=+3.061033873" lastFinishedPulling="2026-05-25 13:06:07.820993695 +0000 UTC m=+21.398504058" observedRunningTime="2026-05-25 13:06:08.192451953 +0000 UTC m=+21.769962338" watchObservedRunningTime="2026-05-25 13:06:08.217826136 +0000 UTC m=+21.795336498" May 25 13:06:08.920120 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.920098 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 25 13:06:08.922136 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.922055 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-25T13:06:08.920116078Z","UUID":"812e1a2a-dcdd-4d17-b44e-9960332642e6","Handler":null,"Name":"","Endpoint":""} May 25 13:06:08.924574 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.924553 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 25 13:06:08.924669 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.924580 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 25 13:06:08.928260 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.928238 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:08.928381 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:08.928357 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:08.928446 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:08.928416 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:08.928521 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:08.928505 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:09.086730 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.086642 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" event={"ID":"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3","Type":"ContainerStarted","Data":"4588e8f0555b0752693787bd7baf03d234d6236006c615d37b5b5032309dc8c8"} May 25 13:06:09.089142 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.089115 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"0f9f63719236e829ee659c973b6124084c220f4af8e742e82ef99c87d6ee6524"} May 25 13:06:09.089250 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.089151 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"5ee75486aab9136dc2ca933ba7053f122a6664da8042fc4e6230aab9f63a75cc"} May 25 13:06:09.089250 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.089161 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"31da812b93c40a180064155eb13c84fbdde292a2be260cab84250a3670e7c875"} May 25 13:06:09.089250 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.089169 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"8f75fa2dd91399f8eee0791b6b467d0e4ac786e30816f96f59a1295eb3191b52"} May 25 13:06:09.089250 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.089176 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"15adb34c82a9d2452e77c178ee205b6f8dcc6d01a644d0fac5e325ca15b7aec9"} May 25 13:06:09.090352 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.090329 2577 generic.go:358] "Generic (PLEG): container finished" podID="bcaff31e-f7bd-4028-8e50-38894ee8c24f" containerID="1e2896d386960cb503d7ad1c25db23322eb3c081d03c098acac2217b29049041" exitCode=0 May 25 13:06:09.090437 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:09.090406 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerDied","Data":"1e2896d386960cb503d7ad1c25db23322eb3c081d03c098acac2217b29049041"} May 25 13:06:10.094022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:10.093983 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lm9jv" event={"ID":"36393f8d-8a92-4e03-9340-4623cf2d2177","Type":"ContainerStarted","Data":"939a38e5ba52338bfd26d4c7f65d6cf4e1f0a8d67ea069f5fb703eeae2bd1e7d"} May 25 13:06:10.109830 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:10.109781 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lm9jv" podStartSLOduration=4.776574458 podStartE2EDuration="23.109763061s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.483575753 +0000 UTC m=+3.061086127" lastFinishedPulling="2026-05-25 13:06:07.816764352 +0000 UTC m=+21.394274730" observedRunningTime="2026-05-25 13:06:10.109470574 +0000 UTC m=+23.686980958" watchObservedRunningTime="2026-05-25 13:06:10.109763061 +0000 UTC m=+23.687273444" May 25 13:06:10.561273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:10.561219 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ddjsl" May 25 13:06:10.562060 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:10.562016 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ddjsl" May 25 13:06:10.928365 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:10.928142 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:10.928545 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:10.928211 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:10.928545 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:10.928442 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:10.928545 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:10.928525 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:11.099237 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:11.099148 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"240eea0b200e78881a92fd4d6203b9e77d50aedf951b4f0ce1a64206fa8a7501"} May 25 13:06:11.101077 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:11.101048 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" event={"ID":"eb9f20b9-0f8f-4b40-b47c-8bc9dee2b6d3","Type":"ContainerStarted","Data":"276221a08db75dcb13ea9d22ab85436c2aa1d7c7fa6af4943090afa25256e69d"} May 25 13:06:11.101533 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:11.101513 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ddjsl" May 25 13:06:11.101771 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:11.101753 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ddjsl" May 25 13:06:11.118816 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:11.118766 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2x57" podStartSLOduration=2.901115606 podStartE2EDuration="24.118754057s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.485827476 +0000 UTC m=+3.063337838" lastFinishedPulling="2026-05-25 13:06:10.70346591 +0000 UTC m=+24.280976289" observedRunningTime="2026-05-25 13:06:11.118185596 +0000 UTC m=+24.695695979" watchObservedRunningTime="2026-05-25 13:06:11.118754057 +0000 UTC m=+24.696264442" May 25 13:06:12.928951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:12.928850 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:12.928951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:12.928850 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:12.929606 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:12.929025 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:12.929606 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:12.929123 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:14.107896 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.107686 2577 generic.go:358] "Generic (PLEG): container finished" podID="bcaff31e-f7bd-4028-8e50-38894ee8c24f" containerID="907ffdfc069bf21df81bd9bbceef3a3470904aa62708aab44a4dd3324ad4ebaa" exitCode=0 May 25 13:06:14.108630 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.107777 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerDied","Data":"907ffdfc069bf21df81bd9bbceef3a3470904aa62708aab44a4dd3324ad4ebaa"} May 25 13:06:14.111285 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.111253 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" event={"ID":"916f48b4-46b7-4562-b64c-f62903759bd3","Type":"ContainerStarted","Data":"4923e7e4cd9a7f090967c7edb3f18d9dda81300b3797733e15b1e820680f3d8d"} May 25 13:06:14.111607 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.111533 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:06:14.111607 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.111559 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:06:14.111607 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.111569 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:06:14.127210 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.127185 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:06:14.127347 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.127269 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:06:14.153977 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.153926 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" podStartSLOduration=8.741738596 podStartE2EDuration="27.153910748s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.479127587 +0000 UTC m=+3.056637964" lastFinishedPulling="2026-05-25 13:06:07.891299741 +0000 UTC m=+21.468810116" observedRunningTime="2026-05-25 13:06:14.153781503 +0000 UTC m=+27.731291886" watchObservedRunningTime="2026-05-25 13:06:14.153910748 +0000 UTC m=+27.731421132" May 25 13:06:14.930700 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.930678 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:14.930819 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:14.930676 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:14.930819 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:14.930774 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:14.930924 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:14.930829 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:15.103010 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:15.102936 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hb8pp"] May 25 13:06:15.103758 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:15.103734 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44458"] May 25 13:06:15.115461 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:15.115437 2577 generic.go:358] "Generic (PLEG): container finished" podID="bcaff31e-f7bd-4028-8e50-38894ee8c24f" containerID="9ee95774e16a84f30a0a53126c2aff1ab02190bde18d90d02a103bcb9ff414e7" exitCode=0 May 25 13:06:15.115835 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:15.115511 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:15.115835 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:15.115540 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerDied","Data":"9ee95774e16a84f30a0a53126c2aff1ab02190bde18d90d02a103bcb9ff414e7"} May 25 13:06:15.115835 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:15.115668 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:15.115835 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:15.115785 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:15.116285 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:15.116261 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:16.119497 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:16.119299 2577 generic.go:358] "Generic (PLEG): container finished" podID="bcaff31e-f7bd-4028-8e50-38894ee8c24f" containerID="5691d2d984d46d9734d9beaf3f8db7dc54b0d90e147af479d6c28e1e582e4446" exitCode=0 May 25 13:06:16.120026 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:16.119379 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerDied","Data":"5691d2d984d46d9734d9beaf3f8db7dc54b0d90e147af479d6c28e1e582e4446"} May 25 13:06:16.929961 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:16.929928 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:16.930114 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:16.929932 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:16.930114 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:16.930063 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:16.930210 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:16.930135 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:18.928271 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:18.928242 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:18.928816 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:18.928369 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8pp" podUID="79ece060-ccbd-4c01-b8a1-64e7856a091e" May 25 13:06:18.928816 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:18.928667 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:18.928816 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:18.928795 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:06:20.603102 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.603060 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:20.603687 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:20.603215 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:06:20.603687 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:20.603286 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:52.603266129 +0000 UTC m=+66.180776491 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : object "openshift-multus"/"metrics-daemon-secret" not registered May 25 13:06:20.704280 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.704197 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:20.704439 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:20.704370 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 25 13:06:20.704439 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:20.704388 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 25 13:06:20.704439 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:20.704397 2577 projected.go:194] Error preparing data for projected volume kube-api-access-n824z for pod openshift-network-diagnostics/network-check-target-hb8pp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:06:20.704576 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:20.704448 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z podName:79ece060-ccbd-4c01-b8a1-64e7856a091e nodeName:}" failed. No retries permitted until 2026-05-25 13:06:52.704434489 +0000 UTC m=+66.281944851 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n824z" (UniqueName: "kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z") pod "network-check-target-hb8pp" (UID: "79ece060-ccbd-4c01-b8a1-64e7856a091e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 25 13:06:20.717118 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.717084 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeReady" May 25 13:06:20.717284 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.717243 2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 25 13:06:20.812503 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.812468 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pcmgs"] May 25 13:06:20.843375 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.843345 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-855w6"] May 25 13:06:20.843563 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.843528 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pcmgs" May 25 13:06:20.845990 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.845963 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dnmsw\"" May 25 13:06:20.846290 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.846263 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 25 13:06:20.848585 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.848537 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 25 13:06:20.859219 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.859198 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-855w6"] May 25 13:06:20.859322 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.859227 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pcmgs"] May 25 13:06:20.859322 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.859311 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:20.873457 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.873437 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 25 13:06:20.874404 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.874382 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 25 13:06:20.879240 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.879225 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 25 13:06:20.879407 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.879382 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6hqsn\"" May 25 13:06:20.928073 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.928048 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:20.928190 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.928083 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:20.933468 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.933450 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 25 13:06:20.933941 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.933915 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 25 13:06:20.934108 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.934091 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxkxl\"" May 25 13:06:20.934202 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.934163 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-stbll\"" May 25 13:06:20.934440 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:20.934422 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 25 13:06:21.006094 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.006068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j765c\" (UniqueName: \"kubernetes.io/projected/6cc87c4f-5424-448c-bc63-5e3b922eff11-kube-api-access-j765c\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:21.006243 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.006170 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-config-volume\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.006243 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.006206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:21.006335 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.006250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xv64\" (UniqueName: \"kubernetes.io/projected/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-kube-api-access-6xv64\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.006335 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.006281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.006414 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.006336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-tmp-dir\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.107256 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107215 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-config-volume\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.107256 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107257 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:21.107488 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107300 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6xv64\" (UniqueName: \"kubernetes.io/projected/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-kube-api-access-6xv64\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.107488 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107331 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.107488 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107368 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-tmp-dir\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.107488 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107391 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j765c\" (UniqueName: \"kubernetes.io/projected/6cc87c4f-5424-448c-bc63-5e3b922eff11-kube-api-access-j765c\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:21.107488 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.107442 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:21.107671 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.107523 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:21.607503462 +0000 UTC m=+35.185013827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:21.107733 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.107694 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:21.107784 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.107739 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:06:21.607726081 +0000 UTC m=+35.185236445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:21.107884 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.107854 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-config-volume\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.108055 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.108026 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-tmp-dir\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.127280 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.127244 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xv64\" (UniqueName: \"kubernetes.io/projected/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-kube-api-access-6xv64\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.127469 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.127445 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j765c\" (UniqueName: \"kubernetes.io/projected/6cc87c4f-5424-448c-bc63-5e3b922eff11-kube-api-access-j765c\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:21.612474 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.612436 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:21.613075 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:21.612541 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:21.613075 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.612610 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:21.613075 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.612646 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:21.613075 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.612684 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:06:22.612663873 +0000 UTC m=+36.190174252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:21.613075 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:21.612702 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:22.612694924 +0000 UTC m=+36.190205285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:22.621885 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:22.621846 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:22.622286 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:22.621949 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:22.622286 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:22.621991 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:22.622286 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:22.622025 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:22.622286 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:22.622063 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:06:24.622047336 +0000 UTC m=+38.199557698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:22.622286 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:22.622077 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:24.622070922 +0000 UTC m=+38.199581283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:23.136849 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:23.136808 2577 generic.go:358] "Generic (PLEG): container finished" podID="bcaff31e-f7bd-4028-8e50-38894ee8c24f" containerID="020c7938212f75b6a9f8efb0ecac002dd17e744ba28d09c0d83f7fd83a1c5e4c" exitCode=0 May 25 13:06:23.137050 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:23.136866 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerDied","Data":"020c7938212f75b6a9f8efb0ecac002dd17e744ba28d09c0d83f7fd83a1c5e4c"} May 25 13:06:24.141273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:24.141094 2577 generic.go:358] "Generic (PLEG): container finished" podID="bcaff31e-f7bd-4028-8e50-38894ee8c24f" containerID="50a1819a04394793ccf27409621973e59f31cd205de30c1b834a93ab0826551a" exitCode=0 May 25 13:06:24.141670 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:24.141175 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerDied","Data":"50a1819a04394793ccf27409621973e59f31cd205de30c1b834a93ab0826551a"} May 25 13:06:24.636053 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:24.636011 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:24.636230 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:24.636078 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:24.636230 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:24.636158 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:24.636230 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:24.636216 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:24.636230 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:24.636227 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:28.636209067 +0000 UTC m=+42.213719433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:24.636392 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:24.636268 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:06:28.636252197 +0000 UTC m=+42.213762560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:25.145888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.145839 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-plnkx" event={"ID":"bcaff31e-f7bd-4028-8e50-38894ee8c24f","Type":"ContainerStarted","Data":"cfc542d1fdd4a4536013909ea32a8b41609d1fed014bd7c9a5b0f59f8bb6756a"} May 25 13:06:25.168852 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.168791 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-plnkx" podStartSLOduration=5.508852008 podStartE2EDuration="38.168769577s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:05:49.486752235 +0000 UTC m=+3.064262598" lastFinishedPulling="2026-05-25 13:06:22.146669782 +0000 UTC m=+35.724180167" observedRunningTime="2026-05-25 13:06:25.167345593 +0000 UTC m=+38.744855978" watchObservedRunningTime="2026-05-25 13:06:25.168769577 +0000 UTC m=+38.746279961" May 25 13:06:25.895197 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.895166 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g"] May 25 13:06:25.906374 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.906354 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:25.907216 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.907192 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g"] May 25 13:06:25.909273 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.909255 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 25 13:06:25.910236 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.910217 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 25 13:06:25.910320 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.910228 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" May 25 13:06:25.910320 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.910246 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 25 13:06:25.935929 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.935899 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd"] May 25 13:06:25.948603 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.948584 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:25.950126 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.950105 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd"] May 25 13:06:25.950976 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.950955 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 25 13:06:25.950976 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.950968 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 25 13:06:25.951169 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.950962 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 25 13:06:25.951225 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:25.951207 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 25 13:06:26.044677 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.044838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.044838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7fsp\" (UniqueName: \"kubernetes.io/projected/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-kube-api-access-v7fsp\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.044838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-ca\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.044838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.044838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044819 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8e6a479c-bd89-43e9-8b33-24abe7f055ac-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.044838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044839 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jr9r\" (UniqueName: \"kubernetes.io/projected/8e6a479c-bd89-43e9-8b33-24abe7f055ac-kube-api-access-2jr9r\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.045076 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.044902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-hub\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.045076 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.045015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e6a479c-bd89-43e9-8b33-24abe7f055ac-tmp\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.146068 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.145988 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-hub\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.146068 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146050 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e6a479c-bd89-43e9-8b33-24abe7f055ac-tmp\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146080 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146105 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146340 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7fsp\" (UniqueName: \"kubernetes.io/projected/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-kube-api-access-v7fsp\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146389 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-ca\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146421 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146446 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8e6a479c-bd89-43e9-8b33-24abe7f055ac-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.146525 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146471 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e6a479c-bd89-43e9-8b33-24abe7f055ac-tmp\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.147566 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.146495 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2jr9r\" (UniqueName: \"kubernetes.io/projected/8e6a479c-bd89-43e9-8b33-24abe7f055ac-kube-api-access-2jr9r\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.147566 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.147182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.149671 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.149646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-ca\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.149825 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.149801 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8e6a479c-bd89-43e9-8b33-24abe7f055ac-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.149920 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.149903 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-hub\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.149977 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.149919 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.149977 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.149965 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.155979 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.155948 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jr9r\" (UniqueName: \"kubernetes.io/projected/8e6a479c-bd89-43e9-8b33-24abe7f055ac-kube-api-access-2jr9r\") pod \"klusterlet-addon-workmgr-7dbc67cb89-ss87g\" (UID: \"8e6a479c-bd89-43e9-8b33-24abe7f055ac\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.156257 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.156240 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7fsp\" (UniqueName: \"kubernetes.io/projected/e3ee6a10-41b8-43c4-bc56-22f846ab0ac6-kube-api-access-v7fsp\") pod \"cluster-proxy-proxy-agent-847d7869c9-gvnjd\" (UID: \"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.215965 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.215948 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:26.265904 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.265857 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:06:26.401715 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.401649 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g"] May 25 13:06:26.405022 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:26.405000 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd"] May 25 13:06:26.405266 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:06:26.405230 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6a479c_bd89_43e9_8b33_24abe7f055ac.slice/crio-5a714db83ace5fe269923c61fa48105e7d3acc877d618b8e39d64ef13194dc45 WatchSource:0}: Error finding container 5a714db83ace5fe269923c61fa48105e7d3acc877d618b8e39d64ef13194dc45: Status 404 returned error can't find the container with id 5a714db83ace5fe269923c61fa48105e7d3acc877d618b8e39d64ef13194dc45 May 25 13:06:26.409361 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:06:26.409328 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3ee6a10_41b8_43c4_bc56_22f846ab0ac6.slice/crio-14b12ce36974112676efb819d12c75e23d4d1ae390081c77f83db3e418a38cc5 WatchSource:0}: Error finding container 14b12ce36974112676efb819d12c75e23d4d1ae390081c77f83db3e418a38cc5: Status 404 returned error can't find the container with id 14b12ce36974112676efb819d12c75e23d4d1ae390081c77f83db3e418a38cc5 May 25 13:06:27.151985 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:27.151852 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" event={"ID":"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6","Type":"ContainerStarted","Data":"14b12ce36974112676efb819d12c75e23d4d1ae390081c77f83db3e418a38cc5"} May 25 13:06:27.153640 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:27.153550 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" event={"ID":"8e6a479c-bd89-43e9-8b33-24abe7f055ac","Type":"ContainerStarted","Data":"5a714db83ace5fe269923c61fa48105e7d3acc877d618b8e39d64ef13194dc45"} May 25 13:06:28.669940 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:28.669445 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:28.669940 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:28.669513 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:28.669940 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:28.669639 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:28.669940 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:28.669701 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:06:36.669680752 +0000 UTC m=+50.247191126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:28.669940 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:28.669806 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:28.669940 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:28.669901 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:36.669864994 +0000 UTC m=+50.247375362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:29.294743 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.294703 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-ldn2b"] May 25 13:06:29.307892 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.307842 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ldn2b"] May 25 13:06:29.308067 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.308041 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.310569 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.310543 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 25 13:06:29.475405 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.475365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f84c632-f831-43da-ab7b-e72fc9dd56f9-kubelet-config\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.475405 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.475402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f84c632-f831-43da-ab7b-e72fc9dd56f9-dbus\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.475649 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.475447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f84c632-f831-43da-ab7b-e72fc9dd56f9-original-pull-secret\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.577117 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.577034 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f84c632-f831-43da-ab7b-e72fc9dd56f9-kubelet-config\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.577117 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.577084 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f84c632-f831-43da-ab7b-e72fc9dd56f9-dbus\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.577314 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.577135 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f84c632-f831-43da-ab7b-e72fc9dd56f9-original-pull-secret\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.577314 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.577173 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f84c632-f831-43da-ab7b-e72fc9dd56f9-kubelet-config\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.577314 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.577301 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f84c632-f831-43da-ab7b-e72fc9dd56f9-dbus\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.579563 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.579537 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f84c632-f831-43da-ab7b-e72fc9dd56f9-original-pull-secret\") pod \"global-pull-secret-syncer-ldn2b\" (UID: \"8f84c632-f831-43da-ab7b-e72fc9dd56f9\") " pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:29.619674 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:29.619634 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ldn2b" May 25 13:06:32.412346 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:32.412310 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ldn2b"] May 25 13:06:32.414930 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:06:32.414869 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f84c632_f831_43da_ab7b_e72fc9dd56f9.slice/crio-d7051e3820df10edefb48ffe4458a199bab142dead5a439a6761e26932fcd67e WatchSource:0}: Error finding container d7051e3820df10edefb48ffe4458a199bab142dead5a439a6761e26932fcd67e: Status 404 returned error can't find the container with id d7051e3820df10edefb48ffe4458a199bab142dead5a439a6761e26932fcd67e May 25 13:06:33.168151 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:33.168111 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" event={"ID":"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6","Type":"ContainerStarted","Data":"aaad8d000ebad80369a3afdfb0fb990c6c818bb18af355e3c124253a614af600"} May 25 13:06:33.169608 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:33.169541 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" event={"ID":"8e6a479c-bd89-43e9-8b33-24abe7f055ac","Type":"ContainerStarted","Data":"5e060c5a7b9870e0fa575545124b92e22ee8a0952ca943853e44948646b6a5a5"} May 25 13:06:33.169787 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:33.169769 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:33.170931 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:33.170904 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ldn2b" event={"ID":"8f84c632-f831-43da-ab7b-e72fc9dd56f9","Type":"ContainerStarted","Data":"d7051e3820df10edefb48ffe4458a199bab142dead5a439a6761e26932fcd67e"} May 25 13:06:33.171685 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:33.171666 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:06:33.185958 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:33.185894 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" podStartSLOduration=2.310744226 podStartE2EDuration="8.185863923s" podCreationTimestamp="2026-05-25 13:06:25 +0000 UTC" firstStartedPulling="2026-05-25 13:06:26.407004735 +0000 UTC m=+39.984515105" lastFinishedPulling="2026-05-25 13:06:32.282124425 +0000 UTC m=+45.859634802" observedRunningTime="2026-05-25 13:06:33.185422262 +0000 UTC m=+46.762932647" watchObservedRunningTime="2026-05-25 13:06:33.185863923 +0000 UTC m=+46.763374377" May 25 13:06:36.735086 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:36.735053 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:36.735463 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:36.735100 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:36.735463 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:36.735209 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:36.735463 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:36.735220 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:36.735463 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:36.735261 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:06:52.735246379 +0000 UTC m=+66.312756761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:36.735463 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:36.735285 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:52.735267241 +0000 UTC m=+66.312777603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:37.182046 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:37.181954 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" event={"ID":"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6","Type":"ContainerStarted","Data":"20a0d839ce18298c687371d39e2b8f1981eb3ed28bf745891fc9d39030d54dfa"} May 25 13:06:37.182046 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:37.181997 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" event={"ID":"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6","Type":"ContainerStarted","Data":"36dd627f3b9a0a9e52e861bda0ca4cabe1539d84574cdee597bd26a21af40487"} May 25 13:06:37.183336 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:37.183309 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ldn2b" event={"ID":"8f84c632-f831-43da-ab7b-e72fc9dd56f9","Type":"ContainerStarted","Data":"d8cee4c690f41bfad7b8adaac65073c655cad9ecb28bb2a205dad44a5ef329b9"} May 25 13:06:37.201359 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:37.201302 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" podStartSLOduration=2.057332384 podStartE2EDuration="12.201286124s" podCreationTimestamp="2026-05-25 13:06:25 +0000 UTC" firstStartedPulling="2026-05-25 13:06:26.410844019 +0000 UTC m=+39.988354381" lastFinishedPulling="2026-05-25 13:06:36.554797749 +0000 UTC m=+50.132308121" observedRunningTime="2026-05-25 13:06:37.20011912 +0000 UTC m=+50.777629543" watchObservedRunningTime="2026-05-25 13:06:37.201286124 +0000 UTC m=+50.778796507" May 25 13:06:37.214343 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:37.214293 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ldn2b" podStartSLOduration=3.725489705 podStartE2EDuration="8.214279588s" podCreationTimestamp="2026-05-25 13:06:29 +0000 UTC" firstStartedPulling="2026-05-25 13:06:32.416530832 +0000 UTC m=+45.994041194" lastFinishedPulling="2026-05-25 13:06:36.905320706 +0000 UTC m=+50.482831077" observedRunningTime="2026-05-25 13:06:37.213603784 +0000 UTC m=+50.791114396" watchObservedRunningTime="2026-05-25 13:06:37.214279588 +0000 UTC m=+50.791789971" May 25 13:06:46.133711 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:46.133683 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rg2g" May 25 13:06:52.647430 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.647377 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:06:52.649798 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.649781 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 25 13:06:52.657915 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:52.657888 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 25 13:06:52.658009 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:52.657986 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:07:56.657960844 +0000 UTC m=+130.235471220 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : secret "metrics-daemon-secret" not found May 25 13:06:52.748430 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.748394 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:06:52.748583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.748459 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:52.748583 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.748477 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:06:52.748583 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:52.748540 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:06:52.748583 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:52.748555 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:06:52.748766 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:52.748607 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:07:24.748591658 +0000 UTC m=+98.326102021 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:06:52.748766 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:06:52.748620 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:07:24.748614725 +0000 UTC m=+98.326125087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:06:52.751059 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.751041 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 25 13:06:52.761259 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.761213 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 25 13:06:52.770951 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:52.770931 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n824z\" (UniqueName: \"kubernetes.io/projected/79ece060-ccbd-4c01-b8a1-64e7856a091e-kube-api-access-n824z\") pod \"network-check-target-hb8pp\" (UID: \"79ece060-ccbd-4c01-b8a1-64e7856a091e\") " pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:53.047597 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:53.047571 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-stbll\"" May 25 13:06:53.055512 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:53.055488 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:53.167335 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:53.167303 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hb8pp"] May 25 13:06:53.170955 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:06:53.170924 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ece060_ccbd_4c01_b8a1_64e7856a091e.slice/crio-e7f33c07007ee843b15453eb46d0913394fe4d904987ee19cef4a69aa94e06e9 WatchSource:0}: Error finding container e7f33c07007ee843b15453eb46d0913394fe4d904987ee19cef4a69aa94e06e9: Status 404 returned error can't find the container with id e7f33c07007ee843b15453eb46d0913394fe4d904987ee19cef4a69aa94e06e9 May 25 13:06:53.217011 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:53.216979 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hb8pp" event={"ID":"79ece060-ccbd-4c01-b8a1-64e7856a091e","Type":"ContainerStarted","Data":"e7f33c07007ee843b15453eb46d0913394fe4d904987ee19cef4a69aa94e06e9"} May 25 13:06:56.226341 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:56.226313 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hb8pp" event={"ID":"79ece060-ccbd-4c01-b8a1-64e7856a091e","Type":"ContainerStarted","Data":"34121ded31523fdd1bf386b801630f7dadbffef9d5edadac97039a8bc1e9ec6b"} May 25 13:06:56.226681 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:56.226447 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:06:56.244720 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:06:56.244675 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hb8pp" podStartSLOduration=66.403956316 podStartE2EDuration="1m9.244663521s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:06:53.172524254 +0000 UTC m=+66.750034618" lastFinishedPulling="2026-05-25 13:06:56.013231457 +0000 UTC m=+69.590741823" observedRunningTime="2026-05-25 13:06:56.243313934 +0000 UTC m=+69.820824318" watchObservedRunningTime="2026-05-25 13:06:56.244663521 +0000 UTC m=+69.822173904" May 25 13:07:24.773301 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:07:24.773264 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:07:24.773699 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:07:24.773309 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:07:24.773699 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:07:24.773404 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 25 13:07:24.773699 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:07:24.773406 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 25 13:07:24.773699 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:07:24.773473 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert podName:6cc87c4f-5424-448c-bc63-5e3b922eff11 nodeName:}" failed. No retries permitted until 2026-05-25 13:08:28.773457828 +0000 UTC m=+162.350968190 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert") pod "ingress-canary-855w6" (UID: "6cc87c4f-5424-448c-bc63-5e3b922eff11") : secret "canary-serving-cert" not found May 25 13:07:24.773699 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:07:24.773486 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls podName:52d3a7f9-dc96-4bf2-8039-95e9dc2a835c nodeName:}" failed. No retries permitted until 2026-05-25 13:08:28.773479992 +0000 UTC m=+162.350990354 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls") pod "dns-default-pcmgs" (UID: "52d3a7f9-dc96-4bf2-8039-95e9dc2a835c") : secret "dns-default-metrics-tls" not found May 25 13:07:27.232992 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:07:27.232955 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hb8pp" May 25 13:07:56.699127 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:07:56.699076 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:07:56.699619 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:07:56.699227 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 25 13:07:56.699619 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:07:56.699302 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs podName:7088f44e-edb0-4b89-89dc-da82bc7e466a nodeName:}" failed. No retries permitted until 2026-05-25 13:09:58.699286547 +0000 UTC m=+252.276796909 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs") pod "network-metrics-daemon-44458" (UID: "7088f44e-edb0-4b89-89dc-da82bc7e466a") : secret "metrics-daemon-secret" not found May 25 13:08:09.770187 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:09.770156 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sls6w_dc5fd3fa-97ed-4118-b25b-fe2d0332652b/dns-node-resolver/0.log" May 25 13:08:10.772700 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:10.772673 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mrcbj_19706064-0ad5-4245-93cf-960a91ff5e6d/node-ca/0.log" May 25 13:08:23.855458 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:08:23.855393 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pcmgs" podUID="52d3a7f9-dc96-4bf2-8039-95e9dc2a835c" May 25 13:08:23.868612 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:08:23.868568 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-855w6" podUID="6cc87c4f-5424-448c-bc63-5e3b922eff11" May 25 13:08:23.939148 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:08:23.939110 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-44458" podUID="7088f44e-edb0-4b89-89dc-da82bc7e466a" May 25 13:08:24.436398 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:24.436366 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pcmgs" May 25 13:08:28.835859 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:28.835817 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:08:28.836281 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:28.835921 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:08:28.838173 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:28.838149 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d3a7f9-dc96-4bf2-8039-95e9dc2a835c-metrics-tls\") pod \"dns-default-pcmgs\" (UID: \"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c\") " pod="openshift-dns/dns-default-pcmgs" May 25 13:08:28.838280 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:28.838222 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cc87c4f-5424-448c-bc63-5e3b922eff11-cert\") pod \"ingress-canary-855w6\" (UID: \"6cc87c4f-5424-448c-bc63-5e3b922eff11\") " pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:08:28.938886 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:28.938846 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dnmsw\"" May 25 13:08:28.947796 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:28.947775 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pcmgs" May 25 13:08:29.065808 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:29.065779 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pcmgs"] May 25 13:08:29.069161 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:08:29.069136 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d3a7f9_dc96_4bf2_8039_95e9dc2a835c.slice/crio-9ce4a2dd3b11758eb72480c3d5ca011b3eed3dc62786d34d2b0fbe0abd444dda WatchSource:0}: Error finding container 9ce4a2dd3b11758eb72480c3d5ca011b3eed3dc62786d34d2b0fbe0abd444dda: Status 404 returned error can't find the container with id 9ce4a2dd3b11758eb72480c3d5ca011b3eed3dc62786d34d2b0fbe0abd444dda May 25 13:08:29.449315 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:29.449276 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pcmgs" event={"ID":"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c","Type":"ContainerStarted","Data":"9ce4a2dd3b11758eb72480c3d5ca011b3eed3dc62786d34d2b0fbe0abd444dda"} May 25 13:08:30.453923 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.453861 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pcmgs" event={"ID":"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c","Type":"ContainerStarted","Data":"dc1f7055dc2cc01c7c53d53d7d19137dbe1144ee61bdbee7b5736b7631933628"} May 25 13:08:30.613399 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.613295 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lcblp"] May 25 13:08:30.616234 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.616216 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.619042 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.619020 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 25 13:08:30.619724 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.619701 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 25 13:08:30.620015 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.619994 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 25 13:08:30.620272 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.620046 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 25 13:08:30.620738 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.620724 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-84t4d\"" May 25 13:08:30.644635 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.644606 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lcblp"] May 25 13:08:30.649941 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.649913 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8mws\" (UniqueName: \"kubernetes.io/projected/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-kube-api-access-t8mws\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.650066 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.649947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-crio-socket\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.650066 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.650017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-data-volume\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.650066 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.650040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.650168 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.650076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.750618 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750580 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8mws\" (UniqueName: \"kubernetes.io/projected/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-kube-api-access-t8mws\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.750618 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750617 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-crio-socket\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.750912 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750647 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-data-volume\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.750912 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750717 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-crio-socket\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.750912 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750764 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.750912 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750786 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.751082 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.750943 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-data-volume\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.751339 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.751315 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.753128 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.753103 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.763223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.763178 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8mws\" (UniqueName: \"kubernetes.io/projected/cfe9ec39-cc14-4afd-b998-2a81061f0fe3-kube-api-access-t8mws\") pod \"insights-runtime-extractor-lcblp\" (UID: \"cfe9ec39-cc14-4afd-b998-2a81061f0fe3\") " pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:30.924528 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:30.924437 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lcblp" May 25 13:08:31.047988 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:31.047952 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lcblp"] May 25 13:08:31.051073 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:08:31.051043 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe9ec39_cc14_4afd_b998_2a81061f0fe3.slice/crio-e4c343aa88d932f072a1c6509d37419b7636c02975c3ec1a0407eb1322e5245f WatchSource:0}: Error finding container e4c343aa88d932f072a1c6509d37419b7636c02975c3ec1a0407eb1322e5245f: Status 404 returned error can't find the container with id e4c343aa88d932f072a1c6509d37419b7636c02975c3ec1a0407eb1322e5245f May 25 13:08:31.457948 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:31.457909 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcblp" event={"ID":"cfe9ec39-cc14-4afd-b998-2a81061f0fe3","Type":"ContainerStarted","Data":"831cebd39f6d77eb94f6649d74b70b7aadf89697cb1213724b5aae7d8b1ab8ce"} May 25 13:08:31.457948 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:31.457954 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcblp" event={"ID":"cfe9ec39-cc14-4afd-b998-2a81061f0fe3","Type":"ContainerStarted","Data":"e4c343aa88d932f072a1c6509d37419b7636c02975c3ec1a0407eb1322e5245f"} May 25 13:08:31.459521 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:31.459494 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pcmgs" event={"ID":"52d3a7f9-dc96-4bf2-8039-95e9dc2a835c","Type":"ContainerStarted","Data":"cd764fdc9e57709af02b69ed91db32ddfb55a61084330287a3d6a7859139cefe"} May 25 13:08:31.459670 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:31.459656 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pcmgs" May 25 13:08:31.478113 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:31.478067 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-pcmgs" podStartSLOduration=130.206361295 podStartE2EDuration="2m11.478053824s" podCreationTimestamp="2026-05-25 13:06:20 +0000 UTC" firstStartedPulling="2026-05-25 13:08:29.071271827 +0000 UTC m=+162.648782192" lastFinishedPulling="2026-05-25 13:08:30.342964355 +0000 UTC m=+163.920474721" observedRunningTime="2026-05-25 13:08:31.477095573 +0000 UTC m=+165.054605957" watchObservedRunningTime="2026-05-25 13:08:31.478053824 +0000 UTC m=+165.055564205" May 25 13:08:32.465009 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:32.464942 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcblp" event={"ID":"cfe9ec39-cc14-4afd-b998-2a81061f0fe3","Type":"ContainerStarted","Data":"7f4a6dad258bcb1f17d4d48e401900a198d9e205d419c32ecfe447ac78e5520a"} May 25 13:08:33.170888 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:33.170826 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" podUID="8e6a479c-bd89-43e9-8b33-24abe7f055ac" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.7:8000/readyz\": dial tcp 10.133.0.7:8000: connect: connection refused" May 25 13:08:33.468736 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:33.468694 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lcblp" event={"ID":"cfe9ec39-cc14-4afd-b998-2a81061f0fe3","Type":"ContainerStarted","Data":"bdc03187b69ab49c784543a1827bf84d41a596a592d68bef794faa4e1319e227"} May 25 13:08:33.469974 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:33.469947 2577 generic.go:358] "Generic (PLEG): container finished" podID="8e6a479c-bd89-43e9-8b33-24abe7f055ac" containerID="5e060c5a7b9870e0fa575545124b92e22ee8a0952ca943853e44948646b6a5a5" exitCode=1 May 25 13:08:33.470094 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:33.470022 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" event={"ID":"8e6a479c-bd89-43e9-8b33-24abe7f055ac","Type":"ContainerDied","Data":"5e060c5a7b9870e0fa575545124b92e22ee8a0952ca943853e44948646b6a5a5"} May 25 13:08:33.470351 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:33.470338 2577 scope.go:117] "RemoveContainer" containerID="5e060c5a7b9870e0fa575545124b92e22ee8a0952ca943853e44948646b6a5a5" May 25 13:08:33.497846 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:33.497793 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lcblp" podStartSLOduration=1.252291024 podStartE2EDuration="3.497777321s" podCreationTimestamp="2026-05-25 13:08:30 +0000 UTC" firstStartedPulling="2026-05-25 13:08:31.120737896 +0000 UTC m=+164.698248265" lastFinishedPulling="2026-05-25 13:08:33.366224201 +0000 UTC m=+166.943734562" observedRunningTime="2026-05-25 13:08:33.497467887 +0000 UTC m=+167.074978271" watchObservedRunningTime="2026-05-25 13:08:33.497777321 +0000 UTC m=+167.075287742" May 25 13:08:34.474519 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:34.474483 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" event={"ID":"8e6a479c-bd89-43e9-8b33-24abe7f055ac","Type":"ContainerStarted","Data":"667474cf873b27f0d26b89ccfc828e3d5d36fb1aa7c6525212b9867880bc9478"} May 25 13:08:34.475025 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:34.474837 2577 status_manager.go:355] "Container readiness changed for unknown container" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" containerID="cri-o://5e060c5a7b9870e0fa575545124b92e22ee8a0952ca943853e44948646b6a5a5" May 25 13:08:34.475025 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:34.474863 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:08:35.478159 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:35.478129 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:08:35.478806 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:35.478785 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc67cb89-ss87g" May 25 13:08:38.928406 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:38.928324 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:08:38.928761 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:38.928324 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:08:38.930834 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:38.930802 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6hqsn\"" May 25 13:08:38.938933 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:38.938901 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-855w6" May 25 13:08:39.057200 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:39.057162 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-855w6"] May 25 13:08:39.060957 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:08:39.060931 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc87c4f_5424_448c_bc63_5e3b922eff11.slice/crio-312aaa1b23ac573140973ac1207417de4df83edda412f2fbcc3c15514248be75 WatchSource:0}: Error finding container 312aaa1b23ac573140973ac1207417de4df83edda412f2fbcc3c15514248be75: Status 404 returned error can't find the container with id 312aaa1b23ac573140973ac1207417de4df83edda412f2fbcc3c15514248be75 May 25 13:08:39.488253 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:39.488215 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-855w6" event={"ID":"6cc87c4f-5424-448c-bc63-5e3b922eff11","Type":"ContainerStarted","Data":"312aaa1b23ac573140973ac1207417de4df83edda412f2fbcc3c15514248be75"} May 25 13:08:41.466899 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:41.466851 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pcmgs" May 25 13:08:41.494948 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:41.494914 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-855w6" event={"ID":"6cc87c4f-5424-448c-bc63-5e3b922eff11","Type":"ContainerStarted","Data":"be5d09bf2af4831b7e55ebcac40bd487d595248706cb789eebe16f105c10609e"} May 25 13:08:41.512437 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:41.512385 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-855w6" podStartSLOduration=139.824280501 podStartE2EDuration="2m21.512369112s" podCreationTimestamp="2026-05-25 13:06:20 +0000 UTC" firstStartedPulling="2026-05-25 13:08:39.063191192 +0000 UTC m=+172.640701554" lastFinishedPulling="2026-05-25 13:08:40.7512798 +0000 UTC m=+174.328790165" observedRunningTime="2026-05-25 13:08:41.511783041 +0000 UTC m=+175.089293426" watchObservedRunningTime="2026-05-25 13:08:41.512369112 +0000 UTC m=+175.089879520" May 25 13:08:45.229085 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.229043 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8"] May 25 13:08:45.232279 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.232257 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.234732 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.234707 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" May 25 13:08:45.234829 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.234708 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zmjzp\"" May 25 13:08:45.235475 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.235453 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 25 13:08:45.235585 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.235494 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 25 13:08:45.235657 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.235605 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" May 25 13:08:45.235718 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.235685 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 25 13:08:45.241179 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.241154 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8"] May 25 13:08:45.253747 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.253724 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xgxfb"] May 25 13:08:45.256673 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.256654 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.258835 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.258813 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.258962 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.258854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvnjt\" (UniqueName: \"kubernetes.io/projected/1f3849a0-f196-4593-9ee6-1a36bf58646d-kube-api-access-nvnjt\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.258962 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.258913 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.259063 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.258998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f3849a0-f196-4593-9ee6-1a36bf58646d-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.259063 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.259008 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 25 13:08:45.259348 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.259332 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mwjdt\"" May 25 13:08:45.259427 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.259339 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 25 13:08:45.259475 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.259462 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 25 13:08:45.359652 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.359652 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359663 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f3849a0-f196-4593-9ee6-1a36bf58646d-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.359857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-sys\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.359857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-root\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.359857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359773 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcdqd\" (UniqueName: \"kubernetes.io/projected/c4635f82-c644-43b0-b698-77668c27c84f-kube-api-access-fcdqd\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.359857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359812 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.359857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359835 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nvnjt\" (UniqueName: \"kubernetes.io/projected/1f3849a0-f196-4593-9ee6-1a36bf58646d-kube-api-access-nvnjt\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.360052 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-tls\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.360052 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359912 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4635f82-c644-43b0-b698-77668c27c84f-metrics-client-ca\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.360052 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-textfile\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.360052 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.359970 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-wtmp\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.360052 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.360010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.360223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.360064 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.360223 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:08:45.360178 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found May 25 13:08:45.360285 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:08:45.360252 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-tls podName:1f3849a0-f196-4593-9ee6-1a36bf58646d nodeName:}" failed. No retries permitted until 2026-05-25 13:08:45.860232708 +0000 UTC m=+179.437743343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-9h6f8" (UID: "1f3849a0-f196-4593-9ee6-1a36bf58646d") : secret "openshift-state-metrics-tls" not found May 25 13:08:45.360379 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.360350 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f3849a0-f196-4593-9ee6-1a36bf58646d-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.362209 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.362193 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.373534 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.373506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvnjt\" (UniqueName: \"kubernetes.io/projected/1f3849a0-f196-4593-9ee6-1a36bf58646d-kube-api-access-nvnjt\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.461072 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461041 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-tls\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461072 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461074 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4635f82-c644-43b0-b698-77668c27c84f-metrics-client-ca\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461288 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461099 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-textfile\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461288 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461209 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-wtmp\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461288 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461261 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461339 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461376 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-sys\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461388 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-wtmp\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461398 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-root\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461426 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-textfile\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461439 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-root\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461498 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461454 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fcdqd\" (UniqueName: \"kubernetes.io/projected/c4635f82-c644-43b0-b698-77668c27c84f-kube-api-access-fcdqd\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461819 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461610 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4635f82-c644-43b0-b698-77668c27c84f-sys\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.461819 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461752 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4635f82-c644-43b0-b698-77668c27c84f-metrics-client-ca\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.462098 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.461995 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.463908 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.463861 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-tls\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.464049 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.464030 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4635f82-c644-43b0-b698-77668c27c84f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.473644 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.473617 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcdqd\" (UniqueName: \"kubernetes.io/projected/c4635f82-c644-43b0-b698-77668c27c84f-kube-api-access-fcdqd\") pod \"node-exporter-xgxfb\" (UID: \"c4635f82-c644-43b0-b698-77668c27c84f\") " pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.565251 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.565169 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xgxfb" May 25 13:08:45.572433 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:08:45.572406 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4635f82_c644_43b0_b698_77668c27c84f.slice/crio-d8cbf3da83accd486a328aae0d74d192f304f7f00f363f08883c24bd44811986 WatchSource:0}: Error finding container d8cbf3da83accd486a328aae0d74d192f304f7f00f363f08883c24bd44811986: Status 404 returned error can't find the container with id d8cbf3da83accd486a328aae0d74d192f304f7f00f363f08883c24bd44811986 May 25 13:08:45.863516 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.863434 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:45.865732 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:45.865713 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f3849a0-f196-4593-9ee6-1a36bf58646d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-9h6f8\" (UID: \"1f3849a0-f196-4593-9ee6-1a36bf58646d\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:46.140646 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:46.140574 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" May 25 13:08:46.276714 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:46.276684 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8"] May 25 13:08:46.435753 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:08:46.435673 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3849a0_f196_4593_9ee6_1a36bf58646d.slice/crio-feeec828adaf30df652bee2cfda294cf49b673e5ead7d182488e72e1b81e2dd2 WatchSource:0}: Error finding container feeec828adaf30df652bee2cfda294cf49b673e5ead7d182488e72e1b81e2dd2: Status 404 returned error can't find the container with id feeec828adaf30df652bee2cfda294cf49b673e5ead7d182488e72e1b81e2dd2 May 25 13:08:46.509530 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:46.509489 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgxfb" event={"ID":"c4635f82-c644-43b0-b698-77668c27c84f","Type":"ContainerStarted","Data":"d8cbf3da83accd486a328aae0d74d192f304f7f00f363f08883c24bd44811986"} May 25 13:08:46.510806 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:46.510774 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" event={"ID":"1f3849a0-f196-4593-9ee6-1a36bf58646d","Type":"ContainerStarted","Data":"ae36c4b3aea5a914ccabbf9a9f61720115744d74535a0ca65ff454feed798a54"} May 25 13:08:46.511016 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:46.510812 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" event={"ID":"1f3849a0-f196-4593-9ee6-1a36bf58646d","Type":"ContainerStarted","Data":"feeec828adaf30df652bee2cfda294cf49b673e5ead7d182488e72e1b81e2dd2"} May 25 13:08:47.514943 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:47.514909 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" event={"ID":"1f3849a0-f196-4593-9ee6-1a36bf58646d","Type":"ContainerStarted","Data":"eafcd83e80dcc553a74baf3eb0b59bdf8eb5e484f5161bc9801e8580cf39c0b1"} May 25 13:08:47.516187 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:47.516160 2577 generic.go:358] "Generic (PLEG): container finished" podID="c4635f82-c644-43b0-b698-77668c27c84f" containerID="ca55353b629fe15693c4a80ba93e8048105dd33d6402b1c0463351b7d0dcc426" exitCode=0 May 25 13:08:47.516309 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:47.516223 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgxfb" event={"ID":"c4635f82-c644-43b0-b698-77668c27c84f","Type":"ContainerDied","Data":"ca55353b629fe15693c4a80ba93e8048105dd33d6402b1c0463351b7d0dcc426"} May 25 13:08:48.520334 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:48.520289 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgxfb" event={"ID":"c4635f82-c644-43b0-b698-77668c27c84f","Type":"ContainerStarted","Data":"b8c66f0200e6bb7771231812238e65b00401311e0e63d3e965069a364729d941"} May 25 13:08:48.520334 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:48.520340 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgxfb" event={"ID":"c4635f82-c644-43b0-b698-77668c27c84f","Type":"ContainerStarted","Data":"6d410d9e5dd4c22ebc310b69444d563007954c0a4b6b92f2f92e2946f5862c98"} May 25 13:08:48.522105 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:48.522082 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" event={"ID":"1f3849a0-f196-4593-9ee6-1a36bf58646d","Type":"ContainerStarted","Data":"63c63c3719d20f15304dd1d787cbcee8c1826224785b6e965d234c61f5d44328"} May 25 13:08:48.539456 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:48.539417 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xgxfb" podStartSLOduration=2.631523591 podStartE2EDuration="3.539405899s" podCreationTimestamp="2026-05-25 13:08:45 +0000 UTC" firstStartedPulling="2026-05-25 13:08:45.573975622 +0000 UTC m=+179.151485987" lastFinishedPulling="2026-05-25 13:08:46.48185793 +0000 UTC m=+180.059368295" observedRunningTime="2026-05-25 13:08:48.538602945 +0000 UTC m=+182.116113330" watchObservedRunningTime="2026-05-25 13:08:48.539405899 +0000 UTC m=+182.116916280" May 25 13:08:48.555143 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:48.555106 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-9h6f8" podStartSLOduration=2.546889595 podStartE2EDuration="3.555095408s" podCreationTimestamp="2026-05-25 13:08:45 +0000 UTC" firstStartedPulling="2026-05-25 13:08:46.563741997 +0000 UTC m=+180.141252364" lastFinishedPulling="2026-05-25 13:08:47.571947814 +0000 UTC m=+181.149458177" observedRunningTime="2026-05-25 13:08:48.554533133 +0000 UTC m=+182.132043516" watchObservedRunningTime="2026-05-25 13:08:48.555095408 +0000 UTC m=+182.132605788" May 25 13:08:49.543537 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.543506 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5665d77bc9-xcxg9"] May 25 13:08:49.546681 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.546665 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.549078 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.549052 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 25 13:08:49.549205 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.549093 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" May 25 13:08:49.549205 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.549115 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-p46mq\"" May 25 13:08:49.549868 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.549823 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" May 25 13:08:49.550340 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.550323 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" May 25 13:08:49.550403 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.550340 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b5nheohavfjgr\"" May 25 13:08:49.558439 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.558417 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5665d77bc9-xcxg9"] May 25 13:08:49.594353 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594323 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-client-ca-bundle\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.594483 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-secret-metrics-server-client-certs\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.594601 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.594731 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-audit-log\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.594800 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-secret-metrics-server-tls\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.594857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-metrics-server-audit-profiles\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.594857 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.594849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snx2m\" (UniqueName: \"kubernetes.io/projected/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-kube-api-access-snx2m\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.695773 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.695737 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.695773 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.695774 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-audit-log\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.695981 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.695794 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-secret-metrics-server-tls\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696028 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696003 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-metrics-server-audit-profiles\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696075 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696061 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-snx2m\" (UniqueName: \"kubernetes.io/projected/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-kube-api-access-snx2m\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696194 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696176 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-audit-log\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696249 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696193 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-client-ca-bundle\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696307 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696287 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-secret-metrics-server-client-certs\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696587 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696565 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.696862 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.696845 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-metrics-server-audit-profiles\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.698580 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.698554 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-secret-metrics-server-tls\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.698751 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.698732 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-client-ca-bundle\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.698803 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.698732 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-secret-metrics-server-client-certs\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.703511 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.703493 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snx2m\" (UniqueName: \"kubernetes.io/projected/a3c31591-98e0-4e2c-a904-17c3f46b5fe5-kube-api-access-snx2m\") pod \"metrics-server-5665d77bc9-xcxg9\" (UID: \"a3c31591-98e0-4e2c-a904-17c3f46b5fe5\") " pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.855987 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.855906 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:08:49.979545 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:49.979518 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5665d77bc9-xcxg9"] May 25 13:08:49.982211 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:08:49.982174 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c31591_98e0_4e2c_a904_17c3f46b5fe5.slice/crio-f1c9ce3f1cb982728ba0225eb4fa9268d94a09dcaee244058f18bae9b8b740a3 WatchSource:0}: Error finding container f1c9ce3f1cb982728ba0225eb4fa9268d94a09dcaee244058f18bae9b8b740a3: Status 404 returned error can't find the container with id f1c9ce3f1cb982728ba0225eb4fa9268d94a09dcaee244058f18bae9b8b740a3 May 25 13:08:50.529007 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:50.528964 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" event={"ID":"a3c31591-98e0-4e2c-a904-17c3f46b5fe5","Type":"ContainerStarted","Data":"f1c9ce3f1cb982728ba0225eb4fa9268d94a09dcaee244058f18bae9b8b740a3"} May 25 13:08:51.537495 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:51.537454 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" event={"ID":"a3c31591-98e0-4e2c-a904-17c3f46b5fe5","Type":"ContainerStarted","Data":"c5b1b5c2b44c1ef146637891fb304140366aa3235efde62a580bbb4be851988d"} May 25 13:08:51.555167 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:08:51.555102 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" podStartSLOduration=1.174101044 podStartE2EDuration="2.555084847s" podCreationTimestamp="2026-05-25 13:08:49 +0000 UTC" firstStartedPulling="2026-05-25 13:08:49.984076439 +0000 UTC m=+183.561586801" lastFinishedPulling="2026-05-25 13:08:51.365060239 +0000 UTC m=+184.942570604" observedRunningTime="2026-05-25 13:08:51.554339451 +0000 UTC m=+185.131849835" watchObservedRunningTime="2026-05-25 13:08:51.555084847 +0000 UTC m=+185.132595232" May 25 13:09:06.267668 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:06.267613 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" podUID="e3ee6a10-41b8-43c4-bc56-22f846ab0ac6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 25 13:09:09.855982 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:09.855935 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:09:09.856405 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:09.856042 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:09:16.266924 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:16.266867 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" podUID="e3ee6a10-41b8-43c4-bc56-22f846ab0ac6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 25 13:09:26.267654 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.267609 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" podUID="e3ee6a10-41b8-43c4-bc56-22f846ab0ac6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 25 13:09:26.268128 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.267680 2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" May 25 13:09:26.268266 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.268229 2577 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"20a0d839ce18298c687371d39e2b8f1981eb3ed28bf745891fc9d39030d54dfa"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 25 13:09:26.268317 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.268302 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" podUID="e3ee6a10-41b8-43c4-bc56-22f846ab0ac6" containerName="service-proxy" containerID="cri-o://20a0d839ce18298c687371d39e2b8f1981eb3ed28bf745891fc9d39030d54dfa" gracePeriod=30 May 25 13:09:26.634153 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.634061 2577 generic.go:358] "Generic (PLEG): container finished" podID="e3ee6a10-41b8-43c4-bc56-22f846ab0ac6" containerID="20a0d839ce18298c687371d39e2b8f1981eb3ed28bf745891fc9d39030d54dfa" exitCode=2 May 25 13:09:26.634153 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.634130 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" event={"ID":"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6","Type":"ContainerDied","Data":"20a0d839ce18298c687371d39e2b8f1981eb3ed28bf745891fc9d39030d54dfa"} May 25 13:09:26.634343 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:26.634169 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-847d7869c9-gvnjd" event={"ID":"e3ee6a10-41b8-43c4-bc56-22f846ab0ac6","Type":"ContainerStarted","Data":"82fa0ba1d70bd13ca0eb0bc375e5b67646d8805c46a4fb4a69f74abcb0a3655e"} May 25 13:09:29.862103 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:29.862072 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:09:29.865895 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:29.865850 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5665d77bc9-xcxg9" May 25 13:09:58.709535 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:58.709483 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:09:58.711788 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:58.711763 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7088f44e-edb0-4b89-89dc-da82bc7e466a-metrics-certs\") pod \"network-metrics-daemon-44458\" (UID: \"7088f44e-edb0-4b89-89dc-da82bc7e466a\") " pod="openshift-multus/network-metrics-daemon-44458" May 25 13:09:58.731478 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:58.731450 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxkxl\"" May 25 13:09:58.740251 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:58.740233 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44458" May 25 13:09:58.867920 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:58.867865 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44458"] May 25 13:09:58.871399 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:09:58.871366 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7088f44e_edb0_4b89_89dc_da82bc7e466a.slice/crio-211caf65f5f9b7304da5ed6427cb391f5075d3c3c469bfc06efd027a7273062d WatchSource:0}: Error finding container 211caf65f5f9b7304da5ed6427cb391f5075d3c3c469bfc06efd027a7273062d: Status 404 returned error can't find the container with id 211caf65f5f9b7304da5ed6427cb391f5075d3c3c469bfc06efd027a7273062d May 25 13:09:59.724375 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:09:59.724339 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44458" event={"ID":"7088f44e-edb0-4b89-89dc-da82bc7e466a","Type":"ContainerStarted","Data":"211caf65f5f9b7304da5ed6427cb391f5075d3c3c469bfc06efd027a7273062d"} May 25 13:10:00.727823 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:10:00.727786 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44458" event={"ID":"7088f44e-edb0-4b89-89dc-da82bc7e466a","Type":"ContainerStarted","Data":"6d120c7f1a5568e1d72fcb51c6654036bc338edcea0805c578af5d32047ad48f"} May 25 13:10:00.727823 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:10:00.727823 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44458" event={"ID":"7088f44e-edb0-4b89-89dc-da82bc7e466a","Type":"ContainerStarted","Data":"d5583572ffb27c0587f7da909de4d7722b07a3d54c01cd1f9fa4d9118cc0f592"} May 25 13:10:00.745364 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:10:00.745319 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-44458" podStartSLOduration=252.880299934 podStartE2EDuration="4m13.745306137s" podCreationTimestamp="2026-05-25 13:05:47 +0000 UTC" firstStartedPulling="2026-05-25 13:09:58.87321748 +0000 UTC m=+252.450727842" lastFinishedPulling="2026-05-25 13:09:59.738223668 +0000 UTC m=+253.315734045" observedRunningTime="2026-05-25 13:10:00.744129648 +0000 UTC m=+254.321640085" watchObservedRunningTime="2026-05-25 13:10:00.745306137 +0000 UTC m=+254.322816521" May 25 13:10:46.858210 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:10:46.858184 2577 kubelet.go:1628] "Image garbage collection succeeded" May 25 13:17:20.321563 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.321527 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-854tt"] May 25 13:17:20.323336 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.323321 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.327037 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.327000 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" May 25 13:17:20.327168 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.327124 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" May 25 13:17:20.327305 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.327286 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-2t6ms\"" May 25 13:17:20.328126 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.328109 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" May 25 13:17:20.328237 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.328184 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" May 25 13:17:20.328614 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.328597 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" May 25 13:17:20.337686 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.337668 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-854tt"] May 25 13:17:20.423442 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.423409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vlgj\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-kube-api-access-4vlgj\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.423588 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.423454 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c4544307-3756-456f-ad9a-b78bbda71a04-cabundle0\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.423588 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.423494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.524474 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.524443 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c4544307-3756-456f-ad9a-b78bbda71a04-cabundle0\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.524621 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.524483 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.524621 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.524519 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4vlgj\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-kube-api-access-4vlgj\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.524707 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.524635 2577 secret.go:281] references non-existent secret key: ca.crt May 25 13:17:20.524707 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.524653 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt May 25 13:17:20.524707 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.524662 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-854tt: references non-existent secret key: ca.crt May 25 13:17:20.524816 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.524745 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates podName:c4544307-3756-456f-ad9a-b78bbda71a04 nodeName:}" failed. No retries permitted until 2026-05-25 13:17:21.024729322 +0000 UTC m=+694.602239685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates") pod "keda-operator-ffbb595cb-854tt" (UID: "c4544307-3756-456f-ad9a-b78bbda71a04") : references non-existent secret key: ca.crt May 25 13:17:20.525205 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.525187 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c4544307-3756-456f-ad9a-b78bbda71a04-cabundle0\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.538475 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.538453 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vlgj\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-kube-api-access-4vlgj\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:20.695985 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.695907 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-xh495"] May 25 13:17:20.698059 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.698043 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.700263 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.700242 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" May 25 13:17:20.708006 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.707986 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-xh495"] May 25 13:17:20.726116 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.726088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.726219 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.726124 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.726219 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.726157 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t57z2\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-kube-api-access-t57z2\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.826806 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.826771 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.827033 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.826822 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t57z2\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-kube-api-access-t57z2\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.827033 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.826904 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.827033 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.826999 2577 secret.go:281] references non-existent secret key: tls.crt May 25 13:17:20.827033 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.827012 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt May 25 13:17:20.827033 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.827027 2577 projected.go:264] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found May 25 13:17:20.827218 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.827042 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-xh495: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] May 25 13:17:20.827218 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:20.827094 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates podName:8e12f4e8-df58-4bc3-bb43-e4ec713e82ad nodeName:}" failed. No retries permitted until 2026-05-25 13:17:21.327076909 +0000 UTC m=+694.904587275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates") pod "keda-metrics-apiserver-7c9f485588-xh495" (UID: "8e12f4e8-df58-4bc3-bb43-e4ec713e82ad") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] May 25 13:17:20.827306 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.827226 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:20.841310 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:20.841268 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t57z2\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-kube-api-access-t57z2\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:21.028361 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:21.028277 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:21.028571 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.028469 2577 secret.go:281] references non-existent secret key: ca.crt May 25 13:17:21.028571 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.028504 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt May 25 13:17:21.028571 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.028518 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-854tt: references non-existent secret key: ca.crt May 25 13:17:21.028753 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.028579 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates podName:c4544307-3756-456f-ad9a-b78bbda71a04 nodeName:}" failed. No retries permitted until 2026-05-25 13:17:22.028562521 +0000 UTC m=+695.606072883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates") pod "keda-operator-ffbb595cb-854tt" (UID: "c4544307-3756-456f-ad9a-b78bbda71a04") : references non-existent secret key: ca.crt May 25 13:17:21.331126 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:21.331037 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:21.331454 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.331142 2577 secret.go:281] references non-existent secret key: tls.crt May 25 13:17:21.331454 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.331154 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt May 25 13:17:21.331454 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.331170 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-xh495: references non-existent secret key: tls.crt May 25 13:17:21.331454 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:21.331217 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates podName:8e12f4e8-df58-4bc3-bb43-e4ec713e82ad nodeName:}" failed. No retries permitted until 2026-05-25 13:17:22.331204548 +0000 UTC m=+695.908714910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates") pod "keda-metrics-apiserver-7c9f485588-xh495" (UID: "8e12f4e8-df58-4bc3-bb43-e4ec713e82ad") : references non-existent secret key: tls.crt May 25 13:17:22.036647 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:22.036610 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:22.036827 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.036767 2577 secret.go:281] references non-existent secret key: ca.crt May 25 13:17:22.036827 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.036786 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt May 25 13:17:22.036827 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.036795 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-854tt: references non-existent secret key: ca.crt May 25 13:17:22.037024 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.036856 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates podName:c4544307-3756-456f-ad9a-b78bbda71a04 nodeName:}" failed. No retries permitted until 2026-05-25 13:17:24.036840876 +0000 UTC m=+697.614351237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates") pod "keda-operator-ffbb595cb-854tt" (UID: "c4544307-3756-456f-ad9a-b78bbda71a04") : references non-existent secret key: ca.crt May 25 13:17:22.339251 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:22.339158 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:22.339591 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.339296 2577 secret.go:281] references non-existent secret key: tls.crt May 25 13:17:22.339591 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.339315 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt May 25 13:17:22.339591 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.339332 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-xh495: references non-existent secret key: tls.crt May 25 13:17:22.339591 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:22.339394 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates podName:8e12f4e8-df58-4bc3-bb43-e4ec713e82ad nodeName:}" failed. No retries permitted until 2026-05-25 13:17:24.339377787 +0000 UTC m=+697.916888149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates") pod "keda-metrics-apiserver-7c9f485588-xh495" (UID: "8e12f4e8-df58-4bc3-bb43-e4ec713e82ad") : references non-existent secret key: tls.crt May 25 13:17:24.051065 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:24.051028 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:24.051434 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.051169 2577 secret.go:281] references non-existent secret key: ca.crt May 25 13:17:24.051434 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.051189 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt May 25 13:17:24.051434 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.051198 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-854tt: references non-existent secret key: ca.crt May 25 13:17:24.051434 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.051253 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates podName:c4544307-3756-456f-ad9a-b78bbda71a04 nodeName:}" failed. No retries permitted until 2026-05-25 13:17:28.051238166 +0000 UTC m=+701.628748528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates") pod "keda-operator-ffbb595cb-854tt" (UID: "c4544307-3756-456f-ad9a-b78bbda71a04") : references non-existent secret key: ca.crt May 25 13:17:24.353364 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:24.353281 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:24.353503 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.353398 2577 secret.go:281] references non-existent secret key: tls.crt May 25 13:17:24.353503 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.353409 2577 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt May 25 13:17:24.353503 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.353427 2577 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-xh495: references non-existent secret key: tls.crt May 25 13:17:24.353503 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:17:24.353476 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates podName:8e12f4e8-df58-4bc3-bb43-e4ec713e82ad nodeName:}" failed. No retries permitted until 2026-05-25 13:17:28.353464313 +0000 UTC m=+701.930974674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates") pod "keda-metrics-apiserver-7c9f485588-xh495" (UID: "8e12f4e8-df58-4bc3-bb43-e4ec713e82ad") : references non-existent secret key: tls.crt May 25 13:17:28.079044 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.079006 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:28.081404 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.081377 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c4544307-3756-456f-ad9a-b78bbda71a04-certificates\") pod \"keda-operator-ffbb595cb-854tt\" (UID: \"c4544307-3756-456f-ad9a-b78bbda71a04\") " pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:28.133032 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.133006 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:28.260733 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.260706 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-854tt"] May 25 13:17:28.262606 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:17:28.262582 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4544307_3756_456f_ad9a_b78bbda71a04.slice/crio-7d01f302f85a169c3cffaa1dfc16f7e81aa0b6659b01164721bccc900e8a6f11 WatchSource:0}: Error finding container 7d01f302f85a169c3cffaa1dfc16f7e81aa0b6659b01164721bccc900e8a6f11: Status 404 returned error can't find the container with id 7d01f302f85a169c3cffaa1dfc16f7e81aa0b6659b01164721bccc900e8a6f11 May 25 13:17:28.263922 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.263906 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 25 13:17:28.381912 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.381805 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:28.384275 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.384251 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8e12f4e8-df58-4bc3-bb43-e4ec713e82ad-certificates\") pod \"keda-metrics-apiserver-7c9f485588-xh495\" (UID: \"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:28.507596 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.507549 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:28.635003 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.634981 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-xh495"] May 25 13:17:28.637108 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:17:28.637077 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e12f4e8_df58_4bc3_bb43_e4ec713e82ad.slice/crio-bd419df2303eb5a871a18f5065b641fe5728ef3721bb0339966c759c0694c08a WatchSource:0}: Error finding container bd419df2303eb5a871a18f5065b641fe5728ef3721bb0339966c759c0694c08a: Status 404 returned error can't find the container with id bd419df2303eb5a871a18f5065b641fe5728ef3721bb0339966c759c0694c08a May 25 13:17:28.861821 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.861778 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-854tt" event={"ID":"c4544307-3756-456f-ad9a-b78bbda71a04","Type":"ContainerStarted","Data":"7d01f302f85a169c3cffaa1dfc16f7e81aa0b6659b01164721bccc900e8a6f11"} May 25 13:17:28.862654 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:28.862633 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" event={"ID":"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad","Type":"ContainerStarted","Data":"bd419df2303eb5a871a18f5065b641fe5728ef3721bb0339966c759c0694c08a"} May 25 13:17:32.876591 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:32.876489 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-854tt" event={"ID":"c4544307-3756-456f-ad9a-b78bbda71a04","Type":"ContainerStarted","Data":"91d459cc3c6204553969c7f7ec4fe71f2a9c5ceb8b3a49676721f031117fe51a"} May 25 13:17:32.877044 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:32.876596 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:17:32.877788 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:32.877768 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" event={"ID":"8e12f4e8-df58-4bc3-bb43-e4ec713e82ad","Type":"ContainerStarted","Data":"e8dcb28a30ecd070a98ae828da5d44ca6ca5f57c7e9922fb10e77924f0cb1ac5"} May 25 13:17:32.877965 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:32.877947 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:32.894205 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:32.894148 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-854tt" podStartSLOduration=8.668720476 podStartE2EDuration="12.894132617s" podCreationTimestamp="2026-05-25 13:17:20 +0000 UTC" firstStartedPulling="2026-05-25 13:17:28.26402748 +0000 UTC m=+701.841537841" lastFinishedPulling="2026-05-25 13:17:32.489439611 +0000 UTC m=+706.066949982" observedRunningTime="2026-05-25 13:17:32.89319829 +0000 UTC m=+706.470708674" watchObservedRunningTime="2026-05-25 13:17:32.894132617 +0000 UTC m=+706.471643003" May 25 13:17:32.911635 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:32.911587 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" podStartSLOduration=9.064645092 podStartE2EDuration="12.91157551s" podCreationTimestamp="2026-05-25 13:17:20 +0000 UTC" firstStartedPulling="2026-05-25 13:17:28.638432269 +0000 UTC m=+702.215942630" lastFinishedPulling="2026-05-25 13:17:32.485362683 +0000 UTC m=+706.062873048" observedRunningTime="2026-05-25 13:17:32.910293752 +0000 UTC m=+706.487804136" watchObservedRunningTime="2026-05-25 13:17:32.91157551 +0000 UTC m=+706.489085894" May 25 13:17:43.885862 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:43.885785 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-xh495" May 25 13:17:53.884138 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:17:53.884106 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-854tt" May 25 13:18:29.133167 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.133119 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-86lwr"] May 25 13:18:29.135001 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.134985 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.141567 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.141545 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" May 25 13:18:29.142675 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.142657 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" May 25 13:18:29.142775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.142659 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" May 25 13:18:29.142775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.142664 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-dqfqb\"" May 25 13:18:29.148247 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.148227 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-86lwr"] May 25 13:18:29.179463 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.179436 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-79f457656b-fd598"] May 25 13:18:29.181151 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.181135 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.183366 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.183347 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" May 25 13:18:29.183453 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.183411 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-jbsqc\"" May 25 13:18:29.191700 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.191678 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-fd598"] May 25 13:18:29.235176 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.235140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btfnt\" (UniqueName: \"kubernetes.io/projected/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-kube-api-access-btfnt\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.235176 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.235182 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.335742 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.335701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfr56\" (UniqueName: \"kubernetes.io/projected/4b84f575-535e-442a-a464-123d304d93a6-kube-api-access-nfr56\") pod \"seaweedfs-79f457656b-fd598\" (UID: \"4b84f575-535e-442a-a464-123d304d93a6\") " pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.335942 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.335763 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4b84f575-535e-442a-a464-123d304d93a6-data\") pod \"seaweedfs-79f457656b-fd598\" (UID: \"4b84f575-535e-442a-a464-123d304d93a6\") " pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.335942 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.335802 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-btfnt\" (UniqueName: \"kubernetes.io/projected/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-kube-api-access-btfnt\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.335942 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.335864 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.336047 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:18:29.335984 2577 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 25 13:18:29.340287 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:18:29.336708 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert podName:8f3ddc07-89d0-431f-93cd-a6f11ea1d580 nodeName:}" failed. No retries permitted until 2026-05-25 13:18:29.836680847 +0000 UTC m=+763.414191226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert") pod "kserve-controller-manager-7cd9b78f5d-86lwr" (UID: "8f3ddc07-89d0-431f-93cd-a6f11ea1d580") : secret "kserve-webhook-server-cert" not found May 25 13:18:29.346584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.346558 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btfnt\" (UniqueName: \"kubernetes.io/projected/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-kube-api-access-btfnt\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.436752 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.436657 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nfr56\" (UniqueName: \"kubernetes.io/projected/4b84f575-535e-442a-a464-123d304d93a6-kube-api-access-nfr56\") pod \"seaweedfs-79f457656b-fd598\" (UID: \"4b84f575-535e-442a-a464-123d304d93a6\") " pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.436752 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.436714 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4b84f575-535e-442a-a464-123d304d93a6-data\") pod \"seaweedfs-79f457656b-fd598\" (UID: \"4b84f575-535e-442a-a464-123d304d93a6\") " pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.437123 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.437106 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4b84f575-535e-442a-a464-123d304d93a6-data\") pod \"seaweedfs-79f457656b-fd598\" (UID: \"4b84f575-535e-442a-a464-123d304d93a6\") " pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.447337 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.447317 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfr56\" (UniqueName: \"kubernetes.io/projected/4b84f575-535e-442a-a464-123d304d93a6-kube-api-access-nfr56\") pod \"seaweedfs-79f457656b-fd598\" (UID: \"4b84f575-535e-442a-a464-123d304d93a6\") " pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.489646 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.489612 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:29.614947 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.614777 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-fd598"] May 25 13:18:29.617513 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:18:29.617481 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b84f575_535e_442a_a464_123d304d93a6.slice/crio-882441f34bc5cfb0020adcde0c992abc48c9f6e1a0d7fecb647f4e0aa8597762 WatchSource:0}: Error finding container 882441f34bc5cfb0020adcde0c992abc48c9f6e1a0d7fecb647f4e0aa8597762: Status 404 returned error can't find the container with id 882441f34bc5cfb0020adcde0c992abc48c9f6e1a0d7fecb647f4e0aa8597762 May 25 13:18:29.841320 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.841282 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:29.843628 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:29.843608 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert\") pod \"kserve-controller-manager-7cd9b78f5d-86lwr\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:30.032756 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:30.032717 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-fd598" event={"ID":"4b84f575-535e-442a-a464-123d304d93a6","Type":"ContainerStarted","Data":"882441f34bc5cfb0020adcde0c992abc48c9f6e1a0d7fecb647f4e0aa8597762"} May 25 13:18:30.050514 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:30.050486 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:30.188551 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:30.188512 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-86lwr"] May 25 13:18:30.192409 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:18:30.192377 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3ddc07_89d0_431f_93cd_a6f11ea1d580.slice/crio-d9ae228760bfb6a688adb2dd1f92ccc661fe24c550012c5c71ef45ab5d2efe07 WatchSource:0}: Error finding container d9ae228760bfb6a688adb2dd1f92ccc661fe24c550012c5c71ef45ab5d2efe07: Status 404 returned error can't find the container with id d9ae228760bfb6a688adb2dd1f92ccc661fe24c550012c5c71ef45ab5d2efe07 May 25 13:18:31.036744 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:31.036694 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" event={"ID":"8f3ddc07-89d0-431f-93cd-a6f11ea1d580","Type":"ContainerStarted","Data":"d9ae228760bfb6a688adb2dd1f92ccc661fe24c550012c5c71ef45ab5d2efe07"} May 25 13:18:35.050459 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:35.050422 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-fd598" event={"ID":"4b84f575-535e-442a-a464-123d304d93a6","Type":"ContainerStarted","Data":"b53e416c76df4899349b817d0cc0b6d8335063a565d27d79c9c7251c23a946aa"} May 25 13:18:35.050946 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:35.050482 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:18:35.051643 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:35.051621 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" event={"ID":"8f3ddc07-89d0-431f-93cd-a6f11ea1d580","Type":"ContainerStarted","Data":"e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82"} May 25 13:18:35.051765 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:35.051748 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:18:35.078579 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:35.078533 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-79f457656b-fd598" podStartSLOduration=1.32297132 podStartE2EDuration="6.078520122s" podCreationTimestamp="2026-05-25 13:18:29 +0000 UTC" firstStartedPulling="2026-05-25 13:18:29.61875917 +0000 UTC m=+763.196269533" lastFinishedPulling="2026-05-25 13:18:34.374307973 +0000 UTC m=+767.951818335" observedRunningTime="2026-05-25 13:18:35.078403567 +0000 UTC m=+768.655913950" watchObservedRunningTime="2026-05-25 13:18:35.078520122 +0000 UTC m=+768.656030506" May 25 13:18:35.109339 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:35.109293 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" podStartSLOduration=2.002913511 podStartE2EDuration="6.109278599s" podCreationTimestamp="2026-05-25 13:18:29 +0000 UTC" firstStartedPulling="2026-05-25 13:18:30.194192019 +0000 UTC m=+763.771702386" lastFinishedPulling="2026-05-25 13:18:34.300557111 +0000 UTC m=+767.878067474" observedRunningTime="2026-05-25 13:18:35.108807147 +0000 UTC m=+768.686317533" watchObservedRunningTime="2026-05-25 13:18:35.109278599 +0000 UTC m=+768.686788981" May 25 13:18:41.056575 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:18:41.056538 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-79f457656b-fd598" May 25 13:19:05.504758 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.504721 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-86lwr"] May 25 13:19:05.505323 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.505062 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" podUID="8f3ddc07-89d0-431f-93cd-a6f11ea1d580" containerName="manager" containerID="cri-o://e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82" gracePeriod=10 May 25 13:19:05.509786 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.509764 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:19:05.531988 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.531960 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-vpzx2"] May 25 13:19:05.534966 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.534947 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.546316 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.546295 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-vpzx2"] May 25 13:19:05.616687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.616659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c1926e3-1a2d-405e-8d80-2e9ac491b607-cert\") pod \"kserve-controller-manager-7cd9b78f5d-vpzx2\" (UID: \"9c1926e3-1a2d-405e-8d80-2e9ac491b607\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.616839 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.616708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5mtt\" (UniqueName: \"kubernetes.io/projected/9c1926e3-1a2d-405e-8d80-2e9ac491b607-kube-api-access-t5mtt\") pod \"kserve-controller-manager-7cd9b78f5d-vpzx2\" (UID: \"9c1926e3-1a2d-405e-8d80-2e9ac491b607\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.717551 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.717510 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c1926e3-1a2d-405e-8d80-2e9ac491b607-cert\") pod \"kserve-controller-manager-7cd9b78f5d-vpzx2\" (UID: \"9c1926e3-1a2d-405e-8d80-2e9ac491b607\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.717703 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.717576 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5mtt\" (UniqueName: \"kubernetes.io/projected/9c1926e3-1a2d-405e-8d80-2e9ac491b607-kube-api-access-t5mtt\") pod \"kserve-controller-manager-7cd9b78f5d-vpzx2\" (UID: \"9c1926e3-1a2d-405e-8d80-2e9ac491b607\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.720101 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.720073 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c1926e3-1a2d-405e-8d80-2e9ac491b607-cert\") pod \"kserve-controller-manager-7cd9b78f5d-vpzx2\" (UID: \"9c1926e3-1a2d-405e-8d80-2e9ac491b607\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.726687 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.726657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5mtt\" (UniqueName: \"kubernetes.io/projected/9c1926e3-1a2d-405e-8d80-2e9ac491b607-kube-api-access-t5mtt\") pod \"kserve-controller-manager-7cd9b78f5d-vpzx2\" (UID: \"9c1926e3-1a2d-405e-8d80-2e9ac491b607\") " pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.743209 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.743186 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:19:05.818574 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.818500 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert\") pod \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " May 25 13:19:05.818574 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.818546 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btfnt\" (UniqueName: \"kubernetes.io/projected/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-kube-api-access-btfnt\") pod \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\" (UID: \"8f3ddc07-89d0-431f-93cd-a6f11ea1d580\") " May 25 13:19:05.820697 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.820671 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert" (OuterVolumeSpecName: "cert") pod "8f3ddc07-89d0-431f-93cd-a6f11ea1d580" (UID: "8f3ddc07-89d0-431f-93cd-a6f11ea1d580"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 25 13:19:05.820794 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.820716 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-kube-api-access-btfnt" (OuterVolumeSpecName: "kube-api-access-btfnt") pod "8f3ddc07-89d0-431f-93cd-a6f11ea1d580" (UID: "8f3ddc07-89d0-431f-93cd-a6f11ea1d580"). InnerVolumeSpecName "kube-api-access-btfnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 25 13:19:05.880524 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.880485 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:05.919648 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.919611 2577 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-cert\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:19:05.919648 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:05.919648 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btfnt\" (UniqueName: \"kubernetes.io/projected/8f3ddc07-89d0-431f-93cd-a6f11ea1d580-kube-api-access-btfnt\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:19:06.005817 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.005788 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-vpzx2"] May 25 13:19:06.008225 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:19:06.008196 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1926e3_1a2d_405e_8d80_2e9ac491b607.slice/crio-478eacc4268be465c9ba849b61271c3cd995e19fdc0d177eae21d481af9f23b5 WatchSource:0}: Error finding container 478eacc4268be465c9ba849b61271c3cd995e19fdc0d177eae21d481af9f23b5: Status 404 returned error can't find the container with id 478eacc4268be465c9ba849b61271c3cd995e19fdc0d177eae21d481af9f23b5 May 25 13:19:06.133600 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.133513 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" event={"ID":"9c1926e3-1a2d-405e-8d80-2e9ac491b607","Type":"ContainerStarted","Data":"478eacc4268be465c9ba849b61271c3cd995e19fdc0d177eae21d481af9f23b5"} May 25 13:19:06.136630 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.136597 2577 generic.go:358] "Generic (PLEG): container finished" podID="8f3ddc07-89d0-431f-93cd-a6f11ea1d580" containerID="e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82" exitCode=0 May 25 13:19:06.136785 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.136646 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" event={"ID":"8f3ddc07-89d0-431f-93cd-a6f11ea1d580","Type":"ContainerDied","Data":"e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82"} May 25 13:19:06.136785 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.136669 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" May 25 13:19:06.136785 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.136685 2577 scope.go:117] "RemoveContainer" containerID="e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82" May 25 13:19:06.136972 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.136673 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7cd9b78f5d-86lwr" event={"ID":"8f3ddc07-89d0-431f-93cd-a6f11ea1d580","Type":"ContainerDied","Data":"d9ae228760bfb6a688adb2dd1f92ccc661fe24c550012c5c71ef45ab5d2efe07"} May 25 13:19:06.144464 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.144433 2577 scope.go:117] "RemoveContainer" containerID="e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82" May 25 13:19:06.144942 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:19:06.144917 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82\": container with ID starting with e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82 not found: ID does not exist" containerID="e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82" May 25 13:19:06.145052 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.144947 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82"} err="failed to get container status \"e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82\": rpc error: code = NotFound desc = could not find container \"e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82\": container with ID starting with e3fd0461387f06723fe3e80b04b088284f6794d4ae39cd2ba13f005d67d3fd82 not found: ID does not exist" May 25 13:19:06.179018 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.178986 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-86lwr"] May 25 13:19:06.184431 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.184408 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7cd9b78f5d-86lwr"] May 25 13:19:06.932152 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:06.932073 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f3ddc07-89d0-431f-93cd-a6f11ea1d580" path="/var/lib/kubelet/pods/8f3ddc07-89d0-431f-93cd-a6f11ea1d580/volumes" May 25 13:19:07.140182 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:07.140145 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" event={"ID":"9c1926e3-1a2d-405e-8d80-2e9ac491b607","Type":"ContainerStarted","Data":"55919fcdbd06c018707b9a1eef9f60ffff97071f7926eca634bf9ffd09b249f0"} May 25 13:19:07.140355 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:07.140212 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:07.164781 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:07.164730 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" podStartSLOduration=1.626946206 podStartE2EDuration="2.164714987s" podCreationTimestamp="2026-05-25 13:19:05 +0000 UTC" firstStartedPulling="2026-05-25 13:19:06.009441717 +0000 UTC m=+799.586952078" lastFinishedPulling="2026-05-25 13:19:06.547210498 +0000 UTC m=+800.124720859" observedRunningTime="2026-05-25 13:19:07.163979615 +0000 UTC m=+800.741490001" watchObservedRunningTime="2026-05-25 13:19:07.164714987 +0000 UTC m=+800.742225421" May 25 13:19:38.149529 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:38.149496 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7cd9b78f5d-vpzx2" May 25 13:19:39.116635 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.116602 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-86f7b4b499-tqc6p"] May 25 13:19:39.116890 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.116864 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f3ddc07-89d0-431f-93cd-a6f11ea1d580" containerName="manager" May 25 13:19:39.116890 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.116889 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f3ddc07-89d0-431f-93cd-a6f11ea1d580" containerName="manager" May 25 13:19:39.116976 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.116936 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f3ddc07-89d0-431f-93cd-a6f11ea1d580" containerName="manager" May 25 13:19:39.119721 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.119705 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.123077 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.123051 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" May 25 13:19:39.123433 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.123418 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-b5ch5\"" May 25 13:19:39.131476 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.131452 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-tqc6p"] May 25 13:19:39.279655 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.279619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67xt7\" (UniqueName: \"kubernetes.io/projected/ea79699a-3ea9-486d-81cb-af35171e349f-kube-api-access-67xt7\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.280090 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.279688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea79699a-3ea9-486d-81cb-af35171e349f-tls-certs\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.380513 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.380433 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea79699a-3ea9-486d-81cb-af35171e349f-tls-certs\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.380513 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.380489 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-67xt7\" (UniqueName: \"kubernetes.io/projected/ea79699a-3ea9-486d-81cb-af35171e349f-kube-api-access-67xt7\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.380691 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:19:39.380583 2577 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found May 25 13:19:39.380691 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:19:39.380660 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea79699a-3ea9-486d-81cb-af35171e349f-tls-certs podName:ea79699a-3ea9-486d-81cb-af35171e349f nodeName:}" failed. No retries permitted until 2026-05-25 13:19:39.880640368 +0000 UTC m=+833.458150752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ea79699a-3ea9-486d-81cb-af35171e349f-tls-certs") pod "model-serving-api-86f7b4b499-tqc6p" (UID: "ea79699a-3ea9-486d-81cb-af35171e349f") : secret "model-serving-api-tls" not found May 25 13:19:39.389161 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.389142 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67xt7\" (UniqueName: \"kubernetes.io/projected/ea79699a-3ea9-486d-81cb-af35171e349f-kube-api-access-67xt7\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.884518 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.884475 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea79699a-3ea9-486d-81cb-af35171e349f-tls-certs\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:39.886952 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:39.886933 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea79699a-3ea9-486d-81cb-af35171e349f-tls-certs\") pod \"model-serving-api-86f7b4b499-tqc6p\" (UID: \"ea79699a-3ea9-486d-81cb-af35171e349f\") " pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:40.029831 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:40.029776 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:40.155065 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:40.155037 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-tqc6p"] May 25 13:19:40.157417 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:19:40.157389 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea79699a_3ea9_486d_81cb_af35171e349f.slice/crio-687ee42d0930b50e326fdfff078aa28b66d122dca1fa4843b82bc6daef26e1e9 WatchSource:0}: Error finding container 687ee42d0930b50e326fdfff078aa28b66d122dca1fa4843b82bc6daef26e1e9: Status 404 returned error can't find the container with id 687ee42d0930b50e326fdfff078aa28b66d122dca1fa4843b82bc6daef26e1e9 May 25 13:19:40.230979 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:40.230946 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-tqc6p" event={"ID":"ea79699a-3ea9-486d-81cb-af35171e349f","Type":"ContainerStarted","Data":"687ee42d0930b50e326fdfff078aa28b66d122dca1fa4843b82bc6daef26e1e9"} May 25 13:19:43.241613 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:43.241573 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-tqc6p" event={"ID":"ea79699a-3ea9-486d-81cb-af35171e349f","Type":"ContainerStarted","Data":"f49c3c1b71f819a0331a2d44a5584198471994b7df80056badf2fde98eb30cf1"} May 25 13:19:43.242031 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:43.241641 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:19:43.259697 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:43.259650 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/model-serving-api-86f7b4b499-tqc6p" podStartSLOduration=1.9263125030000001 podStartE2EDuration="4.259638464s" podCreationTimestamp="2026-05-25 13:19:39 +0000 UTC" firstStartedPulling="2026-05-25 13:19:40.159248513 +0000 UTC m=+833.736758879" lastFinishedPulling="2026-05-25 13:19:42.492574465 +0000 UTC m=+836.070084840" observedRunningTime="2026-05-25 13:19:43.258141802 +0000 UTC m=+836.835652187" watchObservedRunningTime="2026-05-25 13:19:43.259638464 +0000 UTC m=+836.837148847" May 25 13:19:54.248487 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:19:54.248457 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-86f7b4b499-tqc6p" May 25 13:22:02.217073 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.217033 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6"] May 25 13:22:02.220450 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.220425 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.224593 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.224563 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" May 25 13:22:02.224593 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.224569 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\"" May 25 13:22:02.224799 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.224630 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" May 25 13:22:02.225319 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.225302 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-cmtw7\"" May 25 13:22:02.225372 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.225308 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-a698c-predictor-serving-cert\"" May 25 13:22:02.258319 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.258294 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6"] May 25 13:22:02.396585 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.396546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj8jf\" (UniqueName: \"kubernetes.io/projected/7a91b4c6-8d9a-455e-91ee-07693f039fda-kube-api-access-kj8jf\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.396775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.396605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a91b4c6-8d9a-455e-91ee-07693f039fda-isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.396775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.396626 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a91b4c6-8d9a-455e-91ee-07693f039fda-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.396775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.396650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.497863 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.497779 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a91b4c6-8d9a-455e-91ee-07693f039fda-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.497863 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.497822 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.497863 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.497866 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8jf\" (UniqueName: \"kubernetes.io/projected/7a91b4c6-8d9a-455e-91ee-07693f039fda-kube-api-access-kj8jf\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.498117 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.497919 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a91b4c6-8d9a-455e-91ee-07693f039fda-isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.498117 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:22:02.498024 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-serving-cert: secret "isvc-xgboost-graph-raw-a698c-predictor-serving-cert" not found May 25 13:22:02.498117 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:22:02.498100 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls podName:7a91b4c6-8d9a-455e-91ee-07693f039fda nodeName:}" failed. No retries permitted until 2026-05-25 13:22:02.998081043 +0000 UTC m=+976.575591422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls") pod "isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" (UID: "7a91b4c6-8d9a-455e-91ee-07693f039fda") : secret "isvc-xgboost-graph-raw-a698c-predictor-serving-cert" not found May 25 13:22:02.498221 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.498201 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a91b4c6-8d9a-455e-91ee-07693f039fda-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.498506 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.498489 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a91b4c6-8d9a-455e-91ee-07693f039fda-isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:02.508968 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:02.508947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8jf\" (UniqueName: \"kubernetes.io/projected/7a91b4c6-8d9a-455e-91ee-07693f039fda-kube-api-access-kj8jf\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:03.002407 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:03.002370 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:03.004734 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:03.004713 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls\") pod \"isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:03.130372 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:03.130335 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:03.286000 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:03.285968 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6"] May 25 13:22:03.286378 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:22:03.286198 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a91b4c6_8d9a_455e_91ee_07693f039fda.slice/crio-ebae1356e2caa65f1129355b0471f6ce147da18c583c01a6bb0191dddf036e71 WatchSource:0}: Error finding container ebae1356e2caa65f1129355b0471f6ce147da18c583c01a6bb0191dddf036e71: Status 404 returned error can't find the container with id ebae1356e2caa65f1129355b0471f6ce147da18c583c01a6bb0191dddf036e71 May 25 13:22:03.630223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:03.630140 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerStarted","Data":"ebae1356e2caa65f1129355b0471f6ce147da18c583c01a6bb0191dddf036e71"} May 25 13:22:06.641356 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:06.641313 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerStarted","Data":"d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879"} May 25 13:22:10.653663 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:10.653627 2577 generic.go:358] "Generic (PLEG): container finished" podID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerID="d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879" exitCode=0 May 25 13:22:10.654151 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:10.653704 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerDied","Data":"d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879"} May 25 13:22:28.765075 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:28.765052 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 25 13:22:29.713371 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:29.713330 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerStarted","Data":"e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000"} May 25 13:22:31.723405 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:31.723370 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerStarted","Data":"9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d"} May 25 13:22:31.723774 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:31.723521 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:31.743774 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:31.743713 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podStartSLOduration=1.534003754 podStartE2EDuration="29.743695628s" podCreationTimestamp="2026-05-25 13:22:02 +0000 UTC" firstStartedPulling="2026-05-25 13:22:03.287984796 +0000 UTC m=+976.865495157" lastFinishedPulling="2026-05-25 13:22:31.497676669 +0000 UTC m=+1005.075187031" observedRunningTime="2026-05-25 13:22:31.74199002 +0000 UTC m=+1005.319500405" watchObservedRunningTime="2026-05-25 13:22:31.743695628 +0000 UTC m=+1005.321206014" May 25 13:22:32.726787 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:32.726753 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:32.727922 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:32.727886 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:22:33.729337 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:33.729297 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:22:38.734041 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:38.734011 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:22:38.734476 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:38.734450 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:22:48.735219 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:48.735182 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:22:58.735091 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:22:58.735047 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:23:08.734693 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:08.734649 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:23:18.734467 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:18.734422 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused" May 25 13:23:28.735131 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:28.735100 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:23:41.970296 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:41.970215 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6"] May 25 13:23:41.970787 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:41.970552 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" containerID="cri-o://e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000" gracePeriod=30 May 25 13:23:41.970787 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:41.970569 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kube-rbac-proxy" containerID="cri-o://9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d" gracePeriod=30 May 25 13:23:42.004203 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.004162 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt"] May 25 13:23:42.007454 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.007434 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.009973 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.009949 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\"" May 25 13:23:42.010073 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.009956 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-12a34-predictor-serving-cert\"" May 25 13:23:42.017538 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.017510 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt"] May 25 13:23:42.114036 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.114000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfhm\" (UniqueName: \"kubernetes.io/projected/216efa27-e914-40d4-bd1a-dc6bf677e801-kube-api-access-clfhm\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.114190 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.114046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/216efa27-e914-40d4-bd1a-dc6bf677e801-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.114190 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.114134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/216efa27-e914-40d4-bd1a-dc6bf677e801-isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.114190 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.114173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/216efa27-e914-40d4-bd1a-dc6bf677e801-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.215493 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.215451 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/216efa27-e914-40d4-bd1a-dc6bf677e801-isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.215493 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.215501 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/216efa27-e914-40d4-bd1a-dc6bf677e801-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.215736 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.215534 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clfhm\" (UniqueName: \"kubernetes.io/projected/216efa27-e914-40d4-bd1a-dc6bf677e801-kube-api-access-clfhm\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.215736 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.215559 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/216efa27-e914-40d4-bd1a-dc6bf677e801-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.216045 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.216021 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/216efa27-e914-40d4-bd1a-dc6bf677e801-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.216154 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.216136 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/216efa27-e914-40d4-bd1a-dc6bf677e801-isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.218028 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.218007 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/216efa27-e914-40d4-bd1a-dc6bf677e801-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.225580 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.225530 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfhm\" (UniqueName: \"kubernetes.io/projected/216efa27-e914-40d4-bd1a-dc6bf677e801-kube-api-access-clfhm\") pod \"isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.318533 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.318500 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:42.450838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.450810 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt"] May 25 13:23:42.453782 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:23:42.453744 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216efa27_e914_40d4_bd1a_dc6bf677e801.slice/crio-ce2c67255481b41a5c5e5e3ba3d1eb2150fe0860be715beba1c5cde668701fc6 WatchSource:0}: Error finding container ce2c67255481b41a5c5e5e3ba3d1eb2150fe0860be715beba1c5cde668701fc6: Status 404 returned error can't find the container with id ce2c67255481b41a5c5e5e3ba3d1eb2150fe0860be715beba1c5cde668701fc6 May 25 13:23:42.923846 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.923745 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerStarted","Data":"7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072"} May 25 13:23:42.923846 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.923800 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerStarted","Data":"ce2c67255481b41a5c5e5e3ba3d1eb2150fe0860be715beba1c5cde668701fc6"} May 25 13:23:42.925623 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.925597 2577 generic.go:358] "Generic (PLEG): container finished" podID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerID="9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d" exitCode=2 May 25 13:23:42.925735 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:42.925639 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerDied","Data":"9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d"} May 25 13:23:43.730166 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:43.730121 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.19:8643/healthz\": dial tcp 10.133.0.19:8643: connect: connection refused" May 25 13:23:45.616970 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.616938 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:23:45.746667 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.746638 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj8jf\" (UniqueName: \"kubernetes.io/projected/7a91b4c6-8d9a-455e-91ee-07693f039fda-kube-api-access-kj8jf\") pod \"7a91b4c6-8d9a-455e-91ee-07693f039fda\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " May 25 13:23:45.746837 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.746689 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a91b4c6-8d9a-455e-91ee-07693f039fda-isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\") pod \"7a91b4c6-8d9a-455e-91ee-07693f039fda\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " May 25 13:23:45.746837 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.746712 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a91b4c6-8d9a-455e-91ee-07693f039fda-kserve-provision-location\") pod \"7a91b4c6-8d9a-455e-91ee-07693f039fda\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " May 25 13:23:45.746837 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.746731 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls\") pod \"7a91b4c6-8d9a-455e-91ee-07693f039fda\" (UID: \"7a91b4c6-8d9a-455e-91ee-07693f039fda\") " May 25 13:23:45.747070 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.747043 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a91b4c6-8d9a-455e-91ee-07693f039fda-isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config") pod "7a91b4c6-8d9a-455e-91ee-07693f039fda" (UID: "7a91b4c6-8d9a-455e-91ee-07693f039fda"). InnerVolumeSpecName "isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 25 13:23:45.747140 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.747076 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a91b4c6-8d9a-455e-91ee-07693f039fda-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7a91b4c6-8d9a-455e-91ee-07693f039fda" (UID: "7a91b4c6-8d9a-455e-91ee-07693f039fda"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 25 13:23:45.749018 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.748995 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a91b4c6-8d9a-455e-91ee-07693f039fda-kube-api-access-kj8jf" (OuterVolumeSpecName: "kube-api-access-kj8jf") pod "7a91b4c6-8d9a-455e-91ee-07693f039fda" (UID: "7a91b4c6-8d9a-455e-91ee-07693f039fda"). InnerVolumeSpecName "kube-api-access-kj8jf". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 25 13:23:45.749018 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.748998 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7a91b4c6-8d9a-455e-91ee-07693f039fda" (UID: "7a91b4c6-8d9a-455e-91ee-07693f039fda"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 25 13:23:45.847794 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.847755 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj8jf\" (UniqueName: \"kubernetes.io/projected/7a91b4c6-8d9a-455e-91ee-07693f039fda-kube-api-access-kj8jf\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:23:45.847794 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.847787 2577 reconciler_common.go:299] "Volume detached for volume \"isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a91b4c6-8d9a-455e-91ee-07693f039fda-isvc-xgboost-graph-raw-a698c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:23:45.847794 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.847800 2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a91b4c6-8d9a-455e-91ee-07693f039fda-kserve-provision-location\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:23:45.848032 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.847809 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a91b4c6-8d9a-455e-91ee-07693f039fda-proxy-tls\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:23:45.937066 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.937026 2577 generic.go:358] "Generic (PLEG): container finished" podID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerID="e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000" exitCode=0 May 25 13:23:45.937238 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.937100 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerDied","Data":"e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000"} May 25 13:23:45.937238 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.937137 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" event={"ID":"7a91b4c6-8d9a-455e-91ee-07693f039fda","Type":"ContainerDied","Data":"ebae1356e2caa65f1129355b0471f6ce147da18c583c01a6bb0191dddf036e71"} May 25 13:23:45.937238 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.937142 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6" May 25 13:23:45.937238 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.937154 2577 scope.go:117] "RemoveContainer" containerID="9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d" May 25 13:23:45.945421 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.945399 2577 scope.go:117] "RemoveContainer" containerID="e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000" May 25 13:23:45.952272 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.952256 2577 scope.go:117] "RemoveContainer" containerID="d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879" May 25 13:23:45.958972 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.958946 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6"] May 25 13:23:45.959147 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.959104 2577 scope.go:117] "RemoveContainer" containerID="9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d" May 25 13:23:45.959372 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:23:45.959352 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d\": container with ID starting with 9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d not found: ID does not exist" containerID="9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d" May 25 13:23:45.959456 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.959383 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d"} err="failed to get container status \"9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d\": rpc error: code = NotFound desc = could not find container \"9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d\": container with ID starting with 9f0a1b728ea4bc7a795dabd10ea885e8070ffd397f5a19b2e2f75b67248b156d not found: ID does not exist" May 25 13:23:45.959456 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.959409 2577 scope.go:117] "RemoveContainer" containerID="e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000" May 25 13:23:45.959641 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:23:45.959624 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000\": container with ID starting with e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000 not found: ID does not exist" containerID="e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000" May 25 13:23:45.959677 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.959647 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000"} err="failed to get container status \"e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000\": rpc error: code = NotFound desc = could not find container \"e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000\": container with ID starting with e955cb63a614f7bdb8579311e49f1e0c683466c732debd843586beb94c766000 not found: ID does not exist" May 25 13:23:45.959677 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.959662 2577 scope.go:117] "RemoveContainer" containerID="d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879" May 25 13:23:45.959860 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:23:45.959841 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879\": container with ID starting with d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879 not found: ID does not exist" containerID="d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879" May 25 13:23:45.959989 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.959868 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879"} err="failed to get container status \"d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879\": rpc error: code = NotFound desc = could not find container \"d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879\": container with ID starting with d660b3e2902a3cd7cee3be5b790c66b5b0aaa2eee6a261bb8083a0f976433879 not found: ID does not exist" May 25 13:23:45.962451 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:45.962432 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a698c-predictor-68889dbfc-jrwp6"] May 25 13:23:46.932336 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:46.932308 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" path="/var/lib/kubelet/pods/7a91b4c6-8d9a-455e-91ee-07693f039fda/volumes" May 25 13:23:46.940605 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:46.940580 2577 generic.go:358] "Generic (PLEG): container finished" podID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerID="7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072" exitCode=0 May 25 13:23:46.940709 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:46.940647 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerDied","Data":"7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072"} May 25 13:23:47.946055 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:47.946019 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerStarted","Data":"21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5"} May 25 13:23:47.946055 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:47.946060 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerStarted","Data":"2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb"} May 25 13:23:47.946462 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:47.946257 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:47.966230 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:47.966181 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podStartSLOduration=6.9661652069999995 podStartE2EDuration="6.966165207s" podCreationTimestamp="2026-05-25 13:23:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:23:47.965327343 +0000 UTC m=+1081.542837727" watchObservedRunningTime="2026-05-25 13:23:47.966165207 +0000 UTC m=+1081.543675631" May 25 13:23:48.948827 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:48.948800 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:48.949826 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:48.949799 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:23:49.951399 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:49.951357 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:23:54.955458 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:54.955429 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:23:54.956029 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:23:54.956001 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:24:04.955941 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:24:04.955900 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:24:14.956965 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:24:14.956915 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:24:24.956017 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:24:24.955979 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:24:34.956488 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:24:34.956453 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:24:44.956182 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:24:44.956145 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:24:54.956658 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:24:54.956626 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:25:22.137730 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.137694 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj"] May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138036 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kube-rbac-proxy" May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138051 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kube-rbac-proxy" May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138065 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138072 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138083 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="storage-initializer" May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138088 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="storage-initializer" May 25 13:25:22.138144 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138144 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kube-rbac-proxy" May 25 13:25:22.138352 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.138152 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="7a91b4c6-8d9a-455e-91ee-07693f039fda" containerName="kserve-container" May 25 13:25:22.141081 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.141063 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.143479 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.143457 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-df494-predictor-serving-cert\"" May 25 13:25:22.143612 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.143531 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-df494-kube-rbac-proxy-sar-config\"" May 25 13:25:22.153912 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.153886 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj"] May 25 13:25:22.176470 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.176445 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt"] May 25 13:25:22.176791 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.176767 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" containerID="cri-o://2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb" gracePeriod=30 May 25 13:25:22.176913 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.176795 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kube-rbac-proxy" containerID="cri-o://21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5" gracePeriod=30 May 25 13:25:22.262715 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.262683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"message-dumper-raw-df494-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/668932ef-f540-4fdf-800c-13916f31dea5-message-dumper-raw-df494-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.262855 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.262734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp6mw\" (UniqueName: \"kubernetes.io/projected/668932ef-f540-4fdf-800c-13916f31dea5-kube-api-access-vp6mw\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.262855 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.262754 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.363838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.363803 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vp6mw\" (UniqueName: \"kubernetes.io/projected/668932ef-f540-4fdf-800c-13916f31dea5-kube-api-access-vp6mw\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.363838 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.363842 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.364045 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.363924 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"message-dumper-raw-df494-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/668932ef-f540-4fdf-800c-13916f31dea5-message-dumper-raw-df494-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.364045 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:25:22.364038 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/message-dumper-raw-df494-predictor-serving-cert: secret "message-dumper-raw-df494-predictor-serving-cert" not found May 25 13:25:22.364122 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:25:22.364115 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls podName:668932ef-f540-4fdf-800c-13916f31dea5 nodeName:}" failed. No retries permitted until 2026-05-25 13:25:22.864095163 +0000 UTC m=+1176.441605531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls") pod "message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" (UID: "668932ef-f540-4fdf-800c-13916f31dea5") : secret "message-dumper-raw-df494-predictor-serving-cert" not found May 25 13:25:22.364603 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.364585 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"message-dumper-raw-df494-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/668932ef-f540-4fdf-800c-13916f31dea5-message-dumper-raw-df494-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.373053 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.373028 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp6mw\" (UniqueName: \"kubernetes.io/projected/668932ef-f540-4fdf-800c-13916f31dea5-kube-api-access-vp6mw\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.868044 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.868000 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:22.870477 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:22.870445 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls\") pod \"message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:23.051252 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:23.051220 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:23.178723 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:23.178690 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj"] May 25 13:25:23.181838 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:25:23.181800 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668932ef_f540_4fdf_800c_13916f31dea5.slice/crio-c0daada5d10ee6d8e1f850dae9d5ccd47ccaee1383502207cc5000a43ba687ea WatchSource:0}: Error finding container c0daada5d10ee6d8e1f850dae9d5ccd47ccaee1383502207cc5000a43ba687ea: Status 404 returned error can't find the container with id c0daada5d10ee6d8e1f850dae9d5ccd47ccaee1383502207cc5000a43ba687ea May 25 13:25:23.206351 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:23.206328 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" event={"ID":"668932ef-f540-4fdf-800c-13916f31dea5","Type":"ContainerStarted","Data":"c0daada5d10ee6d8e1f850dae9d5ccd47ccaee1383502207cc5000a43ba687ea"} May 25 13:25:23.208090 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:23.208069 2577 generic.go:358] "Generic (PLEG): container finished" podID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerID="21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5" exitCode=2 May 25 13:25:23.208186 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:23.208123 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerDied","Data":"21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5"} May 25 13:25:24.952582 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:24.952540 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused" May 25 13:25:24.956854 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:24.956835 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" May 25 13:25:25.214745 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.214713 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" event={"ID":"668932ef-f540-4fdf-800c-13916f31dea5","Type":"ContainerStarted","Data":"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c"} May 25 13:25:25.214925 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.214751 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" event={"ID":"668932ef-f540-4fdf-800c-13916f31dea5","Type":"ContainerStarted","Data":"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923"} May 25 13:25:25.214925 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.214839 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:25.234637 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.234591 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" podStartSLOduration=2.100254989 podStartE2EDuration="3.234577313s" podCreationTimestamp="2026-05-25 13:25:22 +0000 UTC" firstStartedPulling="2026-05-25 13:25:23.18364257 +0000 UTC m=+1176.761152932" lastFinishedPulling="2026-05-25 13:25:24.317964885 +0000 UTC m=+1177.895475256" observedRunningTime="2026-05-25 13:25:25.233976983 +0000 UTC m=+1178.811487367" watchObservedRunningTime="2026-05-25 13:25:25.234577313 +0000 UTC m=+1178.812087755" May 25 13:25:25.622020 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.621998 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:25:25.791528 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.791451 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clfhm\" (UniqueName: \"kubernetes.io/projected/216efa27-e914-40d4-bd1a-dc6bf677e801-kube-api-access-clfhm\") pod \"216efa27-e914-40d4-bd1a-dc6bf677e801\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " May 25 13:25:25.791528 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.791503 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/216efa27-e914-40d4-bd1a-dc6bf677e801-kserve-provision-location\") pod \"216efa27-e914-40d4-bd1a-dc6bf677e801\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " May 25 13:25:25.791725 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.791547 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/216efa27-e914-40d4-bd1a-dc6bf677e801-isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\") pod \"216efa27-e914-40d4-bd1a-dc6bf677e801\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " May 25 13:25:25.791725 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.791597 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/216efa27-e914-40d4-bd1a-dc6bf677e801-proxy-tls\") pod \"216efa27-e914-40d4-bd1a-dc6bf677e801\" (UID: \"216efa27-e914-40d4-bd1a-dc6bf677e801\") " May 25 13:25:25.791894 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.791851 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/216efa27-e914-40d4-bd1a-dc6bf677e801-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "216efa27-e914-40d4-bd1a-dc6bf677e801" (UID: "216efa27-e914-40d4-bd1a-dc6bf677e801"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 25 13:25:25.791966 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.791899 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/216efa27-e914-40d4-bd1a-dc6bf677e801-isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config") pod "216efa27-e914-40d4-bd1a-dc6bf677e801" (UID: "216efa27-e914-40d4-bd1a-dc6bf677e801"). InnerVolumeSpecName "isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 25 13:25:25.793584 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.793560 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/216efa27-e914-40d4-bd1a-dc6bf677e801-kube-api-access-clfhm" (OuterVolumeSpecName: "kube-api-access-clfhm") pod "216efa27-e914-40d4-bd1a-dc6bf677e801" (UID: "216efa27-e914-40d4-bd1a-dc6bf677e801"). InnerVolumeSpecName "kube-api-access-clfhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 25 13:25:25.793645 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.793585 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/216efa27-e914-40d4-bd1a-dc6bf677e801-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "216efa27-e914-40d4-bd1a-dc6bf677e801" (UID: "216efa27-e914-40d4-bd1a-dc6bf677e801"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 25 13:25:25.892818 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.892787 2577 reconciler_common.go:299] "Volume detached for volume \"isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/216efa27-e914-40d4-bd1a-dc6bf677e801-isvc-xgboost-graph-raw-hpa-12a34-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:25:25.892818 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.892813 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/216efa27-e914-40d4-bd1a-dc6bf677e801-proxy-tls\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:25:25.893069 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.892825 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfhm\" (UniqueName: \"kubernetes.io/projected/216efa27-e914-40d4-bd1a-dc6bf677e801-kube-api-access-clfhm\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:25:25.893069 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:25.892835 2577 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/216efa27-e914-40d4-bd1a-dc6bf677e801-kserve-provision-location\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:25:26.220266 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.220237 2577 generic.go:358] "Generic (PLEG): container finished" podID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerID="2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb" exitCode=0 May 25 13:25:26.220691 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.220303 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerDied","Data":"2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb"} May 25 13:25:26.220691 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.220316 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" May 25 13:25:26.220691 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.220340 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt" event={"ID":"216efa27-e914-40d4-bd1a-dc6bf677e801","Type":"ContainerDied","Data":"ce2c67255481b41a5c5e5e3ba3d1eb2150fe0860be715beba1c5cde668701fc6"} May 25 13:25:26.220691 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.220361 2577 scope.go:117] "RemoveContainer" containerID="21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5" May 25 13:25:26.220905 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.220892 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:26.222625 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.222605 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:25:26.230063 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.230048 2577 scope.go:117] "RemoveContainer" containerID="2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb" May 25 13:25:26.236550 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.236530 2577 scope.go:117] "RemoveContainer" containerID="7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072" May 25 13:25:26.242855 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.242840 2577 scope.go:117] "RemoveContainer" containerID="21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5" May 25 13:25:26.243121 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:25:26.243103 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5\": container with ID starting with 21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5 not found: ID does not exist" containerID="21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5" May 25 13:25:26.243178 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.243129 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5"} err="failed to get container status \"21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5\": rpc error: code = NotFound desc = could not find container \"21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5\": container with ID starting with 21899de283ed2232b8764641f1b5ca935bf8f924dec3c8268aa6a3dddbe0aae5 not found: ID does not exist" May 25 13:25:26.243178 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.243145 2577 scope.go:117] "RemoveContainer" containerID="2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb" May 25 13:25:26.243445 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:25:26.243408 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb\": container with ID starting with 2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb not found: ID does not exist" containerID="2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb" May 25 13:25:26.243445 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.243440 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb"} err="failed to get container status \"2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb\": rpc error: code = NotFound desc = could not find container \"2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb\": container with ID starting with 2789f66594d05cc72103056024e8d3eb7363329ec9bd67e42bcb36c733c36aeb not found: ID does not exist" May 25 13:25:26.243618 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.243456 2577 scope.go:117] "RemoveContainer" containerID="7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072" May 25 13:25:26.243719 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:25:26.243696 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072\": container with ID starting with 7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072 not found: ID does not exist" containerID="7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072" May 25 13:25:26.243763 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.243728 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072"} err="failed to get container status \"7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072\": rpc error: code = NotFound desc = could not find container \"7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072\": container with ID starting with 7a5f3bfaaef4153d82734e51333ab31c6237816d9548c8774b84b47bc2f2c072 not found: ID does not exist" May 25 13:25:26.257775 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.257755 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt"] May 25 13:25:26.264605 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.264584 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-12a34-predictor-6c6cf8d697-r7sqt"] May 25 13:25:26.931962 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:26.931932 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" path="/var/lib/kubelet/pods/216efa27-e914-40d4-bd1a-dc6bf677e801/volumes" May 25 13:25:33.230627 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:25:33.230594 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:27:07.184527 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.184494 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj_668932ef-f540-4fdf-800c-13916f31dea5/kserve-container/0.log" May 25 13:27:07.486128 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.486093 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj"] May 25 13:27:07.486412 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.486383 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kserve-container" containerID="cri-o://90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923" gracePeriod=30 May 25 13:27:07.486477 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.486409 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kube-rbac-proxy" containerID="cri-o://e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c" gracePeriod=30 May 25 13:27:07.728763 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.728738 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:27:07.749595 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.749532 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"message-dumper-raw-df494-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/668932ef-f540-4fdf-800c-13916f31dea5-message-dumper-raw-df494-kube-rbac-proxy-sar-config\") pod \"668932ef-f540-4fdf-800c-13916f31dea5\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " May 25 13:27:07.749595 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.749576 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls\") pod \"668932ef-f540-4fdf-800c-13916f31dea5\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " May 25 13:27:07.749759 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.749608 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp6mw\" (UniqueName: \"kubernetes.io/projected/668932ef-f540-4fdf-800c-13916f31dea5-kube-api-access-vp6mw\") pod \"668932ef-f540-4fdf-800c-13916f31dea5\" (UID: \"668932ef-f540-4fdf-800c-13916f31dea5\") " May 25 13:27:07.749913 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.749870 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/668932ef-f540-4fdf-800c-13916f31dea5-message-dumper-raw-df494-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "message-dumper-raw-df494-kube-rbac-proxy-sar-config") pod "668932ef-f540-4fdf-800c-13916f31dea5" (UID: "668932ef-f540-4fdf-800c-13916f31dea5"). InnerVolumeSpecName "message-dumper-raw-df494-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 25 13:27:07.751770 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.751745 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/668932ef-f540-4fdf-800c-13916f31dea5-kube-api-access-vp6mw" (OuterVolumeSpecName: "kube-api-access-vp6mw") pod "668932ef-f540-4fdf-800c-13916f31dea5" (UID: "668932ef-f540-4fdf-800c-13916f31dea5"). InnerVolumeSpecName "kube-api-access-vp6mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 25 13:27:07.751770 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.751757 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "668932ef-f540-4fdf-800c-13916f31dea5" (UID: "668932ef-f540-4fdf-800c-13916f31dea5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 25 13:27:07.850234 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.850200 2577 reconciler_common.go:299] "Volume detached for volume \"message-dumper-raw-df494-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/668932ef-f540-4fdf-800c-13916f31dea5-message-dumper-raw-df494-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:27:07.850234 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.850231 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/668932ef-f540-4fdf-800c-13916f31dea5-proxy-tls\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:27:07.850234 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:07.850241 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp6mw\" (UniqueName: \"kubernetes.io/projected/668932ef-f540-4fdf-800c-13916f31dea5-kube-api-access-vp6mw\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 25 13:27:08.509450 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509418 2577 generic.go:358] "Generic (PLEG): container finished" podID="668932ef-f540-4fdf-800c-13916f31dea5" containerID="e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c" exitCode=2 May 25 13:27:08.509450 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509444 2577 generic.go:358] "Generic (PLEG): container finished" podID="668932ef-f540-4fdf-800c-13916f31dea5" containerID="90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923" exitCode=2 May 25 13:27:08.509971 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509478 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" event={"ID":"668932ef-f540-4fdf-800c-13916f31dea5","Type":"ContainerDied","Data":"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c"} May 25 13:27:08.509971 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509495 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" May 25 13:27:08.509971 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509505 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" event={"ID":"668932ef-f540-4fdf-800c-13916f31dea5","Type":"ContainerDied","Data":"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923"} May 25 13:27:08.509971 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509516 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj" event={"ID":"668932ef-f540-4fdf-800c-13916f31dea5","Type":"ContainerDied","Data":"c0daada5d10ee6d8e1f850dae9d5ccd47ccaee1383502207cc5000a43ba687ea"} May 25 13:27:08.509971 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.509529 2577 scope.go:117] "RemoveContainer" containerID="e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c" May 25 13:27:08.518042 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.518020 2577 scope.go:117] "RemoveContainer" containerID="90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923" May 25 13:27:08.524479 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.524462 2577 scope.go:117] "RemoveContainer" containerID="e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c" May 25 13:27:08.524719 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:27:08.524699 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c\": container with ID starting with e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c not found: ID does not exist" containerID="e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c" May 25 13:27:08.524762 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.524729 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c"} err="failed to get container status \"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c\": rpc error: code = NotFound desc = could not find container \"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c\": container with ID starting with e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c not found: ID does not exist" May 25 13:27:08.524762 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.524746 2577 scope.go:117] "RemoveContainer" containerID="90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923" May 25 13:27:08.525015 ip-10-0-142-251 kubenswrapper[2577]: E0525 13:27:08.524997 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923\": container with ID starting with 90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923 not found: ID does not exist" containerID="90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923" May 25 13:27:08.525061 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.525023 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923"} err="failed to get container status \"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923\": rpc error: code = NotFound desc = could not find container \"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923\": container with ID starting with 90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923 not found: ID does not exist" May 25 13:27:08.525061 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.525039 2577 scope.go:117] "RemoveContainer" containerID="e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c" May 25 13:27:08.525271 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.525251 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c"} err="failed to get container status \"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c\": rpc error: code = NotFound desc = could not find container \"e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c\": container with ID starting with e675bb87cd5f8774d5d2a57bfb8dcfe84c286b3d7e04bf5369e59d26bf25710c not found: ID does not exist" May 25 13:27:08.525325 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.525271 2577 scope.go:117] "RemoveContainer" containerID="90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923" May 25 13:27:08.525468 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.525454 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923"} err="failed to get container status \"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923\": rpc error: code = NotFound desc = could not find container \"90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923\": container with ID starting with 90dbef31cd6bb7e1a1ea25f899c81a3ab42ce70117292c81e9120a6097df2923 not found: ID does not exist" May 25 13:27:08.530848 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.530829 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj"] May 25 13:27:08.537219 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.537147 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-df494-predictor-9ff7bb8f9-79nhj"] May 25 13:27:08.932902 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:27:08.932793 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="668932ef-f540-4fdf-800c-13916f31dea5" path="/var/lib/kubelet/pods/668932ef-f540-4fdf-800c-13916f31dea5/volumes" May 25 13:34:21.758494 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:21.758465 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ldn2b_8f84c632-f831-43da-ab7b-e72fc9dd56f9/global-pull-secret-syncer/0.log" May 25 13:34:21.839382 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:21.839356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ddjsl_05a827a6-4e72-4283-8967-2c44c405e0c9/konnectivity-agent/0.log" May 25 13:34:21.936954 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:21.936921 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-251.ec2.internal_e8c054cb9a59ce3874e2199aacfd5039/haproxy/0.log" May 25 13:34:25.545223 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.545189 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5665d77bc9-xcxg9_a3c31591-98e0-4e2c-a904-17c3f46b5fe5/metrics-server/0.log" May 25 13:34:25.759788 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.759761 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xgxfb_c4635f82-c644-43b0-b698-77668c27c84f/node-exporter/0.log" May 25 13:34:25.780753 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.780727 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xgxfb_c4635f82-c644-43b0-b698-77668c27c84f/kube-rbac-proxy/0.log" May 25 13:34:25.807954 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.807900 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xgxfb_c4635f82-c644-43b0-b698-77668c27c84f/init-textfile/0.log" May 25 13:34:25.834104 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.834080 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-9h6f8_1f3849a0-f196-4593-9ee6-1a36bf58646d/kube-rbac-proxy-main/0.log" May 25 13:34:25.856258 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.856236 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-9h6f8_1f3849a0-f196-4593-9ee6-1a36bf58646d/kube-rbac-proxy-self/0.log" May 25 13:34:25.878721 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:25.878700 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-9h6f8_1f3849a0-f196-4593-9ee6-1a36bf58646d/openshift-state-metrics/0.log" May 25 13:34:28.676928 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.676898 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6"] May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677182 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kserve-container" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677194 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kserve-container" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677206 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="storage-initializer" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677211 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="storage-initializer" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677219 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kube-rbac-proxy" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677226 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kube-rbac-proxy" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677236 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677241 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677248 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kube-rbac-proxy" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677253 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kube-rbac-proxy" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677292 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kserve-container" May 25 13:34:28.677298 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677299 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kserve-container" May 25 13:34:28.677628 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677306 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="216efa27-e914-40d4-bd1a-dc6bf677e801" containerName="kube-rbac-proxy" May 25 13:34:28.677628 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.677312 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="668932ef-f540-4fdf-800c-13916f31dea5" containerName="kube-rbac-proxy" May 25 13:34:28.680030 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.680014 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.683027 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.683007 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kt9b5\"/\"openshift-service-ca.crt\"" May 25 13:34:28.684067 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.684051 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kt9b5\"/\"default-dockercfg-hkc44\"" May 25 13:34:28.684189 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.684074 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kt9b5\"/\"kube-root-ca.crt\"" May 25 13:34:28.688509 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.688485 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6"] May 25 13:34:28.750579 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.750549 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-proc\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.750715 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.750596 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7kkz\" (UniqueName: \"kubernetes.io/projected/5e806b4a-40f6-4f82-808a-a48285b10606-kube-api-access-h7kkz\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.750715 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.750690 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-podres\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.750805 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.750725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-sys\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.750842 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.750806 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-lib-modules\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851542 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851517 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h7kkz\" (UniqueName: \"kubernetes.io/projected/5e806b4a-40f6-4f82-808a-a48285b10606-kube-api-access-h7kkz\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851643 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851552 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-podres\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851643 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851572 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-sys\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851643 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851604 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-lib-modules\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851645 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-proc\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851655 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-sys\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851699 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-podres\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851739 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-proc\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.851783 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.851757 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e806b4a-40f6-4f82-808a-a48285b10606-lib-modules\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.859365 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.859341 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7kkz\" (UniqueName: \"kubernetes.io/projected/5e806b4a-40f6-4f82-808a-a48285b10606-kube-api-access-h7kkz\") pod \"perf-node-gather-daemonset-7dxn6\" (UID: \"5e806b4a-40f6-4f82-808a-a48285b10606\") " pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:28.990336 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:28.990315 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:29.111664 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.111637 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6"] May 25 13:34:29.114131 ip-10-0-142-251 kubenswrapper[2577]: W0525 13:34:29.114100 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5e806b4a_40f6_4f82_808a_a48285b10606.slice/crio-7aca4888905fc682d791b991adb465f218a7993787edcb45c1892a1e921b302b WatchSource:0}: Error finding container 7aca4888905fc682d791b991adb465f218a7993787edcb45c1892a1e921b302b: Status 404 returned error can't find the container with id 7aca4888905fc682d791b991adb465f218a7993787edcb45c1892a1e921b302b May 25 13:34:29.115600 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.115584 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 25 13:34:29.521175 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.521148 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pcmgs_52d3a7f9-dc96-4bf2-8039-95e9dc2a835c/dns/0.log" May 25 13:34:29.542065 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.542043 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pcmgs_52d3a7f9-dc96-4bf2-8039-95e9dc2a835c/kube-rbac-proxy/0.log" May 25 13:34:29.585738 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.585719 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sls6w_dc5fd3fa-97ed-4118-b25b-fe2d0332652b/dns-node-resolver/0.log" May 25 13:34:29.655307 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.655275 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" event={"ID":"5e806b4a-40f6-4f82-808a-a48285b10606","Type":"ContainerStarted","Data":"a283e7fa8255fa7dbc54bda1f46dc2637f41252ccf34bd9300818a2193de71ed"} May 25 13:34:29.655422 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.655311 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" event={"ID":"5e806b4a-40f6-4f82-808a-a48285b10606","Type":"ContainerStarted","Data":"7aca4888905fc682d791b991adb465f218a7993787edcb45c1892a1e921b302b"} May 25 13:34:29.655422 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.655397 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:29.674134 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:29.674070 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" podStartSLOduration=1.674054236 podStartE2EDuration="1.674054236s" podCreationTimestamp="2026-05-25 13:34:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:34:29.671728298 +0000 UTC m=+1723.249238681" watchObservedRunningTime="2026-05-25 13:34:29.674054236 +0000 UTC m=+1723.251564630" May 25 13:34:30.139065 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:30.139041 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mrcbj_19706064-0ad5-4245-93cf-960a91ff5e6d/node-ca/0.log" May 25 13:34:31.249517 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:31.249482 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-855w6_6cc87c4f-5424-448c-bc63-5e3b922eff11/serve-healthcheck-canary/0.log" May 25 13:34:31.639937 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:31.639842 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lcblp_cfe9ec39-cc14-4afd-b998-2a81061f0fe3/kube-rbac-proxy/0.log" May 25 13:34:31.660558 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:31.660536 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lcblp_cfe9ec39-cc14-4afd-b998-2a81061f0fe3/exporter/0.log" May 25 13:34:31.682526 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:31.682498 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lcblp_cfe9ec39-cc14-4afd-b998-2a81061f0fe3/extractor/0.log" May 25 13:34:33.821019 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:33.820988 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7cd9b78f5d-vpzx2_9c1926e3-1a2d-405e-8d80-2e9ac491b607/manager/0.log" May 25 13:34:33.863005 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:33.862978 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-86f7b4b499-tqc6p_ea79699a-3ea9-486d-81cb-af35171e349f/server/0.log" May 25 13:34:33.996279 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:33.996250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-79f457656b-fd598_4b84f575-535e-442a-a464-123d304d93a6/seaweedfs/0.log" May 25 13:34:35.667260 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:35.667229 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kt9b5/perf-node-gather-daemonset-7dxn6" May 25 13:34:39.185101 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.185068 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4pd99_3dcb6605-7c8a-4f0e-81d0-92c9db492cb4/kube-multus/0.log" May 25 13:34:39.574652 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.574629 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/kube-multus-additional-cni-plugins/0.log" May 25 13:34:39.601373 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.601344 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/egress-router-binary-copy/0.log" May 25 13:34:39.627949 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.627928 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/cni-plugins/0.log" May 25 13:34:39.654383 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.654364 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/bond-cni-plugin/0.log" May 25 13:34:39.679554 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.679535 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/routeoverride-cni/0.log" May 25 13:34:39.703347 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.703326 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/whereabouts-cni-bincopy/0.log" May 25 13:34:39.728538 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.728497 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-plnkx_bcaff31e-f7bd-4028-8e50-38894ee8c24f/whereabouts-cni/0.log" May 25 13:34:39.884420 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.884369 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44458_7088f44e-edb0-4b89-89dc-da82bc7e466a/network-metrics-daemon/0.log" May 25 13:34:39.905692 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:39.905668 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44458_7088f44e-edb0-4b89-89dc-da82bc7e466a/kube-rbac-proxy/0.log" May 25 13:34:40.672640 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.672608 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/ovn-controller/0.log" May 25 13:34:40.703499 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.703473 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/ovn-acl-logging/0.log" May 25 13:34:40.724133 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.724113 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/kube-rbac-proxy-node/0.log" May 25 13:34:40.750821 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.750798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/kube-rbac-proxy-ovn-metrics/0.log" May 25 13:34:40.773029 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.773007 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/northd/0.log" May 25 13:34:40.802261 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.802242 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/nbdb/0.log" May 25 13:34:40.824996 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.824980 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/sbdb/0.log" May 25 13:34:40.927050 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:40.926979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rg2g_916f48b4-46b7-4562-b64c-f62903759bd3/ovnkube-controller/0.log" May 25 13:34:42.604816 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:42.604782 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-hb8pp_79ece060-ccbd-4c01-b8a1-64e7856a091e/network-check-target-container/0.log" May 25 13:34:43.628554 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:43.628526 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lm9jv_36393f8d-8a92-4e03-9340-4623cf2d2177/iptables-alerter/0.log" May 25 13:34:44.364524 ip-10-0-142-251 kubenswrapper[2577]: I0525 13:34:44.364447 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-zfm5s_6cc33bae-9021-427e-a689-a8fffd1378bf/tuned/0.log"