Jun 02 15:44:00.571346 ip-10-0-142-13 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jun 02 15:44:00.571356 ip-10-0-142-13 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jun 02 15:44:00.571365 ip-10-0-142-13 systemd[1]: kubelet.service: Failed with result 'resources'.
Jun 02 15:44:00.571652 ip-10-0-142-13 systemd[1]: Failed to start Kubernetes Kubelet.
Jun 02 15:44:11.798492 ip-10-0-142-13 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jun 02 15:44:11.798512 ip-10-0-142-13 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4524a87439ee4f3893fe0dba9bc729fa --
Jun 02 15:46:34.000287 ip-10-0-142-13 systemd[1]: Starting Kubernetes Kubelet...
Jun 02 15:46:34.363207 ip-10-0-142-13 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 15:46:34.363207 ip-10-0-142-13 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 02 15:46:34.363207 ip-10-0-142-13 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 15:46:34.363207 ip-10-0-142-13 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 02 15:46:34.363207 ip-10-0-142-13 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 15:46:34.364596 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.364505    2562 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 02 15:46:34.368344 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368326    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:34.368344 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368344    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368347    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368350    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368355    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368360    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368363    2562 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368366    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368369    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368372    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368375    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368377    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368380    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368382    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368385    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368388    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368390    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368393    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368395    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368399    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368402    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:34.368419 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368405    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368408    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368410    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368413    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368415    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368418    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368421    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368424    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368427    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368430    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368433    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368436    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368438    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368441    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368443    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368446    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368449    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368452    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368459    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:34.368935 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368465    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368471    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368476    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368480    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368484    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368488    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368492    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368494    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368497    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368500    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368502    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368505    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368507    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368510    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368512    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368516    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368519    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368522    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368524    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:34.369426 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368527    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368529    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368532    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368535    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368537    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368540    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368543    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368548    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368551    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368553    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368555    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368558    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368560    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368563    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368565    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368568    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368570    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368573    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368575    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368577    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:34.369936 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368580    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368582    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368585    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368587    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368590    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368593    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.368595    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369010    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369016    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369020    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369022    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369025    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369028    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369031    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369034    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369036    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369039    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369041    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369044    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369047    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:34.370432 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369050    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369052    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369055    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369057    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369060    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369062    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369065    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369067    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369070    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369072    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369075    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369078    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369080    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369083    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369085    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369088    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369090    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369092    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369095    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369098    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:34.370927 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369101    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369104    2562 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369108    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369112    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369115    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369118    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369121    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369124    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369127    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369130    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369133    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369136    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369140    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369143    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369147    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369150    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369152    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369155    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369157    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:34.371437 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369160    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369162    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369165    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369167    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369170    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369172    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369175    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369177    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369180    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369182    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369185    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369188    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369190    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369194    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369197    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369199    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369202    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369204    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369207    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369210    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:34.371921 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369213    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369216    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369218    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369221    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369223    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369226    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369230    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369233    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369235    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369238    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369240    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369243    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369245    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.369247    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369330    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369338    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369345    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369349    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369354    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369357    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369361    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 02 15:46:34.372417 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369366    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369369    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369372    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369376    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369380    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369384    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369387    2562 flags.go:64] FLAG: --cgroup-root=""
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369390    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369393    2562 flags.go:64] FLAG: --client-ca-file=""
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369395    2562 flags.go:64] FLAG: --cloud-config=""
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369398    2562 flags.go:64] FLAG: --cloud-provider="external"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.369401    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370745    2562 flags.go:64] FLAG: --cluster-domain=""
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370748    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370752    2562 flags.go:64] FLAG: --config-dir=""
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370755    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370759    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370763    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370771    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370775    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370778    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370781    2562 flags.go:64] FLAG: --contention-profiling="false"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370784    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370787    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370790    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 02 15:46:34.372943 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370793    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370798    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370801    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370804    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370807    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370810    2562 flags.go:64] FLAG: --enable-server="true"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370813    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370818    2562 flags.go:64] FLAG: --event-burst="100"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370821    2562 flags.go:64] FLAG: --event-qps="50"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370825    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370828    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370832    2562 flags.go:64] FLAG: --eviction-hard=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370836    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370839    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370842    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370845    2562 flags.go:64] FLAG: --eviction-soft=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370848    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370850    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370853    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370856    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370859    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370862    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370865    2562 flags.go:64] FLAG: --feature-gates=""
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370869    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370872    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 02 15:46:34.373545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370875    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370879    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370882    2562 flags.go:64] FLAG: --healthz-port="10248"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370885    2562 flags.go:64] FLAG: --help="false"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370888    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370891    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370894    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370897    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370901    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370904    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370918    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370921    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370924    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370927    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370930    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370933    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370937    2562 flags.go:64] FLAG: --kube-reserved=""
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370941    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370944    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370947    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370950    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370953    2562 flags.go:64] FLAG: --lock-file=""
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370956    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370959    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 02 15:46:34.374168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370962    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370967    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370970    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370973    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370976    2562 flags.go:64] FLAG: --logging-format="text"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370978    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370982    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370985    2562 flags.go:64] FLAG: --manifest-url=""
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370988    2562 flags.go:64] FLAG: --manifest-url-header=""
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370992    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370995    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.370999    2562 flags.go:64] FLAG: --max-pods="110"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371002    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371005    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371008    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371011    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371014    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371016    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371020    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371027    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371031    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371034    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371037    2562 flags.go:64] FLAG: --pod-cidr=""
Jun 02 15:46:34.374851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371040    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371046    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371050    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371053    2562 flags.go:64] FLAG: --pods-per-core="0"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371056    2562 flags.go:64] FLAG: --port="10250"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371059    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371062    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-011f837544826403d"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371066    2562 flags.go:64] FLAG: --qos-reserved=""
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371068    2562 flags.go:64] FLAG: --read-only-port="10255"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371072    2562 flags.go:64] FLAG: --register-node="true"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371075    2562 flags.go:64] FLAG: --register-schedulable="true"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371077    2562 flags.go:64] FLAG: --register-with-taints=""
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371081    2562 flags.go:64] FLAG: --registry-burst="10"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371084    2562 flags.go:64] FLAG: --registry-qps="5"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371087    2562 flags.go:64] FLAG: --reserved-cpus=""
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371090    2562 flags.go:64] FLAG: --reserved-memory=""
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371093    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371096    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371099    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371102    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371105    2562 flags.go:64] FLAG: --runonce="false"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371108    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371111    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371114    2562 flags.go:64] FLAG: --seccomp-default="false"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371118    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371121    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 02 15:46:34.375430 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371124    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371127    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371130    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371132    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371136    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371139    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371141    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371145    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371148    2562 flags.go:64] FLAG: --system-cgroups=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371151    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371168    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371172    2562 flags.go:64] FLAG: --tls-cert-file=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371175    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371179    2562 flags.go:64] FLAG: --tls-min-version=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371182    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371185    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371188    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371191    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371194    2562 flags.go:64] FLAG: --v="2"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371198    2562 flags.go:64] FLAG: --version="false"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371202    2562 flags.go:64] FLAG: --vmodule=""
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371212    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.371215    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371325    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:34.376068 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371329    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371333    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371336    2562 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371340    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371343    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371345    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371348    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371351    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371355    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371357    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371360    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371362    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371365    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371367    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371370    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371372    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371375    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371377    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371380    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371382    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:34.376646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371385    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371387    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371390    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371392    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371395    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371397    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371400    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371402    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371405    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371407    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371410    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371412    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371415    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371417    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371420    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371422    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371425    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371427    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371430    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371433    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:34.377201 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371435    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371438    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371440    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371443    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371446    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371448    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371451    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371453    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371456    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371458    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371463    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371465    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371468    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371470    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371473    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371475    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371478    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371480    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371483    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371486    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:34.377692 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371488    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371490    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371493    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371496    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371498    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371501    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371504    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371506    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371509    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371512    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371514    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371517    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371519    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371522    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371524    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371528    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371531    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371535    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371539    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:34.378184 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371542    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:34.378698 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371545    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:34.378698 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371548    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:34.378698 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371552    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:34.378698 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371555    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:34.378698 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.371558    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:34.378698 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.372145    2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 15:46:34.378941 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.378922    2562 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
Jun 02 15:46:34.378980 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.378942    2562 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 02 15:46:34.379007 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.378990    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:34.379007 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.378996    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:34.379007 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.378999    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:34.379007 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379003    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:34.379007 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379006    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:34.379007 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379009    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379012    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379015    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379018    2562 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379021    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379024    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379027    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379030    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379032    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379035    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379038    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379040    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379043    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379045    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379048    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379051    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379053    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379055    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379058    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379061    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:34.379163 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379063    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379067    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379069    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379072    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379074    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379077    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379080    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379083    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379085    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379089    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379110    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379113    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379116    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379120    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379123    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379125    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379129    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379131    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379134    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379136    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:34.379661 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379139    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379141    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379146    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379150    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379160    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379165    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379168    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379171    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379174    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379177    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379179    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379182    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379184    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379187    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379190    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379192    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379195    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379198    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379200    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:34.380153 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379204    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379206    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379209    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379212    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379215    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379217    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379220    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379223    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379226    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379229    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379232    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379234    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379237    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379239    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379242    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379245    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379247    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379250    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379253    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379255    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:34.380618 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379257    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379260    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.379265    2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379366    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379370    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379373    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379377    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379380    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379383    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379385    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379388    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379391    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379393    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379397    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379399    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:34.381126 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379402    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379404    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379407    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379410    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379412    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379415    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379417    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379420    2562 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379423    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379425    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379428    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379430    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379433    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379436    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379438    2562 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379441    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379443    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379446    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379448    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379451    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:34.381504 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379454    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379456    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379458    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379462    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379466    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379469    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379472    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379474    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379477    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379479    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379482    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379485    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379488    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379491    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379493    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379496    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379498    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379503    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379506    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:34.382076 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379509    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379512    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379515    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379517    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379520    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379523    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379526    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379528    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379531    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379534    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379536    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379539    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379542    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379544    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379547    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379549    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379552    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379555    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379557    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379560    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:34.382816 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379562    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379565    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379568    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379570    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379573    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379576    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379578    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379581    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379584    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379586    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379589    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379591    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379594    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379596    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:34.379599    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:34.383441 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.379604    2562 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 15:46:34.383819 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.380515    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 02 15:46:34.383819 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.383127    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 02 15:46:34.384345 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.384331    2562 server.go:1019] "Starting client certificate rotation"
Jun 02 15:46:34.384444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.384429    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 15:46:34.384475 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.384469    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 15:46:34.406878 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.406858    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 15:46:34.408055 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.408037    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 15:46:34.419941 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.419923    2562 log.go:25] "Validated CRI v1 runtime API"
Jun 02 15:46:34.426231 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.426216    2562 log.go:25] "Validated CRI v1 image API"
Jun 02 15:46:34.427344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.427329    2562 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 02 15:46:34.429405 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.429386    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e1867588-4fa0-4ebd-92ae-1968ba164339:/dev/nvme0n1p4 e676028a-67c6-4980-b4f6-da41bccf372f:/dev/nvme0n1p3]
Jun 02 15:46:34.429476 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.429405    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 02 15:46:34.434632 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.434240    2562 manager.go:217] Machine: {Timestamp:2026-06-02 15:46:34.433435374 +0000 UTC m=+0.331155753 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100272 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24041845a8d12023d79fdb72a7943b SystemUUID:ec240418-45a8-d120-23d7-9fdb72a7943b BootID:4524a874-39ee-4f38-93fe-0dba9bc729fa Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8d:e5:c7:95:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8d:e5:c7:95:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:f6:34:42:6f:3f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 02 15:46:34.435414 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.435400    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 02 15:46:34.435518 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.435504    2562 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 02 15:46:34.438515 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.438489    2562 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 02 15:46:34.438679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.438518    2562 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-13.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 02 15:46:34.438727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.438690    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jun 02 15:46:34.438727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.438698    2562 container_manager_linux.go:306] "Creating device plugin manager"
Jun 02 15:46:34.438727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.438706    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 15:46:34.439326 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.439315    2562 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 15:46:34.440664 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.440653    2562 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 15:46:34.440774 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.440765    2562 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Jun 02 15:46:34.440803 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.440783    2562 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 15:46:34.442538 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.442528    2562 kubelet.go:491] "Attempting to sync node with API server"
Jun 02 15:46:34.442582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.442542    2562 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 02 15:46:34.442582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.442553    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 02 15:46:34.442582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.442562    2562 kubelet.go:397] "Adding apiserver pod source"
Jun 02 15:46:34.442582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.442569    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 02 15:46:34.443529 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.443517    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 15:46:34.443570 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.443535    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 15:46:34.445966 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.445952    2562 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
Jun 02 15:46:34.447653 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.447639    2562 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 02 15:46:34.449714 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449700    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 02 15:46:34.449714 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449718    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 02 15:46:34.449714 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449725    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449730    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449740    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449745    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449751    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449757    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449764    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449769    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449779    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 02 15:46:34.449851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.449784    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 02 15:46:34.450477 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.450461    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 02 15:46:34.450477 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.450473    2562 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 02 15:46:34.452054 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.452027    2562 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 02 15:46:34.452153 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.452030    2562 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 02 15:46:34.454074 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.454060    2562 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Jun 02 15:46:34.454138 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.454097    2562 server.go:1295] "Started kubelet"
Jun 02 15:46:34.454271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.454236    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 02 15:46:34.454716 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.454662    2562 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 02 15:46:34.454801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.454734    2562 server_v1.go:47] "podresources" method="list" useActivePods=true
Jun 02 15:46:34.455034 ip-10-0-142-13 systemd[1]: Started Kubernetes Kubelet.
Jun 02 15:46:34.455660 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.455643    2562 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 02 15:46:34.457267 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.457252    2562 server.go:317] "Adding debug handlers to kubelet server"
Jun 02 15:46:34.462100 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.462062    2562 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 02 15:46:34.462194 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.462091    2562 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-13.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 15:46:34.462734 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.462721    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 02 15:46:34.462905 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.462071    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-13.ec2.internal.18b54e2fb2249b65  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-13.ec2.internal,UID:ip-10-0-142-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-13.ec2.internal,},FirstTimestamp:2026-06-02 15:46:34.454072165 +0000 UTC m=+0.351792545,LastTimestamp:2026-06-02 15:46:34.454072165 +0000 UTC m=+0.351792545,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-13.ec2.internal,}"
Jun 02 15:46:34.463245 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.463229    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 02 15:46:34.463358 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.463339    2562 volume_manager.go:295] "The desired_state_of_world populator starts"
Jun 02 15:46:34.463429 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.463359    2562 volume_manager.go:297] "Starting Kubelet Volume Manager"
Jun 02 15:46:34.464163 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.463954    2562 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Jun 02 15:46:34.464336 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.464309    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:34.465148 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465036    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 02 15:46:34.465148 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465048    2562 factory.go:55] Registering systemd factory
Jun 02 15:46:34.465148 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465057    2562 factory.go:223] Registration of the systemd container factory successfully
Jun 02 15:46:34.465570 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465550    2562 factory.go:153] Registering CRI-O factory
Jun 02 15:46:34.465649 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465573    2562 factory.go:223] Registration of the crio container factory successfully
Jun 02 15:46:34.465649 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465593    2562 factory.go:103] Registering Raw factory
Jun 02 15:46:34.465649 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465607    2562 manager.go:1196] Started watching for new ooms in manager
Jun 02 15:46:34.465882 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465864    2562 reconstruct.go:97] "Volume reconstruction finished"
Jun 02 15:46:34.466019 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.465997    2562 reconciler.go:26] "Reconciler: start to sync state"
Jun 02 15:46:34.466254 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.466242    2562 manager.go:319] Starting recovery of all containers
Jun 02 15:46:34.469433 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.469391    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jun 02 15:46:34.469596 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.469574    2562 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jun 02 15:46:34.472719 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.472699    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wdq8f"
Jun 02 15:46:34.478941 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.478790    2562 manager.go:324] Recovery completed
Jun 02 15:46:34.480252 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.480229    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wdq8f"
Jun 02 15:46:34.483365 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.483351    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:34.485429 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.485412    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:34.485519 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.485440    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:34.485519 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.485451    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:34.485989 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.485977    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 02 15:46:34.485989 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.485989    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 02 15:46:34.486101 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.486018    2562 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 15:46:34.487251 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.487193    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-13.ec2.internal.18b54e2fb4030a41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-13.ec2.internal,UID:ip-10-0-142-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-13.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-13.ec2.internal,},FirstTimestamp:2026-06-02 15:46:34.485426753 +0000 UTC m=+0.383147134,LastTimestamp:2026-06-02 15:46:34.485426753 +0000 UTC m=+0.383147134,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-13.ec2.internal,}"
Jun 02 15:46:34.488126 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.488113    2562 policy_none.go:49] "None policy: Start"
Jun 02 15:46:34.488191 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.488134    2562 memory_manager.go:186] "Starting memorymanager" policy="None"
Jun 02 15:46:34.488191 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.488147    2562 state_mem.go:35] "Initializing new in-memory state store"
Jun 02 15:46:34.526610 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.526594    2562 manager.go:341] "Starting Device Plugin manager"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.526633    2562 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.526643    2562 server.go:85] "Starting device plugin registration server"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.526934    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.526947    2562 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.527051    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.527149    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.527158    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.527642    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 02 15:46:34.530940 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.527690    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:34.594710 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.594670    2562 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Jun 02 15:46:34.595818 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.595803    2562 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Jun 02 15:46:34.595902 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.595824    2562 status_manager.go:230] "Starting to sync pod status with apiserver"
Jun 02 15:46:34.595902 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.595847    2562 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 02 15:46:34.595902 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.595855    2562 kubelet.go:2451] "Starting kubelet main sync loop"
Jun 02 15:46:34.596051 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.595897    2562 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jun 02 15:46:34.598957 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.598939    2562 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:34.627229 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.627167    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:34.628079 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.628064    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:34.628152 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.628097    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:34.628152 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.628111    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:34.628152 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.628134    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.634929 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.634901    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.634967 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.634936    2562 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-13.ec2.internal\": node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:34.655179 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.655154    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:34.696303 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.696271    2562 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"]
Jun 02 15:46:34.696456 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.696348    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:34.697317 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.697294    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:34.697434 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.697337    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:34.697434 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.697351    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:34.698499 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.698482    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:34.698672 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.698657    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.698725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.698691    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:34.699298 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.699280    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:34.699392 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.699310    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:34.699392 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.699321    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:34.699863 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.699838    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:34.699863 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.699867    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:34.700042 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.699883    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:34.700805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.700791    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.700852 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.700818    2562 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:34.701435 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.701421    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:34.701516 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.701443    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:34.701516 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.701455    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:34.731535 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.731515    2562 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-13.ec2.internal\" not found" node="ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.735972 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.735957    2562 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-13.ec2.internal\" not found" node="ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.755229 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.755209    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:34.767964 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.767944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e38d5a1ef4ecaf3bfbc23c80e60883ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal\" (UID: \"e38d5a1ef4ecaf3bfbc23c80e60883ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.768037 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.767970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e38d5a1ef4ecaf3bfbc23c80e60883ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal\" (UID: \"e38d5a1ef4ecaf3bfbc23c80e60883ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.768037 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.767986    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74ec08f42b5a0b4796702a35bcb3d051-config\") pod \"kube-apiserver-proxy-ip-10-0-142-13.ec2.internal\" (UID: \"74ec08f42b5a0b4796702a35bcb3d051\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.855298 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.855268    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:34.868713 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.868696    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e38d5a1ef4ecaf3bfbc23c80e60883ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal\" (UID: \"e38d5a1ef4ecaf3bfbc23c80e60883ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.868801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.868723    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74ec08f42b5a0b4796702a35bcb3d051-config\") pod \"kube-apiserver-proxy-ip-10-0-142-13.ec2.internal\" (UID: \"74ec08f42b5a0b4796702a35bcb3d051\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.868801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.868742    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e38d5a1ef4ecaf3bfbc23c80e60883ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal\" (UID: \"e38d5a1ef4ecaf3bfbc23c80e60883ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.868801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.868792    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e38d5a1ef4ecaf3bfbc23c80e60883ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal\" (UID: \"e38d5a1ef4ecaf3bfbc23c80e60883ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.868889 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.868798    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e38d5a1ef4ecaf3bfbc23c80e60883ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal\" (UID: \"e38d5a1ef4ecaf3bfbc23c80e60883ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.868889 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:34.868823    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74ec08f42b5a0b4796702a35bcb3d051-config\") pod \"kube-apiserver-proxy-ip-10-0-142-13.ec2.internal\" (UID: \"74ec08f42b5a0b4796702a35bcb3d051\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:34.956157 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:34.956129    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.033605 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.033571    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:35.039385 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.039370    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:35.056423 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.056386    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.157080 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.157039    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.257546 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.257451    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.358017 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.357972    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.384453 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.384427    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 02 15:46:35.384901 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.384580    2562 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 02 15:46:35.458983 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.458952    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.463183 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.463097    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 02 15:46:35.479522 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.479494    2562 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 15:46:35.482381 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.482343    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 15:41:34 +0000 UTC" deadline="2028-03-12 12:28:04.98109561 +0000 UTC"
Jun 02 15:46:35.482381 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.482375    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15572h41m29.498723828s"
Jun 02 15:46:35.482824 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:35.482803    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38d5a1ef4ecaf3bfbc23c80e60883ee.slice/crio-ecfe79e81a24869ddb1b85e6a7b243806554d522ea1daf263042a941f644021f WatchSource:0}: Error finding container ecfe79e81a24869ddb1b85e6a7b243806554d522ea1daf263042a941f644021f: Status 404 returned error can't find the container with id ecfe79e81a24869ddb1b85e6a7b243806554d522ea1daf263042a941f644021f
Jun 02 15:46:35.483038 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:35.483023    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ec08f42b5a0b4796702a35bcb3d051.slice/crio-38e660f3a1f66c8a9cad4d31ddd2196ee4f43f3ddc4b2e0ad71ce6078509de1f WatchSource:0}: Error finding container 38e660f3a1f66c8a9cad4d31ddd2196ee4f43f3ddc4b2e0ad71ce6078509de1f: Status 404 returned error can't find the container with id 38e660f3a1f66c8a9cad4d31ddd2196ee4f43f3ddc4b2e0ad71ce6078509de1f
Jun 02 15:46:35.486052 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.486030    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 15:46:35.505258 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.505230    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ttfst"
Jun 02 15:46:35.511327 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.511276    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ttfst"
Jun 02 15:46:35.559322 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.559276    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.573564 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.573539    2562 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:35.599212 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.599160    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal" event={"ID":"74ec08f42b5a0b4796702a35bcb3d051","Type":"ContainerStarted","Data":"38e660f3a1f66c8a9cad4d31ddd2196ee4f43f3ddc4b2e0ad71ce6078509de1f"}
Jun 02 15:46:35.600156 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.600126    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal" event={"ID":"e38d5a1ef4ecaf3bfbc23c80e60883ee","Type":"ContainerStarted","Data":"ecfe79e81a24869ddb1b85e6a7b243806554d522ea1daf263042a941f644021f"}
Jun 02 15:46:35.660339 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.660311    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.715831 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.715801    2562 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:35.761216 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.761185    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.861708 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.861625    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.962593 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:35.962544    2562 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-13.ec2.internal\" not found"
Jun 02 15:46:35.968303 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:35.968273    2562 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:36.063844 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.063820    2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:36.075953 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.075927    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 15:46:36.076743 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.076724    2562 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal"
Jun 02 15:46:36.085048 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.085019    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 15:46:36.356667 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.356627    2562 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:36.442964 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.442903    2562 apiserver.go:52] "Watching apiserver"
Jun 02 15:46:36.451635 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.451607    2562 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 02 15:46:36.452780 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.452749    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-pzdmg","kube-system/konnectivity-agent-9bdmf","kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs","openshift-multus/multus-6hj2t","openshift-multus/multus-additional-cni-plugins-wtc7p","openshift-network-diagnostics/network-check-target-r7254","openshift-cluster-node-tuning-operator/tuned-nptbl","openshift-dns/node-resolver-sdcph","openshift-image-registry/node-ca-sfwt7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal","openshift-multus/network-metrics-daemon-jjrxf","openshift-network-operator/iptables-alerter-9qt9d","openshift-ovn-kubernetes/ovnkube-node-qvffn"]
Jun 02 15:46:36.455053 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.455027    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:36.455175 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.455135    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:36.455245 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.455170    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.456405 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.456385    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.457464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.457443    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.457582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.457534    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.457950 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.457766    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 02 15:46:36.457950 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.457934    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hz8x6\""
Jun 02 15:46:36.458081 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.458021    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.458633 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.458611    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.459872 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.459390    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 02 15:46:36.459872 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.459668    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 02 15:46:36.459872 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.459799    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cmx6c\""
Jun 02 15:46:36.460070 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.459901    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 02 15:46:36.460683 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.460275    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.460786 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.460686    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.460786 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.460717    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.461009 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.460995    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 02 15:46:36.461370 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.461353    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.461692 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.461677    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.461829 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.461813    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.461994 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.461976    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mnqvv\""
Jun 02 15:46:36.462154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.462136    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f7rsj\""
Jun 02 15:46:36.462520 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.462256    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 02 15:46:36.463723 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.463531    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 02 15:46:36.463723 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.463657    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.463871 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.463760    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.464222 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.464203    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 02 15:46:36.464890 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.464875    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.464994 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.464874    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.465694 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.465676    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vvjdv\""
Jun 02 15:46:36.465694 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.465691    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mxkrk\""
Jun 02 15:46:36.465826 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.465697    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 02 15:46:36.465826 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.465752    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.465826 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.465808    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.466048 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.466032    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 02 15:46:36.467514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.466343    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.467514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.466436    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.467514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.466798    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 02 15:46:36.467514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.467116    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wh6fq\""
Jun 02 15:46:36.467514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.467156    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7nkv9\""
Jun 02 15:46:36.467514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.467291    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 02 15:46:36.467816 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.467599    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:36.467816 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.467679    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:36.468094 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.468071    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 02 15:46:36.468094 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.468085    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.468232 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.468102    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lqc85\""
Jun 02 15:46:36.468288 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.468253    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 02 15:46:36.468339 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.468297    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 02 15:46:36.468670 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.468650    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.468753 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.468717    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:36.477514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-cni-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.477596 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-cni-binary-copy\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.477596 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysctl-d\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.477596 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dd51254-d67e-4784-9cf7-65c6d1985e2b-tmp\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.477749 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-systemd-units\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.477749 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-cni-bin\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.477749 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovnkube-config\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.477749 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-cni-bin\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.477749 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-conf-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.477749 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-os-release\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.478004 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:36.478004 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477810    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dca09013-6eef-48e9-886f-49647c0b34fa-iptables-alerter-script\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.478004 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-socket-dir-parent\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.478004 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/941b237a-3596-4d10-a02e-73e02d93b836-multus-daemon-config\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.478004 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.477981    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5zl5\" (UniqueName: \"kubernetes.io/projected/941b237a-3596-4d10-a02e-73e02d93b836-kube-api-access-d5zl5\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.478206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478019    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-systemd\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-run\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-etc-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.478206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.478206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0bb9f541-1297-48c3-87f1-765335e0b0eb-hosts-file\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478231    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bb9f541-1297-48c3-87f1-765335e0b0eb-tmp-dir\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-log-socket\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4g6m\" (UniqueName: \"kubernetes.io/projected/06c22a5b-522f-484f-b0bf-09f37b5ac339-kube-api-access-r4g6m\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478339    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d4a6f02e-2650-4045-8a5d-b3b0bad6280d-konnectivity-ca\") pod \"konnectivity-agent-9bdmf\" (UID: \"d4a6f02e-2650-4045-8a5d-b3b0bad6280d\") " pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysctl-conf\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-run-netns\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.478444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478434    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-os-release\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-modprobe-d\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478494    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-host\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478512    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovnkube-script-lib\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-cnibin\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-hostroot\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-system-cni-dir\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-cnibin\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478681    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hph9v\" (UniqueName: \"kubernetes.io/projected/0bb9f541-1297-48c3-87f1-765335e0b0eb-kube-api-access-hph9v\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-lib-modules\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-tuned\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.478764 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovn-node-metrics-cert\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-socket-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478806    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-registration-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/941b237a-3596-4d10-a02e-73e02d93b836-cni-binary-copy\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478889    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qqpt\" (UniqueName: \"kubernetes.io/projected/6dd51254-d67e-4784-9cf7-65c6d1985e2b-kube-api-access-8qqpt\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478904    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-kubelet\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-slash\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.478987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-cni-netd\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-env-overrides\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r4gl\" (UniqueName: \"kubernetes.io/projected/bf747868-df96-43d9-8e2b-e0b6cb56045b-kube-api-access-9r4gl\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-cni-multus\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-netns\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-ovn\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dca09013-6eef-48e9-886f-49647c0b34fa-host-slash\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-device-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.479390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-k8s-cni-cncf-io\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-kubelet\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-node-log\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knrj7\" (UniqueName: \"kubernetes.io/projected/dca09013-6eef-48e9-886f-49647c0b34fa-kube-api-access-knrj7\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxpht\" (UniqueName: \"kubernetes.io/projected/f5b9cd40-972b-4d22-afb2-869f6ce601a5-kube-api-access-zxpht\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-var-lib-kubelet\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-systemd\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d4a6f02e-2650-4045-8a5d-b3b0bad6280d-agent-certs\") pod \"konnectivity-agent-9bdmf\" (UID: \"d4a6f02e-2650-4045-8a5d-b3b0bad6280d\") " pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-sys-fs\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479496    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-multus-certs\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-sys\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479543    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-run-ovn-kubernetes\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-system-cni-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.480038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysconfig\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.480572 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-kubernetes\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.480572 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-etc-kubernetes\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.480572 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.479741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-var-lib-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.512454 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.512425    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:41:35 +0000 UTC" deadline="2027-12-16 17:42:49.933666942 +0000 UTC"
Jun 02 15:46:36.512454 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.512451    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13489h56m13.421219008s"
Jun 02 15:46:36.565497 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.565464    2562 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Jun 02 15:46:36.580990 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.580957    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysctl-conf\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.580999    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-run-netns\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581018    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-os-release\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581037    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-modprobe-d\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581060    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-host\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581083    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovnkube-script-lib\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581086    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-run-netns\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581112    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-os-release\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581152    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-host\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581154 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581161    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-cnibin\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581172    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-modprobe-d\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581187    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-hostroot\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581203    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-cnibin\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581170    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysctl-conf\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581216    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-system-cni-dir\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581221    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-hostroot\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581254    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-cnibin\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581282    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581257    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-system-cni-dir\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581305    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-cnibin\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581310    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hph9v\" (UniqueName: \"kubernetes.io/projected/0bb9f541-1297-48c3-87f1-765335e0b0eb-kube-api-access-hph9v\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581358    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-lib-modules\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581384    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-tuned\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581406    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581412    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581442    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovn-node-metrics-cert\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.581620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581468    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-socket-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581495    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-registration-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581537    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/941b237a-3596-4d10-a02e-73e02d93b836-cni-binary-copy\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581550    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-lib-modules\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581563    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581604    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qqpt\" (UniqueName: \"kubernetes.io/projected/6dd51254-d67e-4784-9cf7-65c6d1985e2b-kube-api-access-8qqpt\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581633    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-registration-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581636    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-kubelet\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581641    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-socket-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581663    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-slash\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581706    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-slash\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.581704    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582039    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582107    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-kubelet\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582128    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-cni-netd\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582156    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-env-overrides\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582181    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/941b237a-3596-4d10-a02e-73e02d93b836-cni-binary-copy\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.582346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582184    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9r4gl\" (UniqueName: \"kubernetes.io/projected/bf747868-df96-43d9-8e2b-e0b6cb56045b-kube-api-access-9r4gl\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582231    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-cni-netd\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582234    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-cni-multus\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582274    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-cni-multus\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582277    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-netns\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582312    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-ovn\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582333    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovnkube-script-lib\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582338    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dca09013-6eef-48e9-886f-49647c0b34fa-host-slash\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582369    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-ovn\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582377    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-netns\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582381    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582396    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-device-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582443    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582414    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dca09013-6eef-48e9-886f-49647c0b34fa-host-slash\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582466    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-device-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d858e0a-17c3-4bc8-b185-1c981b666127-serviceca\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582502    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-k8s-cni-cncf-io\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582530    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-kubelet\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582554    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-node-log\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582557    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582582    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-knrj7\" (UniqueName: \"kubernetes.io/projected/dca09013-6eef-48e9-886f-49647c0b34fa-kube-api-access-knrj7\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582588    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-k8s-cni-cncf-io\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582608    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-node-log\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582610    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zxpht\" (UniqueName: \"kubernetes.io/projected/f5b9cd40-972b-4d22-afb2-869f6ce601a5-kube-api-access-zxpht\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582640    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-var-lib-kubelet\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582661    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-systemd\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582667    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-env-overrides\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582695    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-systemd\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582639    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-kubelet\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582722    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-var-lib-kubelet\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582790    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d4a6f02e-2650-4045-8a5d-b3b0bad6280d-agent-certs\") pod \"konnectivity-agent-9bdmf\" (UID: \"d4a6f02e-2650-4045-8a5d-b3b0bad6280d\") " pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582818    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-sys-fs\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d858e0a-17c3-4bc8-b185-1c981b666127-host\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7m57\" (UniqueName: \"kubernetes.io/projected/5d858e0a-17c3-4bc8-b185-1c981b666127-kube-api-access-r7m57\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.583893 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582882    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-sys-fs\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582895    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-multus-certs\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582942    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-sys\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582969    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.582970    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-run-multus-certs\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583005    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-run-ovn-kubernetes\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583021    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-run-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583038    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-sys\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583035    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583064    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-run-ovn-kubernetes\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583083    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-system-cni-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583128    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysconfig\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583131    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583135    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-system-cni-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583156    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-kubernetes\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583178    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysconfig\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583184    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-etc-kubernetes\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.584725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583200    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-kubernetes\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583213    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-var-lib-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583218    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-etc-kubernetes\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583249    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-var-lib-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583249    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-cni-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583276    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-cni-binary-copy\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583299    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysctl-d\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583322    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dd51254-d67e-4784-9cf7-65c6d1985e2b-tmp\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583338    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-cni-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583346    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-systemd-units\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583370    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-cni-bin\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583387    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovnkube-config\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a83d011-679c-4bab-a182-b89f20266181-dbus\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583440    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-cni-bin\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583448    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-host-cni-bin\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583454    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-sysctl-d\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583479    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-conf-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583493    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-host-var-lib-cni-bin\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.585514 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583506    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-os-release\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583623    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-systemd-units\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583671    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-conf-dir\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583703    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583734    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dca09013-6eef-48e9-886f-49647c0b34fa-iptables-alerter-script\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77d2c\" (UniqueName: \"kubernetes.io/projected/d54bcdc8-8c47-4ede-9def-196f262b8a29-kube-api-access-77d2c\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.583955    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf747868-df96-43d9-8e2b-e0b6cb56045b-cni-binary-copy\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584014    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-socket-dir-parent\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584012    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovnkube-config\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584053    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/941b237a-3596-4d10-a02e-73e02d93b836-multus-daemon-config\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584063    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/941b237a-3596-4d10-a02e-73e02d93b836-multus-socket-dir-parent\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584094    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d5zl5\" (UniqueName: \"kubernetes.io/projected/941b237a-3596-4d10-a02e-73e02d93b836-kube-api-access-d5zl5\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584108    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf747868-df96-43d9-8e2b-e0b6cb56045b-os-release\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584121    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-systemd\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584145    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-run\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584170    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-etc-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.586273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584213    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584242    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a83d011-679c-4bab-a182-b89f20266181-kubelet-config\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584273    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0bb9f541-1297-48c3-87f1-765335e0b0eb-hosts-file\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584297    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dca09013-6eef-48e9-886f-49647c0b34fa-iptables-alerter-script\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584325    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-run\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584351    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-etc-openvswitch\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584362    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bb9f541-1297-48c3-87f1-765335e0b0eb-tmp-dir\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584389    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5b9cd40-972b-4d22-afb2-869f6ce601a5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584393    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-systemd\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584392    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-log-socket\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584429    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r4g6m\" (UniqueName: \"kubernetes.io/projected/06c22a5b-522f-484f-b0bf-09f37b5ac339-kube-api-access-r4g6m\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584442    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0bb9f541-1297-48c3-87f1-765335e0b0eb-hosts-file\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584479    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d4a6f02e-2650-4045-8a5d-b3b0bad6280d-konnectivity-ca\") pod \"konnectivity-agent-9bdmf\" (UID: \"d4a6f02e-2650-4045-8a5d-b3b0bad6280d\") " pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584572    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/06c22a5b-522f-484f-b0bf-09f37b5ac339-log-socket\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584645    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/941b237a-3596-4d10-a02e-73e02d93b836-multus-daemon-config\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584646    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bb9f541-1297-48c3-87f1-765335e0b0eb-tmp-dir\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.584975    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d4a6f02e-2650-4045-8a5d-b3b0bad6280d-konnectivity-ca\") pod \"konnectivity-agent-9bdmf\" (UID: \"d4a6f02e-2650-4045-8a5d-b3b0bad6280d\") " pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.585803    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6dd51254-d67e-4784-9cf7-65c6d1985e2b-etc-tuned\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.587044 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.586159    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/06c22a5b-522f-484f-b0bf-09f37b5ac339-ovn-node-metrics-cert\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.587635 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.586323    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d4a6f02e-2650-4045-8a5d-b3b0bad6280d-agent-certs\") pod \"konnectivity-agent-9bdmf\" (UID: \"d4a6f02e-2650-4045-8a5d-b3b0bad6280d\") " pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.587635 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.587214    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dd51254-d67e-4784-9cf7-65c6d1985e2b-tmp\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.592456 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.592408    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knrj7\" (UniqueName: \"kubernetes.io/projected/dca09013-6eef-48e9-886f-49647c0b34fa-kube-api-access-knrj7\") pod \"iptables-alerter-9qt9d\" (UID: \"dca09013-6eef-48e9-886f-49647c0b34fa\") " pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.592607 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.592554    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hph9v\" (UniqueName: \"kubernetes.io/projected/0bb9f541-1297-48c3-87f1-765335e0b0eb-kube-api-access-hph9v\") pod \"node-resolver-sdcph\" (UID: \"0bb9f541-1297-48c3-87f1-765335e0b0eb\") " pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.593448 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.593425    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qqpt\" (UniqueName: \"kubernetes.io/projected/6dd51254-d67e-4784-9cf7-65c6d1985e2b-kube-api-access-8qqpt\") pod \"tuned-nptbl\" (UID: \"6dd51254-d67e-4784-9cf7-65c6d1985e2b\") " pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.598552 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.598530    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:36.598641 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.598555    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:36.598641 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.598569    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:36.598765 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.598673    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:46:37.098626529 +0000 UTC m=+2.996346898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:36.601147 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.601121    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5zl5\" (UniqueName: \"kubernetes.io/projected/941b237a-3596-4d10-a02e-73e02d93b836-kube-api-access-d5zl5\") pod \"multus-6hj2t\" (UID: \"941b237a-3596-4d10-a02e-73e02d93b836\") " pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.601356 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.601338    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4g6m\" (UniqueName: \"kubernetes.io/projected/06c22a5b-522f-484f-b0bf-09f37b5ac339-kube-api-access-r4g6m\") pod \"ovnkube-node-qvffn\" (UID: \"06c22a5b-522f-484f-b0bf-09f37b5ac339\") " pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.601536 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.601518    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r4gl\" (UniqueName: \"kubernetes.io/projected/bf747868-df96-43d9-8e2b-e0b6cb56045b-kube-api-access-9r4gl\") pod \"multus-additional-cni-plugins-wtc7p\" (UID: \"bf747868-df96-43d9-8e2b-e0b6cb56045b\") " pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.601859 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.601842    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxpht\" (UniqueName: \"kubernetes.io/projected/f5b9cd40-972b-4d22-afb2-869f6ce601a5-kube-api-access-zxpht\") pod \"aws-ebs-csi-driver-node-gxtvs\" (UID: \"f5b9cd40-972b-4d22-afb2-869f6ce601a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.685620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685557    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d858e0a-17c3-4bc8-b185-1c981b666127-serviceca\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.685772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685594    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d858e0a-17c3-4bc8-b185-1c981b666127-host\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.685772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685646    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r7m57\" (UniqueName: \"kubernetes.io/projected/5d858e0a-17c3-4bc8-b185-1c981b666127-kube-api-access-r7m57\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.685772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685680    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a83d011-679c-4bab-a182-b89f20266181-dbus\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.685772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685720    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:36.685772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685744    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77d2c\" (UniqueName: \"kubernetes.io/projected/d54bcdc8-8c47-4ede-9def-196f262b8a29-kube-api-access-77d2c\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:36.685772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685760    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d858e0a-17c3-4bc8-b185-1c981b666127-host\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685775    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a83d011-679c-4bab-a182-b89f20266181-kubelet-config\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.685822    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.685868    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.685946    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.685958    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:37.185938534 +0000 UTC m=+3.083658913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:36.686004    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:37.185989021 +0000 UTC m=+3.083709390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.686045    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d858e0a-17c3-4bc8-b185-1c981b666127-serviceca\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.686113 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.686053    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a83d011-679c-4bab-a182-b89f20266181-kubelet-config\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.686455 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.686179    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a83d011-679c-4bab-a182-b89f20266181-dbus\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:36.698055 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.698029    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7m57\" (UniqueName: \"kubernetes.io/projected/5d858e0a-17c3-4bc8-b185-1c981b666127-kube-api-access-r7m57\") pod \"node-ca-sfwt7\" (UID: \"5d858e0a-17c3-4bc8-b185-1c981b666127\") " pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:36.699515 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.699494    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77d2c\" (UniqueName: \"kubernetes.io/projected/d54bcdc8-8c47-4ede-9def-196f262b8a29-kube-api-access-77d2c\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:36.771463 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.771425    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9qt9d"
Jun 02 15:46:36.778294 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.778270    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:46:36.785905 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.785881    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs"
Jun 02 15:46:36.791645 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.791622    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6hj2t"
Jun 02 15:46:36.799296 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.799274    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nptbl"
Jun 02 15:46:36.804897 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.804877    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wtc7p"
Jun 02 15:46:36.810509 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.810488    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sdcph"
Jun 02 15:46:36.820093 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.820068    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:36.824679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:36.824660    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfwt7"
Jun 02 15:46:37.025763 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.025735    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c22a5b_522f_484f_b0bf_09f37b5ac339.slice/crio-94f43a4cf04bc85db02c98e5c152d9ef3b49782099dc92a125d19d0098f326de WatchSource:0}: Error finding container 94f43a4cf04bc85db02c98e5c152d9ef3b49782099dc92a125d19d0098f326de: Status 404 returned error can't find the container with id 94f43a4cf04bc85db02c98e5c152d9ef3b49782099dc92a125d19d0098f326de
Jun 02 15:46:37.027642 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.027616    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf747868_df96_43d9_8e2b_e0b6cb56045b.slice/crio-26c9d4d7fcfee69debf5520f64e1175ec700ee9db5e4345e99a008c755d59b14 WatchSource:0}: Error finding container 26c9d4d7fcfee69debf5520f64e1175ec700ee9db5e4345e99a008c755d59b14: Status 404 returned error can't find the container with id 26c9d4d7fcfee69debf5520f64e1175ec700ee9db5e4345e99a008c755d59b14
Jun 02 15:46:37.028373 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.028349    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b9cd40_972b_4d22_afb2_869f6ce601a5.slice/crio-aec95f8286b83c606d4c6ea33af69be7f89b9b08886d76cb321490685b32967e WatchSource:0}: Error finding container aec95f8286b83c606d4c6ea33af69be7f89b9b08886d76cb321490685b32967e: Status 404 returned error can't find the container with id aec95f8286b83c606d4c6ea33af69be7f89b9b08886d76cb321490685b32967e
Jun 02 15:46:37.030777 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.030744    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd51254_d67e_4784_9cf7_65c6d1985e2b.slice/crio-0d99ceec226508eacb473f196f926069ae09a0ab67fea5ca500e52fb7b1d25a8 WatchSource:0}: Error finding container 0d99ceec226508eacb473f196f926069ae09a0ab67fea5ca500e52fb7b1d25a8: Status 404 returned error can't find the container with id 0d99ceec226508eacb473f196f926069ae09a0ab67fea5ca500e52fb7b1d25a8
Jun 02 15:46:37.032250 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.032090    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a6f02e_2650_4045_8a5d_b3b0bad6280d.slice/crio-32b27e8337d1bff772ddefcc908d1071b54a07fd581e45b9f49b5d7ce8db87b2 WatchSource:0}: Error finding container 32b27e8337d1bff772ddefcc908d1071b54a07fd581e45b9f49b5d7ce8db87b2: Status 404 returned error can't find the container with id 32b27e8337d1bff772ddefcc908d1071b54a07fd581e45b9f49b5d7ce8db87b2
Jun 02 15:46:37.032967 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.032945    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941b237a_3596_4d10_a02e_73e02d93b836.slice/crio-134711cd6f11154d0ac5b79ad3c0838ee40de36eb61d98c9757e79754dfe8efc WatchSource:0}: Error finding container 134711cd6f11154d0ac5b79ad3c0838ee40de36eb61d98c9757e79754dfe8efc: Status 404 returned error can't find the container with id 134711cd6f11154d0ac5b79ad3c0838ee40de36eb61d98c9757e79754dfe8efc
Jun 02 15:46:37.033260 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.033235    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d858e0a_17c3_4bc8_b185_1c981b666127.slice/crio-12b35df916e38583f4475103670c76408bdfeda1f6199e6afdca3308f5dbe181 WatchSource:0}: Error finding container 12b35df916e38583f4475103670c76408bdfeda1f6199e6afdca3308f5dbe181: Status 404 returned error can't find the container with id 12b35df916e38583f4475103670c76408bdfeda1f6199e6afdca3308f5dbe181
Jun 02 15:46:37.035206 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.034878    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb9f541_1297_48c3_87f1_765335e0b0eb.slice/crio-c8f33348da1256e02408fe7659672ab5195edec026901bc1a8ca3ac45dab9eeb WatchSource:0}: Error finding container c8f33348da1256e02408fe7659672ab5195edec026901bc1a8ca3ac45dab9eeb: Status 404 returned error can't find the container with id c8f33348da1256e02408fe7659672ab5195edec026901bc1a8ca3ac45dab9eeb
Jun 02 15:46:37.036110 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:46:37.036082    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca09013_6eef_48e9_886f_49647c0b34fa.slice/crio-47728a7007d2fb765bd91341a949346c77af784c4b559b0c1c6bd9ccb8d74bd8 WatchSource:0}: Error finding container 47728a7007d2fb765bd91341a949346c77af784c4b559b0c1c6bd9ccb8d74bd8: Status 404 returned error can't find the container with id 47728a7007d2fb765bd91341a949346c77af784c4b559b0c1c6bd9ccb8d74bd8
Jun 02 15:46:37.189662 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.189450    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.189684    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.189718    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189605    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189752    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189764    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189800    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:37.189814 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189808    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:37.190104 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189828    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:46:38.189803406 +0000 UTC m=+4.087523778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:37.190104 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189843    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:38.189836788 +0000 UTC m=+4.087557154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:37.190104 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.189852    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:38.189847405 +0000 UTC m=+4.087567771 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:37.513843 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.513600    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:41:35 +0000 UTC" deadline="2028-02-26 02:10:08.817141654 +0000 UTC"
Jun 02 15:46:37.513843 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.513641    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15202h23m31.303503674s"
Jun 02 15:46:37.599305 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.596419    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:37.599305 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:37.596556    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:37.612335 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.612296    2562 generic.go:358] "Generic (PLEG): container finished" podID="e38d5a1ef4ecaf3bfbc23c80e60883ee" containerID="a38cb699fcf960bcd0e53a7de739c05d84257e07c15ca2014c87863a4e5c36ed" exitCode=0
Jun 02 15:46:37.612500 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.612377    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal" event={"ID":"e38d5a1ef4ecaf3bfbc23c80e60883ee","Type":"ContainerDied","Data":"a38cb699fcf960bcd0e53a7de739c05d84257e07c15ca2014c87863a4e5c36ed"}
Jun 02 15:46:37.636196 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.636157    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9qt9d" event={"ID":"dca09013-6eef-48e9-886f-49647c0b34fa","Type":"ContainerStarted","Data":"47728a7007d2fb765bd91341a949346c77af784c4b559b0c1c6bd9ccb8d74bd8"}
Jun 02 15:46:37.640577 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.640547    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfwt7" event={"ID":"5d858e0a-17c3-4bc8-b185-1c981b666127","Type":"ContainerStarted","Data":"12b35df916e38583f4475103670c76408bdfeda1f6199e6afdca3308f5dbe181"}
Jun 02 15:46:37.651695 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.648509    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9bdmf" event={"ID":"d4a6f02e-2650-4045-8a5d-b3b0bad6280d","Type":"ContainerStarted","Data":"32b27e8337d1bff772ddefcc908d1071b54a07fd581e45b9f49b5d7ce8db87b2"}
Jun 02 15:46:37.655694 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.655609    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerStarted","Data":"26c9d4d7fcfee69debf5520f64e1175ec700ee9db5e4345e99a008c755d59b14"}
Jun 02 15:46:37.669396 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.669354    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal" event={"ID":"74ec08f42b5a0b4796702a35bcb3d051","Type":"ContainerStarted","Data":"6d4edb5fa78d97b76867ac15c2f8a80122a52b79331af0a9fbedd635e6058922"}
Jun 02 15:46:37.684831 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.684775    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-13.ec2.internal" podStartSLOduration=1.684758652 podStartE2EDuration="1.684758652s" podCreationTimestamp="2026-06-02 15:46:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:46:37.684214583 +0000 UTC m=+3.581934996" watchObservedRunningTime="2026-06-02 15:46:37.684758652 +0000 UTC m=+3.582479042"
Jun 02 15:46:37.687555 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.687529    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sdcph" event={"ID":"0bb9f541-1297-48c3-87f1-765335e0b0eb","Type":"ContainerStarted","Data":"c8f33348da1256e02408fe7659672ab5195edec026901bc1a8ca3ac45dab9eeb"}
Jun 02 15:46:37.704829 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.704800    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6hj2t" event={"ID":"941b237a-3596-4d10-a02e-73e02d93b836","Type":"ContainerStarted","Data":"134711cd6f11154d0ac5b79ad3c0838ee40de36eb61d98c9757e79754dfe8efc"}
Jun 02 15:46:37.717732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.717695    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs" event={"ID":"f5b9cd40-972b-4d22-afb2-869f6ce601a5","Type":"ContainerStarted","Data":"aec95f8286b83c606d4c6ea33af69be7f89b9b08886d76cb321490685b32967e"}
Jun 02 15:46:37.726451 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.726416    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nptbl" event={"ID":"6dd51254-d67e-4784-9cf7-65c6d1985e2b","Type":"ContainerStarted","Data":"0d99ceec226508eacb473f196f926069ae09a0ab67fea5ca500e52fb7b1d25a8"}
Jun 02 15:46:37.729043 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:37.729016    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"94f43a4cf04bc85db02c98e5c152d9ef3b49782099dc92a125d19d0098f326de"}
Jun 02 15:46:38.198613 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.198530    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:38.198785 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.198718    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:38.198860 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.198783    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:38.198945 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.198852    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:38.198945 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.198882    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:38.198945 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.198896    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:38.199090 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.198971    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:38.199090 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.199028    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:46:40.199004488 +0000 UTC m=+6.096724869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:38.199090 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.199054    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:38.199090 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.199054    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:40.199042935 +0000 UTC m=+6.096763306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:38.199292 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.199110    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:40.199084102 +0000 UTC m=+6.096804481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:38.598779 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.598703    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:38.599153 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.598843    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:38.599333 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.599315    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:38.599444 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:38.599421    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:38.751649 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.751606    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal" event={"ID":"e38d5a1ef4ecaf3bfbc23c80e60883ee","Type":"ContainerStarted","Data":"426478ae2535a53b5a48d5dbe6874025d9cd1825f1f3be171a57f58627d488a0"}
Jun 02 15:46:38.767082 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:38.766413    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-13.ec2.internal" podStartSLOduration=2.766392207 podStartE2EDuration="2.766392207s" podCreationTimestamp="2026-06-02 15:46:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:46:38.766114789 +0000 UTC m=+4.663835180" watchObservedRunningTime="2026-06-02 15:46:38.766392207 +0000 UTC m=+4.664112614"
Jun 02 15:46:39.598649 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:39.596089    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:39.598649 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:39.596231    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:40.217456 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:40.217413    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:40.217621 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:40.217472    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:40.217621 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:40.217520    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:40.217719 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217685    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:40.217764 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217750    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:44.217730679 +0000 UTC m=+10.115451061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:40.217876 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217845    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:40.217947 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217904    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:44.217891385 +0000 UTC m=+10.115611767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:40.217947 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217925    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:40.218033 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217950    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:40.218033 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.217987    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:40.218119 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.218045    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:46:44.218026717 +0000 UTC m=+10.115747086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:40.599581 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:40.599116    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:40.599581 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.599241    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:40.599581 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:40.599424    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:40.600145 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:40.599876    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:41.596285 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:41.596246    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:41.596468 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:41.596399    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:42.596280 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:42.596093    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:42.596280 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:42.596134    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:42.596280 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:42.596230    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:42.596865 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:42.596550    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:43.597037 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:43.596999    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:43.597498 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:43.597147    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:44.253281 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:44.253227    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:44.253281 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:44.253277    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:44.253324    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253417    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253442    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253447    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253457    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253468    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253518    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:52.253499395 +0000 UTC m=+18.151219773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253538    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:52.253528573 +0000 UTC m=+18.151248943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:44.253736 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.253554    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:46:52.253544732 +0000 UTC m=+18.151265100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:44.597601 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:44.597034    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:44.597601 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.597169    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:44.598938 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:44.598247    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:44.598938 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:44.598354    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:45.596136 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:45.596084    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:45.596327 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:45.596246    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:46.597110 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:46.597066    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:46.597543 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:46.597066    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:46.597543 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:46.597211    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:46.597543 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:46.597253    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:47.596669 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:47.596634    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:47.596852 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:47.596771    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:48.596834 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:48.596786    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:48.597285 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:48.596805    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:48.597285 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:48.596944    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:48.597285 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:48.596971    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:49.596174 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:49.596137    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:49.596344 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:49.596270    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:50.596467 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:50.596428    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:50.597053 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:50.596566    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:50.597053 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:50.596620    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:50.597053 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:50.596737    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:51.596218 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:51.596181    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:51.596383 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:51.596322    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:52.317520 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:52.317482    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:52.317529    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:52.317567    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317655    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317683    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317697    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317666    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317749    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:47:08.31773609 +0000 UTC m=+34.215456456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317793    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:08.317778467 +0000 UTC m=+34.215498838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317665    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:52.317987 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.317831    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:08.31782194 +0000 UTC m=+34.215542312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:52.596438 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:52.596353    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:52.596589 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.596481    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:52.596589 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:52.596543    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:52.596700 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:52.596648    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:53.596520 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:53.596487    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:53.596904 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:53.596622    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:54.597501 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:54.597448    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:54.597906 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:54.597556    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:54.597906 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:54.597619    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:54.597906 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:54.597741    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:55.597155 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.596997    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:55.597257 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:55.597234    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:55.780140 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.780107    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sdcph" event={"ID":"0bb9f541-1297-48c3-87f1-765335e0b0eb","Type":"ContainerStarted","Data":"61518dfdd1721b18a80b44dd4dae54781eaed845da4600a221f0e43f20a19d84"}
Jun 02 15:46:55.781591 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.781566    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6hj2t" event={"ID":"941b237a-3596-4d10-a02e-73e02d93b836","Type":"ContainerStarted","Data":"9d975ada9fdfe76e0daf4b97860c2adb4a15b3d6c7a17a757bea99e66fce6bc7"}
Jun 02 15:46:55.782923 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.782887    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs" event={"ID":"f5b9cd40-972b-4d22-afb2-869f6ce601a5","Type":"ContainerStarted","Data":"a9a2985dedc8ebcd786711196489688608781ca1d9b77d95cc225c3c5760837f"}
Jun 02 15:46:55.784390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.784361    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nptbl" event={"ID":"6dd51254-d67e-4784-9cf7-65c6d1985e2b","Type":"ContainerStarted","Data":"7ac6b775e33d46bbe907a3c74c110e20bed6fbce7f7184e4c72b9ba14b6048aa"}
Jun 02 15:46:55.785847 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.785831    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:46:55.786120 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.786093    2562 generic.go:358] "Generic (PLEG): container finished" podID="06c22a5b-522f-484f-b0bf-09f37b5ac339" containerID="a3b73cab0cbb9cfd9a2f14466c7be1f6bc180b4758fee863ec5c6a1ec1ade8e7" exitCode=1
Jun 02 15:46:55.786212 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.786157    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"1eaf69d47a24e2c069f7905c0f776d2154c1e4780de397426398aa693d33027c"}
Jun 02 15:46:55.786212 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.786186    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerDied","Data":"a3b73cab0cbb9cfd9a2f14466c7be1f6bc180b4758fee863ec5c6a1ec1ade8e7"}
Jun 02 15:46:55.786212 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.786197    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"a91e4588759e5656e9483e02e1a323736eb6b047c656fda0b965e38edd02cfb7"}
Jun 02 15:46:55.787632 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.787608    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfwt7" event={"ID":"5d858e0a-17c3-4bc8-b185-1c981b666127","Type":"ContainerStarted","Data":"f63ab41990a9c9fa46a4b88de5ffc62f73672b2af8489ec912f7e46764455373"}
Jun 02 15:46:55.789056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.789033    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9bdmf" event={"ID":"d4a6f02e-2650-4045-8a5d-b3b0bad6280d","Type":"ContainerStarted","Data":"34d8ed2c257a7eefb5ae27ae5eefc0bb02d98285bd5813b82c73d2390c09ae2a"}
Jun 02 15:46:55.790269 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.790245    2562 generic.go:358] "Generic (PLEG): container finished" podID="bf747868-df96-43d9-8e2b-e0b6cb56045b" containerID="152ce7e297f7bea3eac044185fb8f03535cdbec4c451d250450f6f5d0db1f361" exitCode=0
Jun 02 15:46:55.790346 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.790277    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerDied","Data":"152ce7e297f7bea3eac044185fb8f03535cdbec4c451d250450f6f5d0db1f361"}
Jun 02 15:46:55.797018 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.796970    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-sdcph" podStartSLOduration=3.548632715 podStartE2EDuration="21.796958963s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.037023633 +0000 UTC m=+2.934744009" lastFinishedPulling="2026-06-02 15:46:55.285349877 +0000 UTC m=+21.183070257" observedRunningTime="2026-06-02 15:46:55.79639635 +0000 UTC m=+21.694116737" watchObservedRunningTime="2026-06-02 15:46:55.796958963 +0000 UTC m=+21.694679351"
Jun 02 15:46:55.818604 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.818550    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-6hj2t" podStartSLOduration=3.551579878 podStartE2EDuration="21.818532134s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.034288386 +0000 UTC m=+2.932008752" lastFinishedPulling="2026-06-02 15:46:55.301240626 +0000 UTC m=+21.198961008" observedRunningTime="2026-06-02 15:46:55.817394299 +0000 UTC m=+21.715114708" watchObservedRunningTime="2026-06-02 15:46:55.818532134 +0000 UTC m=+21.716252555"
Jun 02 15:46:55.836150 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.836109    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nptbl" podStartSLOduration=3.5819471959999998 podStartE2EDuration="21.836094025s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.032322702 +0000 UTC m=+2.930043069" lastFinishedPulling="2026-06-02 15:46:55.286469525 +0000 UTC m=+21.184189898" observedRunningTime="2026-06-02 15:46:55.835894803 +0000 UTC m=+21.733615216" watchObservedRunningTime="2026-06-02 15:46:55.836094025 +0000 UTC m=+21.733814412"
Jun 02 15:46:55.887404 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.887364    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9bdmf" podStartSLOduration=11.801701514 podStartE2EDuration="21.887349857s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.034359973 +0000 UTC m=+2.932080342" lastFinishedPulling="2026-06-02 15:46:47.120008303 +0000 UTC m=+13.017728685" observedRunningTime="2026-06-02 15:46:55.886711733 +0000 UTC m=+21.784432121" watchObservedRunningTime="2026-06-02 15:46:55.887349857 +0000 UTC m=+21.785070244"
Jun 02 15:46:55.903165 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:55.903108    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sfwt7" podStartSLOduration=3.653287122 podStartE2EDuration="21.903091611s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.035552994 +0000 UTC m=+2.933273360" lastFinishedPulling="2026-06-02 15:46:55.285357467 +0000 UTC m=+21.183077849" observedRunningTime="2026-06-02 15:46:55.902847317 +0000 UTC m=+21.800567704" watchObservedRunningTime="2026-06-02 15:46:55.903091611 +0000 UTC m=+21.800812001"
Jun 02 15:46:56.596738 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.596523    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:56.596847 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.596526    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:56.596847 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:56.596832    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:56.596982 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:56.596953    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:56.717964 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.717830    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 02 15:46:56.795039 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.795009    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:46:56.795588 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.795383    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"a3eceb5997ba9c0e9eb07143da0597dae77e7365be0e27e84811582ab2be6571"}
Jun 02 15:46:56.795588 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.795418    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"1586ee62e2131b40ff7d4d2f957b76f732c16e466b57fda78146eb39dddad542"}
Jun 02 15:46:56.795588 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.795431    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"9ce2a2c9762b845ac65c6a909c0db708cde427e9d5a44fa0438a7652969224bd"}
Jun 02 15:46:56.796774 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.796746    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9qt9d" event={"ID":"dca09013-6eef-48e9-886f-49647c0b34fa","Type":"ContainerStarted","Data":"2282a38f4b8ef353771e41164c2ea44060962c1c2a73176df0bc0f4c055c73da"}
Jun 02 15:46:56.798178 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.798156    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs" event={"ID":"f5b9cd40-972b-4d22-afb2-869f6ce601a5","Type":"ContainerStarted","Data":"ff6f829ff5cef4116a5b35deb358a79d6cc2261684aac580d4c2a3b7faaf1d81"}
Jun 02 15:46:56.814365 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:56.814287    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9qt9d" podStartSLOduration=4.567090498 podStartE2EDuration="22.814274782s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.038187652 +0000 UTC m=+2.935908025" lastFinishedPulling="2026-06-02 15:46:55.28537193 +0000 UTC m=+21.183092309" observedRunningTime="2026-06-02 15:46:56.81411314 +0000 UTC m=+22.711833529" watchObservedRunningTime="2026-06-02 15:46:56.814274782 +0000 UTC m=+22.711995169"
Jun 02 15:46:57.237099 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.237067    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:57.538928 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.538740    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T15:46:56.717861873Z","UUID":"fdc08255-efc8-43b9-9a86-6c190bb6d35d","Handler":null,"Name":"","Endpoint":""}
Jun 02 15:46:57.540748 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.540725    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 02 15:46:57.540748 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.540755    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 02 15:46:57.597002 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.596970    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:57.597146 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:57.597110    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:57.801344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.801279    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs" event={"ID":"f5b9cd40-972b-4d22-afb2-869f6ce601a5","Type":"ContainerStarted","Data":"b3040bc138317326a749a13a8defcde40429ba303881c94acc55be88780e1a34"}
Jun 02 15:46:57.820784 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:57.820739    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtvs" podStartSLOduration=3.172457678 podStartE2EDuration="23.820724731s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.030359449 +0000 UTC m=+2.928079828" lastFinishedPulling="2026-06-02 15:46:57.67862651 +0000 UTC m=+23.576346881" observedRunningTime="2026-06-02 15:46:57.820699847 +0000 UTC m=+23.718420235" watchObservedRunningTime="2026-06-02 15:46:57.820724731 +0000 UTC m=+23.718445120"
Jun 02 15:46:58.596308 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:58.596275    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:46:58.596511 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:58.596321    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:46:58.596511 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:58.596415    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:46:58.596624 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:58.596557    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:46:58.806201 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:58.806019    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:46:58.806664 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:58.806523    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"bb6fb5e5e95756c8589c281e814bcd06cc94bfe58b934e58a0ad08e64871b5ea"}
Jun 02 15:46:59.349406 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:59.349378    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:59.350019 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:59.350002    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:46:59.596784 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:59.596754    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:46:59.596990 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:46:59.596880    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:46:59.809589 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:46:59.809562    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9bdmf"
Jun 02 15:47:00.596387 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:00.596272    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:00.596387 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:00.596322    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:00.596608 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:00.596402    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:47:00.596608 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:00.596493    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:47:01.596405 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.596218    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:01.596999 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:01.596499    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:47:01.817433 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.817405    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:47:01.817770 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.817739    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"fa6d908a72fcf18356f0acea15b25004e13f92dcfefcaab49fe99ec00d8d9e82"}
Jun 02 15:47:01.818124 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.818103    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:47:01.818205 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.818138    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:47:01.818260 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.818248    2562 scope.go:117] "RemoveContainer" containerID="a3b73cab0cbb9cfd9a2f14466c7be1f6bc180b4758fee863ec5c6a1ec1ade8e7"
Jun 02 15:47:01.820852 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.819665    2562 generic.go:358] "Generic (PLEG): container finished" podID="bf747868-df96-43d9-8e2b-e0b6cb56045b" containerID="b10dbfd5a299892594e3372e07ad1034ad391f7c65825d3d8befe3f1bcdbdbb9" exitCode=0
Jun 02 15:47:01.820852 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.819703    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerDied","Data":"b10dbfd5a299892594e3372e07ad1034ad391f7c65825d3d8befe3f1bcdbdbb9"}
Jun 02 15:47:01.835035 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:01.835014    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:47:02.596890 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.596855    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:02.596890 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.596873    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:02.597509 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:02.596991    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:47:02.597509 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:02.597078    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:47:02.824988 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.824963    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:47:02.825295 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.825273    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" event={"ID":"06c22a5b-522f-484f-b0bf-09f37b5ac339","Type":"ContainerStarted","Data":"5685c895903291a390bd3cf829a15b685eb1dacc129ca3ae86f6ebb01452aa46"}
Jun 02 15:47:02.825642 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.825621    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:47:02.840468 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.840441    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:47:02.858349 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:02.858255    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn" podStartSLOduration=10.546319082 podStartE2EDuration="28.858240591s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.027689463 +0000 UTC m=+2.925409842" lastFinishedPulling="2026-06-02 15:46:55.339610972 +0000 UTC m=+21.237331351" observedRunningTime="2026-06-02 15:47:02.857551223 +0000 UTC m=+28.755271611" watchObservedRunningTime="2026-06-02 15:47:02.858240591 +0000 UTC m=+28.755960976"
Jun 02 15:47:03.400808 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.400773    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jjrxf"]
Jun 02 15:47:03.400988 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.400966    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:03.401087 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:03.401068    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:47:03.401376 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.401357    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pzdmg"]
Jun 02 15:47:03.401452 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.401442    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:03.401528 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:03.401513    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:47:03.404010 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.403987    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r7254"]
Jun 02 15:47:03.404126 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.404067    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:03.404189 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:03.404135    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:47:03.829325 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.829294    2562 generic.go:358] "Generic (PLEG): container finished" podID="bf747868-df96-43d9-8e2b-e0b6cb56045b" containerID="ae0e6ade0c0e666aadf23e70d1289f9ebb7421599dd98c417d210248e437cfc6" exitCode=0
Jun 02 15:47:03.829722 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:03.829381    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerDied","Data":"ae0e6ade0c0e666aadf23e70d1289f9ebb7421599dd98c417d210248e437cfc6"}
Jun 02 15:47:04.597973 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:04.597663    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:04.598102 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:04.597724    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:04.598102 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:04.598040    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:47:04.598174 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:04.598156    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:47:05.597011 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:05.596928    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:05.597353 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:05.597034    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:47:05.834497 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:05.834466    2562 generic.go:358] "Generic (PLEG): container finished" podID="bf747868-df96-43d9-8e2b-e0b6cb56045b" containerID="bd74b1d30db50c2a8d95de8b3a5a06da288d40df5800383b888abf72655dfbee" exitCode=0
Jun 02 15:47:05.834670 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:05.834507    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerDied","Data":"bd74b1d30db50c2a8d95de8b3a5a06da288d40df5800383b888abf72655dfbee"}
Jun 02 15:47:06.596611 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:06.596571    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:06.596774 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:06.596694    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzdmg" podUID="4a83d011-679c-4bab-a182-b89f20266181"
Jun 02 15:47:06.596861 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:06.596776    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:06.596967 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:06.596943    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7254" podUID="a6572f3a-8d66-4e30-9ec0-138977fcfaca"
Jun 02 15:47:07.596269 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:07.596233    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:07.596819 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:07.596391    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjrxf" podUID="d54bcdc8-8c47-4ede-9def-196f262b8a29"
Jun 02 15:47:08.347129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.347036    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:08.347129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.347089    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:08.347129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.347130    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347238    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347289    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347313    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret podName:4a83d011-679c-4bab-a182-b89f20266181 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:40.347287607 +0000 UTC m=+66.245007976 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret") pod "global-pull-secret-syncer-pzdmg" (UID: "4a83d011-679c-4bab-a182-b89f20266181") : object "kube-system"/"original-pull-secret" not registered
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347239    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347345    2562 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347353    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs podName:d54bcdc8-8c47-4ede-9def-196f262b8a29 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:40.347333792 +0000 UTC m=+66.245054165 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs") pod "network-metrics-daemon-jjrxf" (UID: "d54bcdc8-8c47-4ede-9def-196f262b8a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347358    2562 projected.go:194] Error preparing data for projected volume kube-api-access-68mn9 for pod openshift-network-diagnostics/network-check-target-r7254: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:08.347429 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.347396    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9 podName:a6572f3a-8d66-4e30-9ec0-138977fcfaca nodeName:}" failed. No retries permitted until 2026-06-02 15:47:40.347384883 +0000 UTC m=+66.245105264 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-68mn9" (UniqueName: "kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9") pod "network-check-target-r7254" (UID: "a6572f3a-8d66-4e30-9ec0-138977fcfaca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:08.481711 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.481682    2562 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-13.ec2.internal" event="NodeReady"
Jun 02 15:47:08.481949 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.481831    2562 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Jun 02 15:47:08.519229 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.519195    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-574b6f686f-zdxlk"]
Jun 02 15:47:08.535333 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.535297    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wbb7q"]
Jun 02 15:47:08.535510 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.535490    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.537293 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.537264    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jun 02 15:47:08.537521 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.537498    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jun 02 15:47:08.537596 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.537571    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vhj6z\""
Jun 02 15:47:08.537997 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.537856    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jun 02 15:47:08.543379 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.543354    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jun 02 15:47:08.556311 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.556105    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bqm8z"]
Jun 02 15:47:08.556311 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.556241    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:08.558677 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.558649    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 02 15:47:08.558677 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.558664    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 02 15:47:08.558865 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.558688    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ws8nx\""
Jun 02 15:47:08.558865 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.558694    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 02 15:47:08.578055 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.578030    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-574b6f686f-zdxlk"]
Jun 02 15:47:08.578055 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.578058    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbb7q"]
Jun 02 15:47:08.578232 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.578069    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bqm8z"]
Jun 02 15:47:08.578232 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.578172    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.580594 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.580565    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 02 15:47:08.580732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.580637    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6cp8\""
Jun 02 15:47:08.580732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.580565    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 02 15:47:08.599529 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.599455    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:08.600127 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.599455    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:08.601420 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.601396    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 15:47:08.601583 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.601396    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 15:47:08.601677 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.601601    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 02 15:47:08.601732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.601674    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6qg5h\""
Jun 02 15:47:08.650109 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f55f\" (UniqueName: \"kubernetes.io/projected/eff42c24-b415-4b7e-b900-3a00e7183c53-kube-api-access-2f55f\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:08.650109 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4bc93ff1-5693-48f6-8910-3820a3d18d1a-ca-trust-extracted\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-image-registry-private-configuration\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-bound-sa-token\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:08.650350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-installation-pull-secrets\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-certificates\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-trusted-ca\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650654 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.650654 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.650389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg6nj\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-kube-api-access-qg6nj\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751402 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751367    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2f55f\" (UniqueName: \"kubernetes.io/projected/eff42c24-b415-4b7e-b900-3a00e7183c53-kube-api-access-2f55f\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:08.751579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751416    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4bc93ff1-5693-48f6-8910-3820a3d18d1a-ca-trust-extracted\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751450    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-image-registry-private-configuration\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-config-volume\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.751579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751517    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-bound-sa-token\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751544    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-tmp-dir\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.751579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn8wm\" (UniqueName: \"kubernetes.io/projected/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-kube-api-access-dn8wm\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.751805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751592    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:08.751805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751615    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-installation-pull-secrets\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.751805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751714    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-certificates\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751740    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-trusted-ca\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.751805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751785    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.752043 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751818    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qg6nj\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-kube-api-access-qg6nj\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.752043 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.751945    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4bc93ff1-5693-48f6-8910-3820a3d18d1a-ca-trust-extracted\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.752572 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.752220    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:47:08.752572 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.752242    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574b6f686f-zdxlk: secret "image-registry-tls" not found
Jun 02 15:47:08.752572 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.752295    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls podName:4bc93ff1-5693-48f6-8910-3820a3d18d1a nodeName:}" failed. No retries permitted until 2026-06-02 15:47:09.252277112 +0000 UTC m=+35.149997494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls") pod "image-registry-574b6f686f-zdxlk" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a") : secret "image-registry-tls" not found
Jun 02 15:47:08.752572 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.752218    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:08.752572 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.752504    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert podName:eff42c24-b415-4b7e-b900-3a00e7183c53 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:09.252490513 +0000 UTC m=+35.150210889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert") pod "ingress-canary-wbb7q" (UID: "eff42c24-b415-4b7e-b900-3a00e7183c53") : secret "canary-serving-cert" not found
Jun 02 15:47:08.752806 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.752649    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-trusted-ca\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.752806 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.752668    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-certificates\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.757361 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.757340    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-image-registry-private-configuration\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.757476 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.757349    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-installation-pull-secrets\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.764349 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.764324    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg6nj\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-kube-api-access-qg6nj\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.764899 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.764879    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-bound-sa-token\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:08.765271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.765249    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f55f\" (UniqueName: \"kubernetes.io/projected/eff42c24-b415-4b7e-b900-3a00e7183c53-kube-api-access-2f55f\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:08.852795 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.852702    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-config-volume\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.852795 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.852748    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-tmp-dir\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.852795 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.852777    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dn8wm\" (UniqueName: \"kubernetes.io/projected/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-kube-api-access-dn8wm\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.853090 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.852806    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.853090 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.852974    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:08.853090 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:08.853044    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls podName:a7be4dd9-3fd8-4a8f-9697-785ae760e14e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:09.353025816 +0000 UTC m=+35.250746185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls") pod "dns-default-bqm8z" (UID: "a7be4dd9-3fd8-4a8f-9697-785ae760e14e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:08.853257 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.853120    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-tmp-dir\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.853389 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.853366    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-config-volume\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:08.862508 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:08.862484    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn8wm\" (UniqueName: \"kubernetes.io/projected/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-kube-api-access-dn8wm\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:09.256473 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:09.256434    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:09.256754 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:09.256523    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:09.256754 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.256605    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:09.256754 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.256673    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert podName:eff42c24-b415-4b7e-b900-3a00e7183c53 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:10.256654004 +0000 UTC m=+36.154374382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert") pod "ingress-canary-wbb7q" (UID: "eff42c24-b415-4b7e-b900-3a00e7183c53") : secret "canary-serving-cert" not found
Jun 02 15:47:09.256754 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.256687    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:47:09.256754 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.256709    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574b6f686f-zdxlk: secret "image-registry-tls" not found
Jun 02 15:47:09.257053 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.256773    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls podName:4bc93ff1-5693-48f6-8910-3820a3d18d1a nodeName:}" failed. No retries permitted until 2026-06-02 15:47:10.256757827 +0000 UTC m=+36.154478214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls") pod "image-registry-574b6f686f-zdxlk" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a") : secret "image-registry-tls" not found
Jun 02 15:47:09.357085 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:09.357038    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:09.357287 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.357186    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:09.357287 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:09.357250    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls podName:a7be4dd9-3fd8-4a8f-9697-785ae760e14e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:10.357235908 +0000 UTC m=+36.254956274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls") pod "dns-default-bqm8z" (UID: "a7be4dd9-3fd8-4a8f-9697-785ae760e14e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:09.597067 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:09.596989    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:09.599643 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:09.599620    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vmzq\""
Jun 02 15:47:09.600089 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:09.599646    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 15:47:10.263400 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:10.263347    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:10.263579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:10.263455    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:10.263638 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.263607    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:47:10.263638 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.263626    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574b6f686f-zdxlk: secret "image-registry-tls" not found
Jun 02 15:47:10.263728 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.263692    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls podName:4bc93ff1-5693-48f6-8910-3820a3d18d1a nodeName:}" failed. No retries permitted until 2026-06-02 15:47:12.263674148 +0000 UTC m=+38.161394518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls") pod "image-registry-574b6f686f-zdxlk" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a") : secret "image-registry-tls" not found
Jun 02 15:47:10.263902 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.263882    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:10.264024 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.263975    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert podName:eff42c24-b415-4b7e-b900-3a00e7183c53 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:12.263956986 +0000 UTC m=+38.161677353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert") pod "ingress-canary-wbb7q" (UID: "eff42c24-b415-4b7e-b900-3a00e7183c53") : secret "canary-serving-cert" not found
Jun 02 15:47:10.364649 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:10.364608    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:10.364845 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.364771    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:10.364938 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:10.364848    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls podName:a7be4dd9-3fd8-4a8f-9697-785ae760e14e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:12.364828822 +0000 UTC m=+38.262549194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls") pod "dns-default-bqm8z" (UID: "a7be4dd9-3fd8-4a8f-9697-785ae760e14e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:11.847994 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:11.847896    2562 generic.go:358] "Generic (PLEG): container finished" podID="bf747868-df96-43d9-8e2b-e0b6cb56045b" containerID="b6cfed3cfef1d94d1587919783a3e0c7d1df8bf86d1b6b18c69d8a17de50f194" exitCode=0
Jun 02 15:47:11.847994 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:11.847945    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerDied","Data":"b6cfed3cfef1d94d1587919783a3e0c7d1df8bf86d1b6b18c69d8a17de50f194"}
Jun 02 15:47:12.280771 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:12.280588    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:12.280966 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:12.280813    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:12.280966 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.280731    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:12.280966 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.280887    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert podName:eff42c24-b415-4b7e-b900-3a00e7183c53 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:16.280869747 +0000 UTC m=+42.178590116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert") pod "ingress-canary-wbb7q" (UID: "eff42c24-b415-4b7e-b900-3a00e7183c53") : secret "canary-serving-cert" not found
Jun 02 15:47:12.280966 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.280935    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:47:12.280966 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.280947    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574b6f686f-zdxlk: secret "image-registry-tls" not found
Jun 02 15:47:12.281140 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.280989    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls podName:4bc93ff1-5693-48f6-8910-3820a3d18d1a nodeName:}" failed. No retries permitted until 2026-06-02 15:47:16.28097689 +0000 UTC m=+42.178697260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls") pod "image-registry-574b6f686f-zdxlk" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a") : secret "image-registry-tls" not found
Jun 02 15:47:12.381644 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:12.381606    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:12.381807 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.381743    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:12.381844 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:12.381806    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls podName:a7be4dd9-3fd8-4a8f-9697-785ae760e14e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:16.381791756 +0000 UTC m=+42.279512122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls") pod "dns-default-bqm8z" (UID: "a7be4dd9-3fd8-4a8f-9697-785ae760e14e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:12.853061 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:12.853025    2562 generic.go:358] "Generic (PLEG): container finished" podID="bf747868-df96-43d9-8e2b-e0b6cb56045b" containerID="5f99180a66dccd7bbf042c3756601d2e0b28475befa725ea202b608924751ce5" exitCode=0
Jun 02 15:47:12.853379 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:12.853093    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerDied","Data":"5f99180a66dccd7bbf042c3756601d2e0b28475befa725ea202b608924751ce5"}
Jun 02 15:47:13.857975 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:13.857940    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" event={"ID":"bf747868-df96-43d9-8e2b-e0b6cb56045b","Type":"ContainerStarted","Data":"0906a2c44766a81fa2ed76288845e410ab037ba513b6b62e0c731737d3b854d3"}
Jun 02 15:47:13.889449 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:13.889402    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wtc7p" podStartSLOduration=5.455432588 podStartE2EDuration="39.889387749s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:46:37.030359936 +0000 UTC m=+2.928080315" lastFinishedPulling="2026-06-02 15:47:11.464315111 +0000 UTC m=+37.362035476" observedRunningTime="2026-06-02 15:47:13.886347462 +0000 UTC m=+39.784067849" watchObservedRunningTime="2026-06-02 15:47:13.889387749 +0000 UTC m=+39.787108136"
Jun 02 15:47:16.309323 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:16.309293    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:16.309780 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:16.309357    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:16.309780 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.309455    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:16.309780 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.309517    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert podName:eff42c24-b415-4b7e-b900-3a00e7183c53 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:24.309503089 +0000 UTC m=+50.207223461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert") pod "ingress-canary-wbb7q" (UID: "eff42c24-b415-4b7e-b900-3a00e7183c53") : secret "canary-serving-cert" not found
Jun 02 15:47:16.309780 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.309458    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:47:16.309780 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.309563    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574b6f686f-zdxlk: secret "image-registry-tls" not found
Jun 02 15:47:16.309780 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.309600    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls podName:4bc93ff1-5693-48f6-8910-3820a3d18d1a nodeName:}" failed. No retries permitted until 2026-06-02 15:47:24.30958979 +0000 UTC m=+50.207310169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls") pod "image-registry-574b6f686f-zdxlk" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a") : secret "image-registry-tls" not found
Jun 02 15:47:16.409763 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:16.409729    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:16.409894 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.409874    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:16.409979 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:16.409952    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls podName:a7be4dd9-3fd8-4a8f-9697-785ae760e14e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:24.409938422 +0000 UTC m=+50.307658787 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls") pod "dns-default-bqm8z" (UID: "a7be4dd9-3fd8-4a8f-9697-785ae760e14e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:17.704529 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:17.704502    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sdcph_0bb9f541-1297-48c3-87f1-765335e0b0eb/dns-node-resolver/0.log"
Jun 02 15:47:18.709811 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.709782    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfwt7_5d858e0a-17c3-4bc8-b185-1c981b666127/node-ca/0.log"
Jun 02 15:47:18.733029 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.732999    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-4gqjc"]
Jun 02 15:47:18.737351 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.737331    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:18.742344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.742324    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jun 02 15:47:18.742688 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.742672    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jun 02 15:47:18.742790 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.742703    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tj62d\""
Jun 02 15:47:18.742961 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.742943    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jun 02 15:47:18.743020 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.742943    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jun 02 15:47:18.751188 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.751166    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-4gqjc"]
Jun 02 15:47:18.926986 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.926957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-signing-key\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:18.927182 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.927033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-signing-cabundle\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:18.927182 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:18.927083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnxlp\" (UniqueName: \"kubernetes.io/projected/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-kube-api-access-pnxlp\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.028434 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.028343    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-signing-key\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.028566 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.028515    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-signing-cabundle\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.028630 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.028569    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pnxlp\" (UniqueName: \"kubernetes.io/projected/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-kube-api-access-pnxlp\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.029166 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.029148    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-signing-cabundle\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.031410 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.031387    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-signing-key\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.039703 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.039681    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnxlp\" (UniqueName: \"kubernetes.io/projected/dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76-kube-api-access-pnxlp\") pod \"service-ca-685df94d57-4gqjc\" (UID: \"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76\") " pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.046060 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.046041    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-4gqjc"
Jun 02 15:47:19.180171 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.180137    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-4gqjc"]
Jun 02 15:47:19.183247 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:19.183221    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2f2f70_f6cd_45d3_8ed4_f29e47ee3b76.slice/crio-f13f0f58dcb3543847b345d7c449bca76ce1d119a33d60895723d237052e6a5d WatchSource:0}: Error finding container f13f0f58dcb3543847b345d7c449bca76ce1d119a33d60895723d237052e6a5d: Status 404 returned error can't find the container with id f13f0f58dcb3543847b345d7c449bca76ce1d119a33d60895723d237052e6a5d
Jun 02 15:47:19.869670 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:19.869633    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-4gqjc" event={"ID":"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76","Type":"ContainerStarted","Data":"f13f0f58dcb3543847b345d7c449bca76ce1d119a33d60895723d237052e6a5d"}
Jun 02 15:47:21.874315 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:21.874283    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-4gqjc" event={"ID":"dd2f2f70-f6cd-45d3-8ed4-f29e47ee3b76","Type":"ContainerStarted","Data":"44d548f48a44c723a4c987741c86139b77d7a6827b334194d443c45fe39051db"}
Jun 02 15:47:21.894096 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:21.894029    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-4gqjc" podStartSLOduration=2.074201014 podStartE2EDuration="3.894011385s" podCreationTimestamp="2026-06-02 15:47:18 +0000 UTC" firstStartedPulling="2026-06-02 15:47:19.185044921 +0000 UTC m=+45.082765288" lastFinishedPulling="2026-06-02 15:47:21.004855289 +0000 UTC m=+46.902575659" observedRunningTime="2026-06-02 15:47:21.893372752 +0000 UTC m=+47.791093141" watchObservedRunningTime="2026-06-02 15:47:21.894011385 +0000 UTC m=+47.791731780"
Jun 02 15:47:24.374090 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:24.374043    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:24.374575 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:24.374137    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") pod \"image-registry-574b6f686f-zdxlk\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") " pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:24.374575 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.374191    2562 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:24.374575 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.374268    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert podName:eff42c24-b415-4b7e-b900-3a00e7183c53 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:40.374252452 +0000 UTC m=+66.271972824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert") pod "ingress-canary-wbb7q" (UID: "eff42c24-b415-4b7e-b900-3a00e7183c53") : secret "canary-serving-cert" not found
Jun 02 15:47:24.374575 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.374285    2562 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:47:24.374575 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.374298    2562 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574b6f686f-zdxlk: secret "image-registry-tls" not found
Jun 02 15:47:24.374575 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.374358    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls podName:4bc93ff1-5693-48f6-8910-3820a3d18d1a nodeName:}" failed. No retries permitted until 2026-06-02 15:47:40.374344397 +0000 UTC m=+66.272064775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls") pod "image-registry-574b6f686f-zdxlk" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a") : secret "image-registry-tls" not found
Jun 02 15:47:24.474722 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:24.474681    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:24.474895 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.474832    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:24.474969 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:24.474899    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls podName:a7be4dd9-3fd8-4a8f-9697-785ae760e14e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:40.474881702 +0000 UTC m=+66.372602071 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls") pod "dns-default-bqm8z" (UID: "a7be4dd9-3fd8-4a8f-9697-785ae760e14e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:34.848477 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:34.848450    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qvffn"
Jun 02 15:47:38.491143 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.491107    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-574b6f686f-zdxlk"]
Jun 02 15:47:38.491519 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:38.491278    2562 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-574b6f686f-zdxlk" podUID="4bc93ff1-5693-48f6-8910-3820a3d18d1a"
Jun 02 15:47:38.641197 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.641160    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c44974544-6nsl4"]
Jun 02 15:47:38.648679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.648653    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.659007 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.658983    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c44974544-6nsl4"]
Jun 02 15:47:38.659580 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.659553    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jcv2c"]
Jun 02 15:47:38.662459 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.662436    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.664608 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.664592    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tl6r5\""
Jun 02 15:47:38.665947 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.665903    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 02 15:47:38.666051 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.665975    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 02 15:47:38.666051 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.666045    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 02 15:47:38.666286 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.666269    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 02 15:47:38.680637 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3389aba7-0fa7-43fe-b5b9-994861a49a21-registry-certificates\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.680814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3389aba7-0fa7-43fe-b5b9-994861a49a21-ca-trust-extracted\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.680814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-registry-tls\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.680814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxnfp\" (UniqueName: \"kubernetes.io/projected/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-kube-api-access-sxnfp\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.680814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680749    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3389aba7-0fa7-43fe-b5b9-994861a49a21-installation-pull-secrets\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.680814 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680806    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-data-volume\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.681041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680854    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-bound-sa-token\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.681041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680885    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75mtj\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-kube-api-access-75mtj\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.681041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.680960    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3389aba7-0fa7-43fe-b5b9-994861a49a21-trusted-ca\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.681041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.681032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3389aba7-0fa7-43fe-b5b9-994861a49a21-image-registry-private-configuration\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.681181 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.681091    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.681181 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.681167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-crio-socket\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.681256 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.681214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.681742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.681720    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jcv2c"]
Jun 02 15:47:38.782452 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782353    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.782452 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782415    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3389aba7-0fa7-43fe-b5b9-994861a49a21-registry-certificates\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.782661 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782455    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3389aba7-0fa7-43fe-b5b9-994861a49a21-ca-trust-extracted\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.782661 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782495    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-registry-tls\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.782661 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782519    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sxnfp\" (UniqueName: \"kubernetes.io/projected/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-kube-api-access-sxnfp\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.782661 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782550    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3389aba7-0fa7-43fe-b5b9-994861a49a21-installation-pull-secrets\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.782661 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782587    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-data-volume\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.782661 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782637    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-bound-sa-token\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.783005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782687    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75mtj\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-kube-api-access-75mtj\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.783005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782725    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3389aba7-0fa7-43fe-b5b9-994861a49a21-trusted-ca\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.783005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782757    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3389aba7-0fa7-43fe-b5b9-994861a49a21-image-registry-private-configuration\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.783005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782794    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.783005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782829    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-crio-socket\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.783005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.782952    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-crio-socket\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.783306 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.783006    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3389aba7-0fa7-43fe-b5b9-994861a49a21-ca-trust-extracted\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.783306 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.783012    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-data-volume\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.783453 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.783423    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3389aba7-0fa7-43fe-b5b9-994861a49a21-registry-certificates\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.783584 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.783550    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.784130 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.784103    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3389aba7-0fa7-43fe-b5b9-994861a49a21-trusted-ca\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.785091 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.785065    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.785280 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.785262    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3389aba7-0fa7-43fe-b5b9-994861a49a21-image-registry-private-configuration\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.785392 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.785306    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-registry-tls\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.785525 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.785506    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3389aba7-0fa7-43fe-b5b9-994861a49a21-installation-pull-secrets\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.794642 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.794609    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-bound-sa-token\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.794773 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.794643    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxnfp\" (UniqueName: \"kubernetes.io/projected/bb5e1206-06b4-4509-89d2-e4b6cdc5d966-kube-api-access-sxnfp\") pod \"insights-runtime-extractor-jcv2c\" (UID: \"bb5e1206-06b4-4509-89d2-e4b6cdc5d966\") " pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.794773 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.794685    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75mtj\" (UniqueName: \"kubernetes.io/projected/3389aba7-0fa7-43fe-b5b9-994861a49a21-kube-api-access-75mtj\") pod \"image-registry-7c44974544-6nsl4\" (UID: \"3389aba7-0fa7-43fe-b5b9-994861a49a21\") " pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.908005 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.907976    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:38.912600 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.912580    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:38.959443 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.959409    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vhj6z\""
Jun 02 15:47:38.968370 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.968343    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:38.971141 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.971117    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jcv2c"
Jun 02 15:47:38.984006 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.983980    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-trusted-ca\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984035    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-installation-pull-secrets\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984080    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-certificates\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984111    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-image-registry-private-configuration\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984293 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984141    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-bound-sa-token\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984293 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984165    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4bc93ff1-5693-48f6-8910-3820a3d18d1a-ca-trust-extracted\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984293 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984192    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg6nj\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-kube-api-access-qg6nj\") pod \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\" (UID: \"4bc93ff1-5693-48f6-8910-3820a3d18d1a\") "
Jun 02 15:47:38.984484 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984454    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:47:38.984837 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984771    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc93ff1-5693-48f6-8910-3820a3d18d1a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:47:38.984941 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.984848    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:47:38.987115 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.987075    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:47:38.987201 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.987079    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-kube-api-access-qg6nj" (OuterVolumeSpecName: "kube-api-access-qg6nj") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "kube-api-access-qg6nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:47:38.987201 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.987128    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:47:38.987370 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:38.987350    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4bc93ff1-5693-48f6-8910-3820a3d18d1a" (UID: "4bc93ff1-5693-48f6-8910-3820a3d18d1a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:47:39.085482 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085447    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-trusted-ca\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.085482 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085485    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-installation-pull-secrets\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.085657 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085501    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-certificates\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.085657 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085516    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4bc93ff1-5693-48f6-8910-3820a3d18d1a-image-registry-private-configuration\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.085657 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085534    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-bound-sa-token\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.085657 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085547    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4bc93ff1-5693-48f6-8910-3820a3d18d1a-ca-trust-extracted\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.085657 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.085559    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg6nj\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-kube-api-access-qg6nj\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:39.128979 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.128944    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jcv2c"]
Jun 02 15:47:39.132649 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:39.132621    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5e1206_06b4_4509_89d2_e4b6cdc5d966.slice/crio-9a0f07fadf6398318197f520156d186571ecbc5d94519c44fb322f7f70b0c137 WatchSource:0}: Error finding container 9a0f07fadf6398318197f520156d186571ecbc5d94519c44fb322f7f70b0c137: Status 404 returned error can't find the container with id 9a0f07fadf6398318197f520156d186571ecbc5d94519c44fb322f7f70b0c137
Jun 02 15:47:39.134022 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.133996    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c44974544-6nsl4"]
Jun 02 15:47:39.138287 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:39.138263    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3389aba7_0fa7_43fe_b5b9_994861a49a21.slice/crio-de437548bfaac4e2ef9700ab63a9afc121ec7164ea6eede2e9ea8c8d74a6142c WatchSource:0}: Error finding container de437548bfaac4e2ef9700ab63a9afc121ec7164ea6eede2e9ea8c8d74a6142c: Status 404 returned error can't find the container with id de437548bfaac4e2ef9700ab63a9afc121ec7164ea6eede2e9ea8c8d74a6142c
Jun 02 15:47:39.911423 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.911389    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c44974544-6nsl4" event={"ID":"3389aba7-0fa7-43fe-b5b9-994861a49a21","Type":"ContainerStarted","Data":"7f6a4c2a54c1d9b0473e56bf4776f867c3aa8ff78b8f7e0026c0c25ed19833ca"}
Jun 02 15:47:39.911423 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.911431    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c44974544-6nsl4" event={"ID":"3389aba7-0fa7-43fe-b5b9-994861a49a21","Type":"ContainerStarted","Data":"de437548bfaac4e2ef9700ab63a9afc121ec7164ea6eede2e9ea8c8d74a6142c"}
Jun 02 15:47:39.911929 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.911463    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:47:39.913091 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.913076    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574b6f686f-zdxlk"
Jun 02 15:47:39.913169 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.913089    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcv2c" event={"ID":"bb5e1206-06b4-4509-89d2-e4b6cdc5d966","Type":"ContainerStarted","Data":"65125dbcc1ca3331d73649b75dc32aba0deb1666dd2726bbe41c4281bf7e96b4"}
Jun 02 15:47:39.913169 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.913116    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcv2c" event={"ID":"bb5e1206-06b4-4509-89d2-e4b6cdc5d966","Type":"ContainerStarted","Data":"28ea1846ab81b15ed99c22e87d197d4894cd0d073c0d11a69ac83e86900736bd"}
Jun 02 15:47:39.913169 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.913130    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcv2c" event={"ID":"bb5e1206-06b4-4509-89d2-e4b6cdc5d966","Type":"ContainerStarted","Data":"9a0f07fadf6398318197f520156d186571ecbc5d94519c44fb322f7f70b0c137"}
Jun 02 15:47:39.940342 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.940284    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c44974544-6nsl4" podStartSLOduration=1.940266362 podStartE2EDuration="1.940266362s" podCreationTimestamp="2026-06-02 15:47:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:39.940066233 +0000 UTC m=+65.837786620" watchObservedRunningTime="2026-06-02 15:47:39.940266362 +0000 UTC m=+65.837986732"
Jun 02 15:47:39.979059 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.977364    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-574b6f686f-zdxlk"]
Jun 02 15:47:39.986088 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:39.986055    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-574b6f686f-zdxlk"]
Jun 02 15:47:40.094675 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.094583    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4bc93ff1-5693-48f6-8910-3820a3d18d1a-registry-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:47:40.396511 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.396426    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:40.396511 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.396473    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:40.396769 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.396597    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:40.396769 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.396647    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:40.398827 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.398799    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eff42c24-b415-4b7e-b900-3a00e7183c53-cert\") pod \"ingress-canary-wbb7q\" (UID: \"eff42c24-b415-4b7e-b900-3a00e7183c53\") " pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:40.401292 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.401274    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 15:47:40.402041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.402015    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 15:47:40.402163 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.402060    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 02 15:47:40.409131 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.409102    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a83d011-679c-4bab-a182-b89f20266181-original-pull-secret\") pod \"global-pull-secret-syncer-pzdmg\" (UID: \"4a83d011-679c-4bab-a182-b89f20266181\") " pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:40.409247 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.409151    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54bcdc8-8c47-4ede-9def-196f262b8a29-metrics-certs\") pod \"network-metrics-daemon-jjrxf\" (UID: \"d54bcdc8-8c47-4ede-9def-196f262b8a29\") " pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:40.409247 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.409229    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 15:47:40.416922 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.416889    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzdmg"
Jun 02 15:47:40.419608 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.419584    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68mn9\" (UniqueName: \"kubernetes.io/projected/a6572f3a-8d66-4e30-9ec0-138977fcfaca-kube-api-access-68mn9\") pod \"network-check-target-r7254\" (UID: \"a6572f3a-8d66-4e30-9ec0-138977fcfaca\") " pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:40.498139 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.498103    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:40.501092 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.501064    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7be4dd9-3fd8-4a8f-9697-785ae760e14e-metrics-tls\") pod \"dns-default-bqm8z\" (UID: \"a7be4dd9-3fd8-4a8f-9697-785ae760e14e\") " pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:40.511388 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.511358    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vmzq\""
Jun 02 15:47:40.519372 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.519346    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjrxf"
Jun 02 15:47:40.549386 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.549327    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pzdmg"]
Jun 02 15:47:40.553398 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:40.553352    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a83d011_679c_4bab_a182_b89f20266181.slice/crio-ddf5213180d15265f679a0469f773e24effc4381a51484d4ee4bec065b78b1db WatchSource:0}: Error finding container ddf5213180d15265f679a0469f773e24effc4381a51484d4ee4bec065b78b1db: Status 404 returned error can't find the container with id ddf5213180d15265f679a0469f773e24effc4381a51484d4ee4bec065b78b1db
Jun 02 15:47:40.599697 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.599665    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc93ff1-5693-48f6-8910-3820a3d18d1a" path="/var/lib/kubelet/pods/4bc93ff1-5693-48f6-8910-3820a3d18d1a/volumes"
Jun 02 15:47:40.668987 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.668897    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ws8nx\""
Jun 02 15:47:40.675899 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.675851    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jjrxf"]
Jun 02 15:47:40.676642 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.676596    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbb7q"
Jun 02 15:47:40.680035 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:40.680008    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54bcdc8_8c47_4ede_9def_196f262b8a29.slice/crio-19ebb63ada695fbbd8fe8ae49bbb69c6592bae956d8271718ee2a4c080a09174 WatchSource:0}: Error finding container 19ebb63ada695fbbd8fe8ae49bbb69c6592bae956d8271718ee2a4c080a09174: Status 404 returned error can't find the container with id 19ebb63ada695fbbd8fe8ae49bbb69c6592bae956d8271718ee2a4c080a09174
Jun 02 15:47:40.690426 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.690407    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6cp8\""
Jun 02 15:47:40.699125 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.699059    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:40.714855 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.714825    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6qg5h\""
Jun 02 15:47:40.721854 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.721822    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:40.841575 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.841518    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbb7q"]
Jun 02 15:47:40.847176 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:40.847023    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff42c24_b415_4b7e_b900_3a00e7183c53.slice/crio-7bf4cc789509671e6d43cb908d08b9a6a0e6bb4a537f6804c9ebae2bd0bafea3 WatchSource:0}: Error finding container 7bf4cc789509671e6d43cb908d08b9a6a0e6bb4a537f6804c9ebae2bd0bafea3: Status 404 returned error can't find the container with id 7bf4cc789509671e6d43cb908d08b9a6a0e6bb4a537f6804c9ebae2bd0bafea3
Jun 02 15:47:40.869727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.869696    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bqm8z"]
Jun 02 15:47:40.874002 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:40.873957    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7be4dd9_3fd8_4a8f_9697_785ae760e14e.slice/crio-eee0ceccdac29628c24bd327d2e08e8c646c979d984fa857dd1d998b446f90f5 WatchSource:0}: Error finding container eee0ceccdac29628c24bd327d2e08e8c646c979d984fa857dd1d998b446f90f5: Status 404 returned error can't find the container with id eee0ceccdac29628c24bd327d2e08e8c646c979d984fa857dd1d998b446f90f5
Jun 02 15:47:40.904341 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.904311    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r7254"]
Jun 02 15:47:40.908021 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:40.907993    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6572f3a_8d66_4e30_9ec0_138977fcfaca.slice/crio-b53c969f1ba2bae3fcc5d986fb7644f794f765b8caa304bce2a5457127bae199 WatchSource:0}: Error finding container b53c969f1ba2bae3fcc5d986fb7644f794f765b8caa304bce2a5457127bae199: Status 404 returned error can't find the container with id b53c969f1ba2bae3fcc5d986fb7644f794f765b8caa304bce2a5457127bae199
Jun 02 15:47:40.916029 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.916001    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bqm8z" event={"ID":"a7be4dd9-3fd8-4a8f-9697-785ae760e14e","Type":"ContainerStarted","Data":"eee0ceccdac29628c24bd327d2e08e8c646c979d984fa857dd1d998b446f90f5"}
Jun 02 15:47:40.917080 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.917059    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjrxf" event={"ID":"d54bcdc8-8c47-4ede-9def-196f262b8a29","Type":"ContainerStarted","Data":"19ebb63ada695fbbd8fe8ae49bbb69c6592bae956d8271718ee2a4c080a09174"}
Jun 02 15:47:40.918054 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.918036    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r7254" event={"ID":"a6572f3a-8d66-4e30-9ec0-138977fcfaca","Type":"ContainerStarted","Data":"b53c969f1ba2bae3fcc5d986fb7644f794f765b8caa304bce2a5457127bae199"}
Jun 02 15:47:40.919049 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.918991    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbb7q" event={"ID":"eff42c24-b415-4b7e-b900-3a00e7183c53","Type":"ContainerStarted","Data":"7bf4cc789509671e6d43cb908d08b9a6a0e6bb4a537f6804c9ebae2bd0bafea3"}
Jun 02 15:47:40.920053 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:40.920033    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pzdmg" event={"ID":"4a83d011-679c-4bab-a182-b89f20266181","Type":"ContainerStarted","Data":"ddf5213180d15265f679a0469f773e24effc4381a51484d4ee4bec065b78b1db"}
Jun 02 15:47:41.925873 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:41.925787    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcv2c" event={"ID":"bb5e1206-06b4-4509-89d2-e4b6cdc5d966","Type":"ContainerStarted","Data":"d816a9925dc1895bf07c7c5b2b91a32dddc6e36be19b4b48942be4bb5931a7bb"}
Jun 02 15:47:44.085946 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.085805    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jcv2c" podStartSLOduration=3.768415825 podStartE2EDuration="6.085785739s" podCreationTimestamp="2026-06-02 15:47:38 +0000 UTC" firstStartedPulling="2026-06-02 15:47:39.19318433 +0000 UTC m=+65.090904696" lastFinishedPulling="2026-06-02 15:47:41.510554232 +0000 UTC m=+67.408274610" observedRunningTime="2026-06-02 15:47:41.951806551 +0000 UTC m=+67.849526953" watchObservedRunningTime="2026-06-02 15:47:44.085785739 +0000 UTC m=+69.983506132"
Jun 02 15:47:44.087047 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.087022    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"]
Jun 02 15:47:44.091676 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.091651    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.101962 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.101936    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jun 02 15:47:44.102081 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.102017    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t5zlr\""
Jun 02 15:47:44.102464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.102438    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jun 02 15:47:44.102597 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.102505    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 02 15:47:44.102597 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.102517    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 02 15:47:44.102924 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.102894    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 02 15:47:44.107805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.107779    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"]
Jun 02 15:47:44.125942 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.125894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4ff345a-bfac-4494-a4ef-da9a56a5821d-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.126109 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.125958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjksb\" (UniqueName: \"kubernetes.io/projected/b4ff345a-bfac-4494-a4ef-da9a56a5821d-kube-api-access-wjksb\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.126109 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.126014    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.126228 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.126112    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.227372 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.227338    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4ff345a-bfac-4494-a4ef-da9a56a5821d-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.227549 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.227394    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjksb\" (UniqueName: \"kubernetes.io/projected/b4ff345a-bfac-4494-a4ef-da9a56a5821d-kube-api-access-wjksb\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.227549 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.227450    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.227549 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.227528    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.227741 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:44.227596    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jun 02 15:47:44.227741 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:44.227662    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-tls podName:b4ff345a-bfac-4494-a4ef-da9a56a5821d nodeName:}" failed. No retries permitted until 2026-06-02 15:47:44.72764051 +0000 UTC m=+70.625360898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-tls") pod "prometheus-operator-658bcd8bdf-f9bql" (UID: "b4ff345a-bfac-4494-a4ef-da9a56a5821d") : secret "prometheus-operator-tls" not found
Jun 02 15:47:44.228180 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.228129    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4ff345a-bfac-4494-a4ef-da9a56a5821d-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.230476 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.230446    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.238987 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.238960    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjksb\" (UniqueName: \"kubernetes.io/projected/b4ff345a-bfac-4494-a4ef-da9a56a5821d-kube-api-access-wjksb\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.731164 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.731125    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:44.734030 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:44.733992    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4ff345a-bfac-4494-a4ef-da9a56a5821d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-f9bql\" (UID: \"b4ff345a-bfac-4494-a4ef-da9a56a5821d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:45.003235 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:45.003144    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"
Jun 02 15:47:46.332627 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.332040    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql"]
Jun 02 15:47:46.344721 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:46.344686    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ff345a_bfac_4494_a4ef_da9a56a5821d.slice/crio-47c837b59a5181f8303a0ef6a639cf18df94625fa03776da121b26162788e379 WatchSource:0}: Error finding container 47c837b59a5181f8303a0ef6a639cf18df94625fa03776da121b26162788e379: Status 404 returned error can't find the container with id 47c837b59a5181f8303a0ef6a639cf18df94625fa03776da121b26162788e379
Jun 02 15:47:46.940097 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.940055    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbb7q" event={"ID":"eff42c24-b415-4b7e-b900-3a00e7183c53","Type":"ContainerStarted","Data":"8acf4ec3cac57daae0da4711961208d141a44752ade2c10d40b1ac8eade2c8ff"}
Jun 02 15:47:46.941552 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.941510    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pzdmg" event={"ID":"4a83d011-679c-4bab-a182-b89f20266181","Type":"ContainerStarted","Data":"352eb04a0729dd56f5adbdf3fee3c5f1a32db319ea376c465529ee2aee2df75b"}
Jun 02 15:47:46.943167 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.943142    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bqm8z" event={"ID":"a7be4dd9-3fd8-4a8f-9697-785ae760e14e","Type":"ContainerStarted","Data":"081b4e1cc3ffb80915974ad6d997906ad7b9c24be026e5da26690121869617eb"}
Jun 02 15:47:46.943167 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.943168    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bqm8z" event={"ID":"a7be4dd9-3fd8-4a8f-9697-785ae760e14e","Type":"ContainerStarted","Data":"607cfc8f912aca45e7956265d1d56d5476a00726e3ff5261ed14b88580ffab29"}
Jun 02 15:47:46.943354 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.943232    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:46.944971 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.944940    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjrxf" event={"ID":"d54bcdc8-8c47-4ede-9def-196f262b8a29","Type":"ContainerStarted","Data":"3f2a498a8ce605cc3fe0426ce8072e42e659a69ac832d4543d2b45df6493012c"}
Jun 02 15:47:46.945089 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.944977    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjrxf" event={"ID":"d54bcdc8-8c47-4ede-9def-196f262b8a29","Type":"ContainerStarted","Data":"8343be4cf8cb1f023fb29673e6dcc4a1cc8caa914c07ae3d53d4d03132d22555"}
Jun 02 15:47:46.946172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.946148    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql" event={"ID":"b4ff345a-bfac-4494-a4ef-da9a56a5821d","Type":"ContainerStarted","Data":"47c837b59a5181f8303a0ef6a639cf18df94625fa03776da121b26162788e379"}
Jun 02 15:47:46.947427 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.947409    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r7254" event={"ID":"a6572f3a-8d66-4e30-9ec0-138977fcfaca","Type":"ContainerStarted","Data":"93ed797818dd82f3ab4cc61de19e9d0dae8d90ecdc93403b6692146d8f07c9b4"}
Jun 02 15:47:46.947595 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.947575    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:47:46.979763 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:46.979717    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wbb7q" podStartSLOduration=33.674426258 podStartE2EDuration="38.979699668s" podCreationTimestamp="2026-06-02 15:47:08 +0000 UTC" firstStartedPulling="2026-06-02 15:47:40.848906751 +0000 UTC m=+66.746627121" lastFinishedPulling="2026-06-02 15:47:46.15418015 +0000 UTC m=+72.051900531" observedRunningTime="2026-06-02 15:47:46.97847132 +0000 UTC m=+72.876191708" watchObservedRunningTime="2026-06-02 15:47:46.979699668 +0000 UTC m=+72.877420056"
Jun 02 15:47:47.023079 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.023014    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-bqm8z" podStartSLOduration=33.744801875 podStartE2EDuration="39.022999033s" podCreationTimestamp="2026-06-02 15:47:08 +0000 UTC" firstStartedPulling="2026-06-02 15:47:40.875984146 +0000 UTC m=+66.773704512" lastFinishedPulling="2026-06-02 15:47:46.154181304 +0000 UTC m=+72.051901670" observedRunningTime="2026-06-02 15:47:47.02197952 +0000 UTC m=+72.919699910" watchObservedRunningTime="2026-06-02 15:47:47.022999033 +0000 UTC m=+72.920719422"
Jun 02 15:47:47.046863 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.046760    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jjrxf" podStartSLOduration=67.622439355 podStartE2EDuration="1m13.046744065s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:47:40.68258433 +0000 UTC m=+66.580304717" lastFinishedPulling="2026-06-02 15:47:46.10688905 +0000 UTC m=+72.004609427" observedRunningTime="2026-06-02 15:47:47.045855074 +0000 UTC m=+72.943575463" watchObservedRunningTime="2026-06-02 15:47:47.046744065 +0000 UTC m=+72.944464453"
Jun 02 15:47:47.080432 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.080383    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r7254" podStartSLOduration=67.820589399 podStartE2EDuration="1m13.080368439s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:47:40.909826211 +0000 UTC m=+66.807546576" lastFinishedPulling="2026-06-02 15:47:46.169605228 +0000 UTC m=+72.067325616" observedRunningTime="2026-06-02 15:47:47.078116835 +0000 UTC m=+72.975837223" watchObservedRunningTime="2026-06-02 15:47:47.080368439 +0000 UTC m=+72.978088825"
Jun 02 15:47:47.100789 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.100718    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pzdmg" podStartSLOduration=65.47613702 podStartE2EDuration="1m11.100700953s" podCreationTimestamp="2026-06-02 15:46:36 +0000 UTC" firstStartedPulling="2026-06-02 15:47:40.555609835 +0000 UTC m=+66.453330204" lastFinishedPulling="2026-06-02 15:47:46.180173764 +0000 UTC m=+72.077894137" observedRunningTime="2026-06-02 15:47:47.099256022 +0000 UTC m=+72.996976410" watchObservedRunningTime="2026-06-02 15:47:47.100700953 +0000 UTC m=+72.998421404"
Jun 02 15:47:47.224041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.223959    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-dfd8c5f65-l28xk"]
Jun 02 15:47:47.227374 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.227352    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.230637 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.230574    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jun 02 15:47:47.230772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.230725    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 02 15:47:47.230835 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.230810    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pwqm5\""
Jun 02 15:47:47.230902 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.230886    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jun 02 15:47:47.231144 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.231063    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jun 02 15:47:47.231144 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.231076    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jun 02 15:47:47.231350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.231321    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jun 02 15:47:47.231504 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.231479    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 02 15:47:47.242094 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.242062    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfd8c5f65-l28xk"]
Jun 02 15:47:47.352273 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.352230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-service-ca\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.352742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.352283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-oauth-config\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.352742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.352374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-oauth-serving-cert\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.352742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.352437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9n7q\" (UniqueName: \"kubernetes.io/projected/54faf369-6448-45f4-ae9f-64a3e7f53eaa-kube-api-access-x9n7q\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.352742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.352491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-config\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.352742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.352550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-serving-cert\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.453825 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.453783    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-config\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454034 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.453843    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-serving-cert\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454034 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.453872    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-service-ca\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454034 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.453899    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-oauth-config\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454034 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.453940    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-oauth-serving-cert\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454034 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.453981    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x9n7q\" (UniqueName: \"kubernetes.io/projected/54faf369-6448-45f4-ae9f-64a3e7f53eaa-kube-api-access-x9n7q\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454678 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.454643    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-oauth-serving-cert\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454789 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.454643    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-config\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.454789 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.454721    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-service-ca\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.456615 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.456585    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-oauth-config\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.456851 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.456831    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-serving-cert\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.465079 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.465054    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9n7q\" (UniqueName: \"kubernetes.io/projected/54faf369-6448-45f4-ae9f-64a3e7f53eaa-kube-api-access-x9n7q\") pod \"console-dfd8c5f65-l28xk\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") " pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.538376 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.538354    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:47.683203 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.683138    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfd8c5f65-l28xk"]
Jun 02 15:47:47.687088 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:47.687050    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54faf369_6448_45f4_ae9f_64a3e7f53eaa.slice/crio-da6c94a7ad9f1d9d59cd6524b2e86ddb9a1af8704868a42b7338ce822397c34f WatchSource:0}: Error finding container da6c94a7ad9f1d9d59cd6524b2e86ddb9a1af8704868a42b7338ce822397c34f: Status 404 returned error can't find the container with id da6c94a7ad9f1d9d59cd6524b2e86ddb9a1af8704868a42b7338ce822397c34f
Jun 02 15:47:47.951515 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.951476    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql" event={"ID":"b4ff345a-bfac-4494-a4ef-da9a56a5821d","Type":"ContainerStarted","Data":"badea680e65fd801aace1ffea29a6367813a796dd28778ef0a8a97400efc1f2d"}
Jun 02 15:47:47.951515 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.951517    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql" event={"ID":"b4ff345a-bfac-4494-a4ef-da9a56a5821d","Type":"ContainerStarted","Data":"68d04633a3e48c1872bb3cfa1b64dfad98d62d0bb8e485de478e39f6c8a7ab22"}
Jun 02 15:47:47.952639 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.952608    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfd8c5f65-l28xk" event={"ID":"54faf369-6448-45f4-ae9f-64a3e7f53eaa","Type":"ContainerStarted","Data":"da6c94a7ad9f1d9d59cd6524b2e86ddb9a1af8704868a42b7338ce822397c34f"}
Jun 02 15:47:47.974961 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:47.974887    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-f9bql" podStartSLOduration=2.793285707 podStartE2EDuration="3.974872018s" podCreationTimestamp="2026-06-02 15:47:44 +0000 UTC" firstStartedPulling="2026-06-02 15:47:46.349634113 +0000 UTC m=+72.247354493" lastFinishedPulling="2026-06-02 15:47:47.531220434 +0000 UTC m=+73.428940804" observedRunningTime="2026-06-02 15:47:47.973456517 +0000 UTC m=+73.871176906" watchObservedRunningTime="2026-06-02 15:47:47.974872018 +0000 UTC m=+73.872592406"
Jun 02 15:47:49.750329 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.750284    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x8vhv"]
Jun 02 15:47:49.754049 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.753992    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.756369 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.756337    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 02 15:47:49.756608 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.756578    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 02 15:47:49.756741 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.756679    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 02 15:47:49.756839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.756824    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zfv5r\""
Jun 02 15:47:49.876060 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876021    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-sys\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876231 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccc88934-7c14-4802-aee3-cb92ed967f0b-metrics-client-ca\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876231 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876231 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-wtmp\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876231 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876206    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdh2p\" (UniqueName: \"kubernetes.io/projected/ccc88934-7c14-4802-aee3-cb92ed967f0b-kube-api-access-jdh2p\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876420 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-tls\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876420 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-textfile\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876420 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.876420 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.876359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-root\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.976808 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.976769    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-sys\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977012 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.976818    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccc88934-7c14-4802-aee3-cb92ed967f0b-metrics-client-ca\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977012 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.976856    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977012 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.976880    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-sys\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977012 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.976888    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-wtmp\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977012 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.976989    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jdh2p\" (UniqueName: \"kubernetes.io/projected/ccc88934-7c14-4802-aee3-cb92ed967f0b-kube-api-access-jdh2p\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977015    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-tls\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977030    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-wtmp\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977046    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-textfile\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977095    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977127    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-root\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977271 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977204    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ccc88934-7c14-4802-aee3-cb92ed967f0b-root\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977561 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:49.977538    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jun 02 15:47:49.977643 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977571    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-textfile\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977643 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977607    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccc88934-7c14-4802-aee3-cb92ed967f0b-metrics-client-ca\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.977643 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:47:49.977615    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-tls podName:ccc88934-7c14-4802-aee3-cb92ed967f0b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:50.477597599 +0000 UTC m=+76.375317971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-tls") pod "node-exporter-x8vhv" (UID: "ccc88934-7c14-4802-aee3-cb92ed967f0b") : secret "node-exporter-tls" not found
Jun 02 15:47:49.977808 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.977770    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.980415 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.980392    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:49.994613 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:49.994588    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdh2p\" (UniqueName: \"kubernetes.io/projected/ccc88934-7c14-4802-aee3-cb92ed967f0b-kube-api-access-jdh2p\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:50.483128 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:50.483075    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-tls\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:50.485545 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:50.485508    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccc88934-7c14-4802-aee3-cb92ed967f0b-node-exporter-tls\") pod \"node-exporter-x8vhv\" (UID: \"ccc88934-7c14-4802-aee3-cb92ed967f0b\") " pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:50.666960 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:50.666893    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x8vhv"
Jun 02 15:47:50.675655 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:50.675614    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc88934_7c14_4802_aee3_cb92ed967f0b.slice/crio-e7ec2dbad5932203232f020392eac4b170d86b696dd9840c884298bdcb59cf91 WatchSource:0}: Error finding container e7ec2dbad5932203232f020392eac4b170d86b696dd9840c884298bdcb59cf91: Status 404 returned error can't find the container with id e7ec2dbad5932203232f020392eac4b170d86b696dd9840c884298bdcb59cf91
Jun 02 15:47:50.966034 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:50.965997    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8vhv" event={"ID":"ccc88934-7c14-4802-aee3-cb92ed967f0b","Type":"ContainerStarted","Data":"e7ec2dbad5932203232f020392eac4b170d86b696dd9840c884298bdcb59cf91"}
Jun 02 15:47:50.967355 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:50.967332    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfd8c5f65-l28xk" event={"ID":"54faf369-6448-45f4-ae9f-64a3e7f53eaa","Type":"ContainerStarted","Data":"ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b"}
Jun 02 15:47:51.061200 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:51.061134    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-dfd8c5f65-l28xk" podStartSLOduration=1.099827535 podStartE2EDuration="4.061113801s" podCreationTimestamp="2026-06-02 15:47:47 +0000 UTC" firstStartedPulling="2026-06-02 15:47:47.689731775 +0000 UTC m=+73.587452141" lastFinishedPulling="2026-06-02 15:47:50.651018038 +0000 UTC m=+76.548738407" observedRunningTime="2026-06-02 15:47:51.059630077 +0000 UTC m=+76.957350465" watchObservedRunningTime="2026-06-02 15:47:51.061113801 +0000 UTC m=+76.958834190"
Jun 02 15:47:51.971634 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:51.971595    2562 generic.go:358] "Generic (PLEG): container finished" podID="ccc88934-7c14-4802-aee3-cb92ed967f0b" containerID="659ed2f0475fafd6bea3ade7b349e441d0d054b4f780d0c136403cb9fcd678a1" exitCode=0
Jun 02 15:47:51.972055 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:51.971683    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8vhv" event={"ID":"ccc88934-7c14-4802-aee3-cb92ed967f0b","Type":"ContainerDied","Data":"659ed2f0475fafd6bea3ade7b349e441d0d054b4f780d0c136403cb9fcd678a1"}
Jun 02 15:47:52.780226 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.780192    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"]
Jun 02 15:47:52.783728 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.783703    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.788053 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.788028    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jun 02 15:47:52.788194 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.788028    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jun 02 15:47:52.788194 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.788074    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jun 02 15:47:52.788785 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.788765    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jun 02 15:47:52.789017 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.788963    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-77n06m6n9rprf\""
Jun 02 15:47:52.789250 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.789227    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lr75l\""
Jun 02 15:47:52.789418 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.789378    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jun 02 15:47:52.809166 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.809138    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"]
Jun 02 15:47:52.902396 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902361    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmst\" (UniqueName: \"kubernetes.io/projected/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-kube-api-access-sfmst\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902396 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-metrics-client-ca\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902638 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902638 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902638 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902796 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-tls\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902796 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.902796 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.902730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-grpc-tls\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:52.976294 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.976257    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8vhv" event={"ID":"ccc88934-7c14-4802-aee3-cb92ed967f0b","Type":"ContainerStarted","Data":"ffd20faee5d1ee85a8d4d8b8ba23325e1574bb940211e9684afd37c2efba5db4"}
Jun 02 15:47:52.976294 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:52.976293    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8vhv" event={"ID":"ccc88934-7c14-4802-aee3-cb92ed967f0b","Type":"ContainerStarted","Data":"fe050ec9cf2e84df8ec4f29327a9c1d1e211bd42df341159377a863f2960a7d2"}
Jun 02 15:47:53.001179 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.001125    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x8vhv" podStartSLOduration=3.200383559 podStartE2EDuration="4.001110584s" podCreationTimestamp="2026-06-02 15:47:49 +0000 UTC" firstStartedPulling="2026-06-02 15:47:50.677543348 +0000 UTC m=+76.575263729" lastFinishedPulling="2026-06-02 15:47:51.478270389 +0000 UTC m=+77.375990754" observedRunningTime="2026-06-02 15:47:52.999172204 +0000 UTC m=+78.896892592" watchObservedRunningTime="2026-06-02 15:47:53.001110584 +0000 UTC m=+78.898830970"
Jun 02 15:47:53.003251 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003230    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003325 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003263    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003325 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003295    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003325 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003317    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-tls\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003478 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003441    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003600 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003498    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-grpc-tls\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003600 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003546    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmst\" (UniqueName: \"kubernetes.io/projected/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-kube-api-access-sfmst\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.003600 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.003586    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-metrics-client-ca\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.004327 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.004305    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-metrics-client-ca\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.006291 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.006268    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.006432 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.006327    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-grpc-tls\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.006542 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.006525    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.006684 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.006663    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.006725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.006683    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.006829 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.006806    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-secret-thanos-querier-tls\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.018413 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.018394    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmst\" (UniqueName: \"kubernetes.io/projected/dddfcd5b-00f9-4746-badc-0ccdfa2340a7-kube-api-access-sfmst\") pod \"thanos-querier-6fdb99758b-wrqfg\" (UID: \"dddfcd5b-00f9-4746-badc-0ccdfa2340a7\") " pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.092423 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.092336    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:53.232009 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.231975    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"]
Jun 02 15:47:53.235179 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:53.235151    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddfcd5b_00f9_4746_badc_0ccdfa2340a7.slice/crio-864544ae7b58d20d1b49237dd0775bb0cb3d87a63e328b5780b128da5eead18f WatchSource:0}: Error finding container 864544ae7b58d20d1b49237dd0775bb0cb3d87a63e328b5780b128da5eead18f: Status 404 returned error can't find the container with id 864544ae7b58d20d1b49237dd0775bb0cb3d87a63e328b5780b128da5eead18f
Jun 02 15:47:53.980236 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:53.980203    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"864544ae7b58d20d1b49237dd0775bb0cb3d87a63e328b5780b128da5eead18f"}
Jun 02 15:47:55.988714 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:55.988672    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"eeb39e7c930c11309c33fac81dc4ae210796eeedf6900654e183877067588f9d"}
Jun 02 15:47:55.988714 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:55.988720    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"71265e7472fc46899c0a07fea4a102ec2874d52bcf7fef25eb9b69107b006f9e"}
Jun 02 15:47:55.988714 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:55.988730    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"8b1cce357cd410765ea0e2a0b7868412d64d368e410dc318f6b72b4c7f8e144d"}
Jun 02 15:47:56.181632 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.181597    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-94ddccdf8-2fpmk"]
Jun 02 15:47:56.186008 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.185985    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.194416 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.194390    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jun 02 15:47:56.210279 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.210250    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94ddccdf8-2fpmk"]
Jun 02 15:47:56.334151 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-trusted-ca-bundle\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.334151 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbtkr\" (UniqueName: \"kubernetes.io/projected/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-kube-api-access-sbtkr\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.334385 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-serving-cert\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.334385 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-oauth-config\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.334385 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-service-ca\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.334385 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-oauth-serving-cert\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.334385 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.334347    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-config\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435215 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435177    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sbtkr\" (UniqueName: \"kubernetes.io/projected/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-kube-api-access-sbtkr\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435400 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435243    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-serving-cert\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435400 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435276    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-oauth-config\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435400 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435309    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-service-ca\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435400 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435336    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-oauth-serving-cert\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435400 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435377    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-config\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.435679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.435418    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-trusted-ca-bundle\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.436344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.436213    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-oauth-serving-cert\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.436344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.436233    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-config\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.436344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.436300    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-service-ca\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.436795 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.436768    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-trusted-ca-bundle\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.438206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.438182    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-oauth-config\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.438437 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.438409    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-serving-cert\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.447076 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.447052    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbtkr\" (UniqueName: \"kubernetes.io/projected/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-kube-api-access-sbtkr\") pod \"console-94ddccdf8-2fpmk\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") " pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.495957 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.495905    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:47:56.654876 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.654846    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94ddccdf8-2fpmk"]
Jun 02 15:47:56.665106 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:47:56.665078    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcf3d1d_0fa1_4630_a7d5_5f11edcd6d71.slice/crio-4245751d7eb7fa422e37fad1df3dc6da0cb978a38e4db70bda86811f5fe51da5 WatchSource:0}: Error finding container 4245751d7eb7fa422e37fad1df3dc6da0cb978a38e4db70bda86811f5fe51da5: Status 404 returned error can't find the container with id 4245751d7eb7fa422e37fad1df3dc6da0cb978a38e4db70bda86811f5fe51da5
Jun 02 15:47:56.955260 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.955231    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bqm8z"
Jun 02 15:47:56.993687 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.993645    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94ddccdf8-2fpmk" event={"ID":"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71","Type":"ContainerStarted","Data":"7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb"}
Jun 02 15:47:56.994136 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.993695    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94ddccdf8-2fpmk" event={"ID":"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71","Type":"ContainerStarted","Data":"4245751d7eb7fa422e37fad1df3dc6da0cb978a38e4db70bda86811f5fe51da5"}
Jun 02 15:47:56.996609 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.996584    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"2829a79b988b5a7f1fc67930e11894d4697824e3b296b2728ab5003c06bd9c17"}
Jun 02 15:47:56.996742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.996624    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"67b04bd360c28d558c5445a70e7357cfa70a784f9748756ecccd19d56a80bdd7"}
Jun 02 15:47:56.996742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.996641    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" event={"ID":"dddfcd5b-00f9-4746-badc-0ccdfa2340a7","Type":"ContainerStarted","Data":"25a34c13e46259dddfb52fa7903210eeee6b4693ff730470d91484dd400278fe"}
Jun 02 15:47:56.996859 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:56.996816    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:47:57.015183 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:57.015142    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-94ddccdf8-2fpmk" podStartSLOduration=1.015129749 podStartE2EDuration="1.015129749s" podCreationTimestamp="2026-06-02 15:47:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:57.013885212 +0000 UTC m=+82.911605625" watchObservedRunningTime="2026-06-02 15:47:57.015129749 +0000 UTC m=+82.912850216"
Jun 02 15:47:57.041517 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:57.041467    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg" podStartSLOduration=1.707976177 podStartE2EDuration="5.041454169s" podCreationTimestamp="2026-06-02 15:47:52 +0000 UTC" firstStartedPulling="2026-06-02 15:47:53.23728258 +0000 UTC m=+79.135002946" lastFinishedPulling="2026-06-02 15:47:56.570760562 +0000 UTC m=+82.468480938" observedRunningTime="2026-06-02 15:47:57.040210244 +0000 UTC m=+82.937930631" watchObservedRunningTime="2026-06-02 15:47:57.041454169 +0000 UTC m=+82.939174556"
Jun 02 15:47:57.538774 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:57.538724    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:57.538976 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:57.538808    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:57.543373 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:57.543344    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:47:58.002679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:47:58.002652    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:48:00.924308 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:00.924276    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c44974544-6nsl4"
Jun 02 15:48:03.004929 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:03.004878    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6fdb99758b-wrqfg"
Jun 02 15:48:06.496780 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:06.496745    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:48:06.497217 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:06.496831    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:48:06.501406 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:06.501383    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:48:07.029338 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:07.029307    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:48:07.096610 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:07.096580    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfd8c5f65-l28xk"]
Jun 02 15:48:17.955244 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:17.955213    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r7254"
Jun 02 15:48:32.116626 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.116584    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-dfd8c5f65-l28xk" podUID="54faf369-6448-45f4-ae9f-64a3e7f53eaa" containerName="console" containerID="cri-o://ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b" gracePeriod=15
Jun 02 15:48:32.354782 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.354761    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-dfd8c5f65-l28xk_54faf369-6448-45f4-ae9f-64a3e7f53eaa/console/0.log"
Jun 02 15:48:32.354905 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.354820    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:48:32.424411 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424330    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-service-ca\") pod \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") "
Jun 02 15:48:32.424411 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424367    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-oauth-config\") pod \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") "
Jun 02 15:48:32.424411 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424391    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9n7q\" (UniqueName: \"kubernetes.io/projected/54faf369-6448-45f4-ae9f-64a3e7f53eaa-kube-api-access-x9n7q\") pod \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") "
Jun 02 15:48:32.424660 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424432    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-serving-cert\") pod \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") "
Jun 02 15:48:32.424660 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424464    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-oauth-serving-cert\") pod \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") "
Jun 02 15:48:32.424660 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424500    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-config\") pod \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\" (UID: \"54faf369-6448-45f4-ae9f-64a3e7f53eaa\") "
Jun 02 15:48:32.424871 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424835    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-service-ca" (OuterVolumeSpecName: "service-ca") pod "54faf369-6448-45f4-ae9f-64a3e7f53eaa" (UID: "54faf369-6448-45f4-ae9f-64a3e7f53eaa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:48:32.425025 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424945    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-config" (OuterVolumeSpecName: "console-config") pod "54faf369-6448-45f4-ae9f-64a3e7f53eaa" (UID: "54faf369-6448-45f4-ae9f-64a3e7f53eaa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:48:32.425025 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.424957    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "54faf369-6448-45f4-ae9f-64a3e7f53eaa" (UID: "54faf369-6448-45f4-ae9f-64a3e7f53eaa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:48:32.426700 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.426670    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "54faf369-6448-45f4-ae9f-64a3e7f53eaa" (UID: "54faf369-6448-45f4-ae9f-64a3e7f53eaa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:48:32.426791 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.426716    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "54faf369-6448-45f4-ae9f-64a3e7f53eaa" (UID: "54faf369-6448-45f4-ae9f-64a3e7f53eaa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:48:32.426791 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.426733    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54faf369-6448-45f4-ae9f-64a3e7f53eaa-kube-api-access-x9n7q" (OuterVolumeSpecName: "kube-api-access-x9n7q") pod "54faf369-6448-45f4-ae9f-64a3e7f53eaa" (UID: "54faf369-6448-45f4-ae9f-64a3e7f53eaa"). InnerVolumeSpecName "kube-api-access-x9n7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:48:32.525732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.525700    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-service-ca\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:48:32.525732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.525729    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-oauth-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:48:32.525732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.525740    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9n7q\" (UniqueName: \"kubernetes.io/projected/54faf369-6448-45f4-ae9f-64a3e7f53eaa-kube-api-access-x9n7q\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:48:32.525979 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.525749    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:48:32.525979 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.525758    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-oauth-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:48:32.525979 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:32.525767    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54faf369-6448-45f4-ae9f-64a3e7f53eaa-console-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:48:33.098979 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.098952    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-dfd8c5f65-l28xk_54faf369-6448-45f4-ae9f-64a3e7f53eaa/console/0.log"
Jun 02 15:48:33.099157 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.098989    2562 generic.go:358] "Generic (PLEG): container finished" podID="54faf369-6448-45f4-ae9f-64a3e7f53eaa" containerID="ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b" exitCode=2
Jun 02 15:48:33.099157 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.099036    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfd8c5f65-l28xk" event={"ID":"54faf369-6448-45f4-ae9f-64a3e7f53eaa","Type":"ContainerDied","Data":"ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b"}
Jun 02 15:48:33.099157 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.099059    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfd8c5f65-l28xk" event={"ID":"54faf369-6448-45f4-ae9f-64a3e7f53eaa","Type":"ContainerDied","Data":"da6c94a7ad9f1d9d59cd6524b2e86ddb9a1af8704868a42b7338ce822397c34f"}
Jun 02 15:48:33.099157 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.099072    2562 scope.go:117] "RemoveContainer" containerID="ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b"
Jun 02 15:48:33.099157 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.099077    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfd8c5f65-l28xk"
Jun 02 15:48:33.106491 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.106475    2562 scope.go:117] "RemoveContainer" containerID="ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b"
Jun 02 15:48:33.106757 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:48:33.106734    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b\": container with ID starting with ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b not found: ID does not exist" containerID="ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b"
Jun 02 15:48:33.106849 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.106763    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b"} err="failed to get container status \"ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b\": rpc error: code = NotFound desc = could not find container \"ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b\": container with ID starting with ba6cbbacdaa1ffdb232dc1fbae5f5e958bd0028423c61ebd8646587083561c4b not found: ID does not exist"
Jun 02 15:48:33.127884 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.127856    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfd8c5f65-l28xk"]
Jun 02 15:48:33.137637 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:33.137611    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dfd8c5f65-l28xk"]
Jun 02 15:48:34.602988 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:34.602843    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54faf369-6448-45f4-ae9f-64a3e7f53eaa" path="/var/lib/kubelet/pods/54faf369-6448-45f4-ae9f-64a3e7f53eaa/volumes"
Jun 02 15:48:52.192850 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.192816    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"]
Jun 02 15:48:52.193368 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.193088    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54faf369-6448-45f4-ae9f-64a3e7f53eaa" containerName="console"
Jun 02 15:48:52.193368 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.193099    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54faf369-6448-45f4-ae9f-64a3e7f53eaa" containerName="console"
Jun 02 15:48:52.193368 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.193163    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="54faf369-6448-45f4-ae9f-64a3e7f53eaa" containerName="console"
Jun 02 15:48:52.197185 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.197170    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.200023 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.199998    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6lzft\""
Jun 02 15:48:52.200164 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.200052    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jun 02 15:48:52.200164 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.200007    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 02 15:48:52.200164 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.200094    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 02 15:48:52.201008 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.200988    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 02 15:48:52.208431 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.208407    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"]
Jun 02 15:48:52.280406 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.280371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccfnc\" (UniqueName: \"kubernetes.io/projected/dfc81a6c-3289-4fe9-9c11-689c0bfb3970-kube-api-access-ccfnc\") pod \"managed-serviceaccount-addon-agent-f95c45954-zh5rl\" (UID: \"dfc81a6c-3289-4fe9-9c11-689c0bfb3970\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.280594 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.280445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfc81a6c-3289-4fe9-9c11-689c0bfb3970-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f95c45954-zh5rl\" (UID: \"dfc81a6c-3289-4fe9-9c11-689c0bfb3970\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.381452 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.381417    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfc81a6c-3289-4fe9-9c11-689c0bfb3970-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f95c45954-zh5rl\" (UID: \"dfc81a6c-3289-4fe9-9c11-689c0bfb3970\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.381601 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.381473    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ccfnc\" (UniqueName: \"kubernetes.io/projected/dfc81a6c-3289-4fe9-9c11-689c0bfb3970-kube-api-access-ccfnc\") pod \"managed-serviceaccount-addon-agent-f95c45954-zh5rl\" (UID: \"dfc81a6c-3289-4fe9-9c11-689c0bfb3970\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.383927 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.383891    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfc81a6c-3289-4fe9-9c11-689c0bfb3970-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f95c45954-zh5rl\" (UID: \"dfc81a6c-3289-4fe9-9c11-689c0bfb3970\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.394773 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.394745    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccfnc\" (UniqueName: \"kubernetes.io/projected/dfc81a6c-3289-4fe9-9c11-689c0bfb3970-kube-api-access-ccfnc\") pod \"managed-serviceaccount-addon-agent-f95c45954-zh5rl\" (UID: \"dfc81a6c-3289-4fe9-9c11-689c0bfb3970\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.517945 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.517829    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"
Jun 02 15:48:52.642846 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:52.642812    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl"]
Jun 02 15:48:52.646769 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:48:52.646740    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfc81a6c_3289_4fe9_9c11_689c0bfb3970.slice/crio-d5106cbfa0d150d46d106b70078f82e5dbcf99a346e5556b64d2e0e49914e7e4 WatchSource:0}: Error finding container d5106cbfa0d150d46d106b70078f82e5dbcf99a346e5556b64d2e0e49914e7e4: Status 404 returned error can't find the container with id d5106cbfa0d150d46d106b70078f82e5dbcf99a346e5556b64d2e0e49914e7e4
Jun 02 15:48:53.154602 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:53.154565    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl" event={"ID":"dfc81a6c-3289-4fe9-9c11-689c0bfb3970","Type":"ContainerStarted","Data":"d5106cbfa0d150d46d106b70078f82e5dbcf99a346e5556b64d2e0e49914e7e4"}
Jun 02 15:48:56.164742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:56.164701    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl" event={"ID":"dfc81a6c-3289-4fe9-9c11-689c0bfb3970","Type":"ContainerStarted","Data":"3f0823fe7942f2f7d4ea7ca6da5195230807b13c2633d4e4a3153fbefcbf13e1"}
Jun 02 15:48:56.183712 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:48:56.183657    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f95c45954-zh5rl" podStartSLOduration=1.2492665299999999 podStartE2EDuration="4.183638221s" podCreationTimestamp="2026-06-02 15:48:52 +0000 UTC" firstStartedPulling="2026-06-02 15:48:52.649045202 +0000 UTC m=+138.546765569" lastFinishedPulling="2026-06-02 15:48:55.583416893 +0000 UTC m=+141.481137260" observedRunningTime="2026-06-02 15:48:56.183217331 +0000 UTC m=+142.080937710" watchObservedRunningTime="2026-06-02 15:48:56.183638221 +0000 UTC m=+142.081358610"
Jun 02 15:49:17.618539 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.618505    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cbf4bbc94-45l7j"]
Jun 02 15:49:17.621639 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.621620    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.652011 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.651975    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbf4bbc94-45l7j"]
Jun 02 15:49:17.794259 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-serving-cert\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.794259 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-oauth-config\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.794488 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-service-ca\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.794488 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-console-config\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.794488 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-oauth-serving-cert\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.794488 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-trusted-ca-bundle\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.794679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.794504    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjgr8\" (UniqueName: \"kubernetes.io/projected/899951e3-cab0-4495-b711-0a77a3916b04-kube-api-access-gjgr8\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895503 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895427    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-serving-cert\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895503 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895465    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-oauth-config\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895503 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895484    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-service-ca\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895520    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-console-config\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895544    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-oauth-serving-cert\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895565    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-trusted-ca-bundle\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.895744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.895611    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjgr8\" (UniqueName: \"kubernetes.io/projected/899951e3-cab0-4495-b711-0a77a3916b04-kube-api-access-gjgr8\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.896398 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.896372    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-console-config\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.896507 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.896373    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-oauth-serving-cert\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.896604 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.896579    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-service-ca\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.896681 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.896660    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-trusted-ca-bundle\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.898164 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.898140    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-oauth-config\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.898164 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.898164    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-serving-cert\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.905854 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.905829    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjgr8\" (UniqueName: \"kubernetes.io/projected/899951e3-cab0-4495-b711-0a77a3916b04-kube-api-access-gjgr8\") pod \"console-5cbf4bbc94-45l7j\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") " pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:17.929824 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:17.929796    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:18.078582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:18.078549    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbf4bbc94-45l7j"]
Jun 02 15:49:18.082313 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:49:18.082286    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899951e3_cab0_4495_b711_0a77a3916b04.slice/crio-da68473c20c884ce86eb957986b40325bc0728ac9bd1fbb17145bbcaad3afe0e WatchSource:0}: Error finding container da68473c20c884ce86eb957986b40325bc0728ac9bd1fbb17145bbcaad3afe0e: Status 404 returned error can't find the container with id da68473c20c884ce86eb957986b40325bc0728ac9bd1fbb17145bbcaad3afe0e
Jun 02 15:49:18.224467 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:18.224430    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbf4bbc94-45l7j" event={"ID":"899951e3-cab0-4495-b711-0a77a3916b04","Type":"ContainerStarted","Data":"8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618"}
Jun 02 15:49:18.224467 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:18.224472    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbf4bbc94-45l7j" event={"ID":"899951e3-cab0-4495-b711-0a77a3916b04","Type":"ContainerStarted","Data":"da68473c20c884ce86eb957986b40325bc0728ac9bd1fbb17145bbcaad3afe0e"}
Jun 02 15:49:18.254723 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:18.254670    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5cbf4bbc94-45l7j" podStartSLOduration=1.25465533 podStartE2EDuration="1.25465533s" podCreationTimestamp="2026-06-02 15:49:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:49:18.253906295 +0000 UTC m=+164.151626682" watchObservedRunningTime="2026-06-02 15:49:18.25465533 +0000 UTC m=+164.152375718"
Jun 02 15:49:27.929956 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:27.929882    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:27.929956 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:27.929968    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:27.934849 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:27.934827    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:28.254846 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:28.254815    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:49:28.314761 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:28.314727    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94ddccdf8-2fpmk"]
Jun 02 15:49:53.334610 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.334549    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-94ddccdf8-2fpmk" podUID="5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" containerName="console" containerID="cri-o://7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb" gracePeriod=15
Jun 02 15:49:53.581027 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.581003    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-94ddccdf8-2fpmk_5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71/console/0.log"
Jun 02 15:49:53.581159 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.581072    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:49:53.684047 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.683956    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-oauth-serving-cert\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684047 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684025    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-serving-cert\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684272 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684062    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-config\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684272 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684085    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-oauth-config\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684272 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684103    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-service-ca\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684272 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684130    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-trusted-ca-bundle\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684272 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684147    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbtkr\" (UniqueName: \"kubernetes.io/projected/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-kube-api-access-sbtkr\") pod \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\" (UID: \"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71\") "
Jun 02 15:49:53.684518 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684466    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:49:53.684572 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684552    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-config" (OuterVolumeSpecName: "console-config") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:49:53.684625 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684545    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-service-ca" (OuterVolumeSpecName: "service-ca") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:49:53.684932 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.684885    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:49:53.686259 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.686233    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:49:53.686355 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.686296    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-kube-api-access-sbtkr" (OuterVolumeSpecName: "kube-api-access-sbtkr") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "kube-api-access-sbtkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:49:53.686355 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.686333    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" (UID: "5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:49:53.785733 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785678    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:53.785733 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785722    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-oauth-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:53.785733 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785735    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-service-ca\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:53.785733 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785744    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-trusted-ca-bundle\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:53.785733 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785753    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbtkr\" (UniqueName: \"kubernetes.io/projected/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-kube-api-access-sbtkr\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:53.786058 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785763    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-oauth-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:53.786058 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:53.785773    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71-console-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:49:54.318041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.318015    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-94ddccdf8-2fpmk_5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71/console/0.log"
Jun 02 15:49:54.318205 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.318055    2562 generic.go:358] "Generic (PLEG): container finished" podID="5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" containerID="7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb" exitCode=2
Jun 02 15:49:54.318205 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.318092    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94ddccdf8-2fpmk" event={"ID":"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71","Type":"ContainerDied","Data":"7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb"}
Jun 02 15:49:54.318205 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.318130    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94ddccdf8-2fpmk"
Jun 02 15:49:54.318205 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.318143    2562 scope.go:117] "RemoveContainer" containerID="7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb"
Jun 02 15:49:54.318353 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.318133    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94ddccdf8-2fpmk" event={"ID":"5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71","Type":"ContainerDied","Data":"4245751d7eb7fa422e37fad1df3dc6da0cb978a38e4db70bda86811f5fe51da5"}
Jun 02 15:49:54.326524 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.326507    2562 scope.go:117] "RemoveContainer" containerID="7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb"
Jun 02 15:49:54.326758 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:49:54.326742    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb\": container with ID starting with 7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb not found: ID does not exist" containerID="7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb"
Jun 02 15:49:54.326801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.326765    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb"} err="failed to get container status \"7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb\": rpc error: code = NotFound desc = could not find container \"7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb\": container with ID starting with 7f4f558ee3d445f9683334a8db3eb008965071068c61f5f38908e3fb4c524adb not found: ID does not exist"
Jun 02 15:49:54.345792 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.345762    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94ddccdf8-2fpmk"]
Jun 02 15:49:54.351255 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.351231    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-94ddccdf8-2fpmk"]
Jun 02 15:49:54.600968 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:49:54.600890    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" path="/var/lib/kubelet/pods/5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71/volumes"
Jun 02 15:50:30.010924 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.010887    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-665f8df445-n5mdj"]
Jun 02 15:50:30.011405 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.011158    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" containerName="console"
Jun 02 15:50:30.011405 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.011169    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" containerName="console"
Jun 02 15:50:30.011405 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.011214    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="5dcf3d1d-0fa1-4630-a7d5-5f11edcd6d71" containerName="console"
Jun 02 15:50:30.014113 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.014098    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.023444 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.023421    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-665f8df445-n5mdj"]
Jun 02 15:50:30.048307 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6j46\" (UniqueName: \"kubernetes.io/projected/279064f6-c588-4a08-ba85-4ce349a2c8d9-kube-api-access-l6j46\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.048464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-config\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.048464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-oauth-serving-cert\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.048464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-trusted-ca-bundle\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.048464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-service-ca\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.048464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-serving-cert\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.048464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.048452    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-oauth-config\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.148762 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.148723    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6j46\" (UniqueName: \"kubernetes.io/projected/279064f6-c588-4a08-ba85-4ce349a2c8d9-kube-api-access-l6j46\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.148762 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.148768    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-config\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.148984 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.148789    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-oauth-serving-cert\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.148984 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.148839    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-trusted-ca-bundle\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149060 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149038    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-service-ca\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149101 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149073    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-serving-cert\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149101 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149092    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-oauth-config\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149533 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149512    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-oauth-serving-cert\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149711 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149688    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-config\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149787 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149732    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-trusted-ca-bundle\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.149843 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.149785    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-service-ca\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.151642 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.151610    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-serving-cert\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.151724 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.151666    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-oauth-config\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.158175 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.158156    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6j46\" (UniqueName: \"kubernetes.io/projected/279064f6-c588-4a08-ba85-4ce349a2c8d9-kube-api-access-l6j46\") pod \"console-665f8df445-n5mdj\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") " pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.323014 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.322903    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:30.458050 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:30.458013    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-665f8df445-n5mdj"]
Jun 02 15:50:30.462013 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:50:30.461984    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279064f6_c588_4a08_ba85_4ce349a2c8d9.slice/crio-4232d2427e88e4bb837359e61395a6eab3ff2459e0700950e5077a85e35503a0 WatchSource:0}: Error finding container 4232d2427e88e4bb837359e61395a6eab3ff2459e0700950e5077a85e35503a0: Status 404 returned error can't find the container with id 4232d2427e88e4bb837359e61395a6eab3ff2459e0700950e5077a85e35503a0
Jun 02 15:50:31.420705 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:31.420665    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665f8df445-n5mdj" event={"ID":"279064f6-c588-4a08-ba85-4ce349a2c8d9","Type":"ContainerStarted","Data":"35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6"}
Jun 02 15:50:31.420705 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:31.420702    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665f8df445-n5mdj" event={"ID":"279064f6-c588-4a08-ba85-4ce349a2c8d9","Type":"ContainerStarted","Data":"4232d2427e88e4bb837359e61395a6eab3ff2459e0700950e5077a85e35503a0"}
Jun 02 15:50:31.439635 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:31.439546    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-665f8df445-n5mdj" podStartSLOduration=2.439530633 podStartE2EDuration="2.439530633s" podCreationTimestamp="2026-06-02 15:50:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:50:31.438802387 +0000 UTC m=+237.336522775" watchObservedRunningTime="2026-06-02 15:50:31.439530633 +0000 UTC m=+237.337251021"
Jun 02 15:50:40.323540 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:40.323500    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:40.323540 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:40.323550    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:40.328609 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:40.328582    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:40.448112 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:40.448084    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 15:50:40.501861 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:50:40.501824    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbf4bbc94-45l7j"]
Jun 02 15:51:05.521465 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.521366    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5cbf4bbc94-45l7j" podUID="899951e3-cab0-4495-b711-0a77a3916b04" containerName="console" containerID="cri-o://8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618" gracePeriod=15
Jun 02 15:51:05.762555 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.762532    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cbf4bbc94-45l7j_899951e3-cab0-4495-b711-0a77a3916b04/console/0.log"
Jun 02 15:51:05.762680 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.762597    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:51:05.834541 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834457    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjgr8\" (UniqueName: \"kubernetes.io/projected/899951e3-cab0-4495-b711-0a77a3916b04-kube-api-access-gjgr8\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.834541 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834499    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-service-ca\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.834541 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834534    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-console-config\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.834807 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834578    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-oauth-config\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.834807 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834595    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-oauth-serving-cert\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.834807 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834626    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-trusted-ca-bundle\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.834807 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.834643    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-serving-cert\") pod \"899951e3-cab0-4495-b711-0a77a3916b04\" (UID: \"899951e3-cab0-4495-b711-0a77a3916b04\") "
Jun 02 15:51:05.835035 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.835009    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-console-config" (OuterVolumeSpecName: "console-config") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:51:05.835035 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.835021    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:51:05.835129 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.835014    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-service-ca" (OuterVolumeSpecName: "service-ca") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:51:05.835269 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.835230    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:51:05.836744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.836712    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:51:05.836744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.836727    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/899951e3-cab0-4495-b711-0a77a3916b04-kube-api-access-gjgr8" (OuterVolumeSpecName: "kube-api-access-gjgr8") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "kube-api-access-gjgr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:51:05.836899 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.836863    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "899951e3-cab0-4495-b711-0a77a3916b04" (UID: "899951e3-cab0-4495-b711-0a77a3916b04"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:51:05.935172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935130    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-trusted-ca-bundle\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:05.935172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935163    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:05.935172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935173    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjgr8\" (UniqueName: \"kubernetes.io/projected/899951e3-cab0-4495-b711-0a77a3916b04-kube-api-access-gjgr8\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:05.935172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935183    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-service-ca\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:05.935431 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935194    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-console-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:05.935431 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935202    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/899951e3-cab0-4495-b711-0a77a3916b04-console-oauth-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:05.935431 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:05.935210    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/899951e3-cab0-4495-b711-0a77a3916b04-oauth-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:51:06.515498 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.515472    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cbf4bbc94-45l7j_899951e3-cab0-4495-b711-0a77a3916b04/console/0.log"
Jun 02 15:51:06.515668 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.515510    2562 generic.go:358] "Generic (PLEG): container finished" podID="899951e3-cab0-4495-b711-0a77a3916b04" containerID="8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618" exitCode=2
Jun 02 15:51:06.515668 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.515581    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbf4bbc94-45l7j" event={"ID":"899951e3-cab0-4495-b711-0a77a3916b04","Type":"ContainerDied","Data":"8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618"}
Jun 02 15:51:06.515668 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.515583    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbf4bbc94-45l7j"
Jun 02 15:51:06.515668 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.515607    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbf4bbc94-45l7j" event={"ID":"899951e3-cab0-4495-b711-0a77a3916b04","Type":"ContainerDied","Data":"da68473c20c884ce86eb957986b40325bc0728ac9bd1fbb17145bbcaad3afe0e"}
Jun 02 15:51:06.515668 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.515622    2562 scope.go:117] "RemoveContainer" containerID="8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618"
Jun 02 15:51:06.523957 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.523729    2562 scope.go:117] "RemoveContainer" containerID="8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618"
Jun 02 15:51:06.524181 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:51:06.524026    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618\": container with ID starting with 8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618 not found: ID does not exist" containerID="8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618"
Jun 02 15:51:06.524181 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.524049    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618"} err="failed to get container status \"8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618\": rpc error: code = NotFound desc = could not find container \"8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618\": container with ID starting with 8c911e6929b275acf21f41e527ce0e1412ae1bf605a97054303f6c1c762f2618 not found: ID does not exist"
Jun 02 15:51:06.540038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.540016    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbf4bbc94-45l7j"]
Jun 02 15:51:06.551450 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.551427    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cbf4bbc94-45l7j"]
Jun 02 15:51:06.600205 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:06.600179    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="899951e3-cab0-4495-b711-0a77a3916b04" path="/var/lib/kubelet/pods/899951e3-cab0-4495-b711-0a77a3916b04/volumes"
Jun 02 15:51:34.478778 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:34.478751    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:51:34.481197 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:34.481179    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:51:34.481348 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:51:34.481333    2562 kubelet.go:1628] "Image garbage collection succeeded"
Jun 02 15:52:08.163207 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.163168    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"]
Jun 02 15:52:08.165344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.163420    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="899951e3-cab0-4495-b711-0a77a3916b04" containerName="console"
Jun 02 15:52:08.165344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.163430    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="899951e3-cab0-4495-b711-0a77a3916b04" containerName="console"
Jun 02 15:52:08.165344 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.163476    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="899951e3-cab0-4495-b711-0a77a3916b04" containerName="console"
Jun 02 15:52:08.166294 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.166278    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.169048 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.169021    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7ztzt\""
Jun 02 15:52:08.169232 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.169210    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:52:08.169390 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.169146    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:52:08.176120 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.176092    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"]
Jun 02 15:52:08.317434 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.317394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.317434 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.317441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.317653 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.317474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mjc9\" (UniqueName: \"kubernetes.io/projected/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-kube-api-access-2mjc9\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.418826 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.418731    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.418826 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.418785    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.419029 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.418834    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2mjc9\" (UniqueName: \"kubernetes.io/projected/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-kube-api-access-2mjc9\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.419227 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.419207    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.419270 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.419255    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.430294 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.430264    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mjc9\" (UniqueName: \"kubernetes.io/projected/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-kube-api-access-2mjc9\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.479433 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.479382    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:08.606907 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.606875    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"]
Jun 02 15:52:08.610219 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:52:08.610192    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c3cbbf7_a021_489e_9d5b_92d82cb9fed2.slice/crio-e8a1dc57ba7c7a1986ea4b67e1ed26f4cc44b866d0b67edfcf0436d61d3fc854 WatchSource:0}: Error finding container e8a1dc57ba7c7a1986ea4b67e1ed26f4cc44b866d0b67edfcf0436d61d3fc854: Status 404 returned error can't find the container with id e8a1dc57ba7c7a1986ea4b67e1ed26f4cc44b866d0b67edfcf0436d61d3fc854
Jun 02 15:52:08.612181 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.612163    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 15:52:08.684742 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:08.684656    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7" event={"ID":"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2","Type":"ContainerStarted","Data":"e8a1dc57ba7c7a1986ea4b67e1ed26f4cc44b866d0b67edfcf0436d61d3fc854"}
Jun 02 15:52:15.704275 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:15.704237    2562 generic.go:358] "Generic (PLEG): container finished" podID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerID="1c2271daa4d4e86828efca199518fd0a25b729d43772eb05d57f81db9a782678" exitCode=0
Jun 02 15:52:15.704658 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:15.704325    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7" event={"ID":"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2","Type":"ContainerDied","Data":"1c2271daa4d4e86828efca199518fd0a25b729d43772eb05d57f81db9a782678"}
Jun 02 15:52:18.715160 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:18.715120    2562 generic.go:358] "Generic (PLEG): container finished" podID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerID="1df15b1864e4c35d3f205e1bd1217510db7d2861778a46a1b1e254cac4dcdce2" exitCode=0
Jun 02 15:52:18.715541 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:18.715183    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7" event={"ID":"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2","Type":"ContainerDied","Data":"1df15b1864e4c35d3f205e1bd1217510db7d2861778a46a1b1e254cac4dcdce2"}
Jun 02 15:52:26.738987 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:26.738954    2562 generic.go:358] "Generic (PLEG): container finished" podID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerID="ed84b5374516e1317b7d69ee86261e68ae87cc749810b6b1192fc0fb8cf2aaaa" exitCode=0
Jun 02 15:52:26.739345 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:26.739004    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7" event={"ID":"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2","Type":"ContainerDied","Data":"ed84b5374516e1317b7d69ee86261e68ae87cc749810b6b1192fc0fb8cf2aaaa"}
Jun 02 15:52:27.864499 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.864478    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:27.879644 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.879621    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-util\") pod \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") "
Jun 02 15:52:27.879719 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.879656    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mjc9\" (UniqueName: \"kubernetes.io/projected/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-kube-api-access-2mjc9\") pod \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") "
Jun 02 15:52:27.879719 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.879690    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-bundle\") pod \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\" (UID: \"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2\") "
Jun 02 15:52:27.880242 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.880216    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-bundle" (OuterVolumeSpecName: "bundle") pod "5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" (UID: "5c3cbbf7-a021-489e-9d5b-92d82cb9fed2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:52:27.881810 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.881779    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-kube-api-access-2mjc9" (OuterVolumeSpecName: "kube-api-access-2mjc9") pod "5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" (UID: "5c3cbbf7-a021-489e-9d5b-92d82cb9fed2"). InnerVolumeSpecName "kube-api-access-2mjc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:52:27.884955 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.884930    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-util" (OuterVolumeSpecName: "util") pod "5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" (UID: "5c3cbbf7-a021-489e-9d5b-92d82cb9fed2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:52:27.980371 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.980336    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-util\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:52:27.980371 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.980366    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mjc9\" (UniqueName: \"kubernetes.io/projected/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-kube-api-access-2mjc9\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:52:27.980371 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:27.980378    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c3cbbf7-a021-489e-9d5b-92d82cb9fed2-bundle\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:52:28.745849 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:28.745815    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7" event={"ID":"5c3cbbf7-a021-489e-9d5b-92d82cb9fed2","Type":"ContainerDied","Data":"e8a1dc57ba7c7a1986ea4b67e1ed26f4cc44b866d0b67edfcf0436d61d3fc854"}
Jun 02 15:52:28.745849 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:28.745834    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cwfdt7"
Jun 02 15:52:28.745849 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:28.745847    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8a1dc57ba7c7a1986ea4b67e1ed26f4cc44b866d0b67edfcf0436d61d3fc854"
Jun 02 15:52:35.558276 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558218    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"]
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558593    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="extract"
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558611    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="extract"
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558631    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="pull"
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558639    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="pull"
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558664    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="util"
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558673    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="util"
Jun 02 15:52:35.558839 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.558737    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="5c3cbbf7-a021-489e-9d5b-92d82cb9fed2" containerName="extract"
Jun 02 15:52:35.565298 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.565279    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.567356 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.567337    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jun 02 15:52:35.567732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.567716    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jun 02 15:52:35.567936 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.567903    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-txvf4\""
Jun 02 15:52:35.568133 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.568117    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jun 02 15:52:35.574772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.574751    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"]
Jun 02 15:52:35.631348 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.631320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0913f10e-6095-40d1-884a-d120d5308f5d-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk\" (UID: \"0913f10e-6095-40d1-884a-d120d5308f5d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.631513 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.631364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfbh4\" (UniqueName: \"kubernetes.io/projected/0913f10e-6095-40d1-884a-d120d5308f5d-kube-api-access-tfbh4\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk\" (UID: \"0913f10e-6095-40d1-884a-d120d5308f5d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.732411 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.732376    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0913f10e-6095-40d1-884a-d120d5308f5d-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk\" (UID: \"0913f10e-6095-40d1-884a-d120d5308f5d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.732411 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.732417    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tfbh4\" (UniqueName: \"kubernetes.io/projected/0913f10e-6095-40d1-884a-d120d5308f5d-kube-api-access-tfbh4\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk\" (UID: \"0913f10e-6095-40d1-884a-d120d5308f5d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.734637 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.734608    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/0913f10e-6095-40d1-884a-d120d5308f5d-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk\" (UID: \"0913f10e-6095-40d1-884a-d120d5308f5d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.742656 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.742630    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfbh4\" (UniqueName: \"kubernetes.io/projected/0913f10e-6095-40d1-884a-d120d5308f5d-kube-api-access-tfbh4\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk\" (UID: \"0913f10e-6095-40d1-884a-d120d5308f5d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:35.876579 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:35.876499    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:36.020033 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:36.020005    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"]
Jun 02 15:52:36.022445 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:52:36.022415    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0913f10e_6095_40d1_884a_d120d5308f5d.slice/crio-b29a6bc9ac98860b3857f1c06ff2bdd087ba02fef82574623d36ffff7a11d962 WatchSource:0}: Error finding container b29a6bc9ac98860b3857f1c06ff2bdd087ba02fef82574623d36ffff7a11d962: Status 404 returned error can't find the container with id b29a6bc9ac98860b3857f1c06ff2bdd087ba02fef82574623d36ffff7a11d962
Jun 02 15:52:36.782925 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:36.782881    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk" event={"ID":"0913f10e-6095-40d1-884a-d120d5308f5d","Type":"ContainerStarted","Data":"b29a6bc9ac98860b3857f1c06ff2bdd087ba02fef82574623d36ffff7a11d962"}
Jun 02 15:52:40.795049 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.795009    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk" event={"ID":"0913f10e-6095-40d1-884a-d120d5308f5d","Type":"ContainerStarted","Data":"42854b6683f3f36a4f070b788428c3c03a66a5d53481bae55642d3447c1746e5"}
Jun 02 15:52:40.795443 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.795140    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:52:40.819454 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.819394    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk" podStartSLOduration=1.460412254 podStartE2EDuration="5.819376824s" podCreationTimestamp="2026-06-02 15:52:35 +0000 UTC" firstStartedPulling="2026-06-02 15:52:36.024144775 +0000 UTC m=+361.921865141" lastFinishedPulling="2026-06-02 15:52:40.38310933 +0000 UTC m=+366.280829711" observedRunningTime="2026-06-02 15:52:40.817657565 +0000 UTC m=+366.715377954" watchObservedRunningTime="2026-06-02 15:52:40.819376824 +0000 UTC m=+366.717097215"
Jun 02 15:52:40.973050 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.973013    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-8ttld"]
Jun 02 15:52:40.976320 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.976301    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:40.978867 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.978843    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jun 02 15:52:40.979012 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.978843    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jun 02 15:52:40.979128 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.979114    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-ftbhv\""
Jun 02 15:52:40.984522 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:40.984501    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-8ttld"]
Jun 02 15:52:41.075435 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.075346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtdg7\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-kube-api-access-xtdg7\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.075435 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.075410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.075630 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.075472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/fab182e6-1aba-470c-aeb1-114d1cf458c1-cabundle0\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.176690 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.176649    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xtdg7\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-kube-api-access-xtdg7\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.176886 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.176737    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.176886 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.176772    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/fab182e6-1aba-470c-aeb1-114d1cf458c1-cabundle0\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.177062 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.176891    2562 projected.go:264] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jun 02 15:52:41.177062 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.176938    2562 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:52:41.177062 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.176950    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:52:41.177062 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.176968    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-8ttld: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jun 02 15:52:41.177062 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.177034    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates podName:fab182e6-1aba-470c-aeb1-114d1cf458c1 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:41.677013204 +0000 UTC m=+367.574733572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates") pod "keda-operator-ffbb595cb-8ttld" (UID: "fab182e6-1aba-470c-aeb1-114d1cf458c1") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jun 02 15:52:41.177523 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.177502    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/fab182e6-1aba-470c-aeb1-114d1cf458c1-cabundle0\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.189172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.189143    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtdg7\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-kube-api-access-xtdg7\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.322226 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.322180    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"]
Jun 02 15:52:41.325975 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.325901    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.328123 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.328093    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jun 02 15:52:41.337878 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.337852    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"]
Jun 02 15:52:41.378401 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.378367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r4zc\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-kube-api-access-5r4zc\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.378401 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.378410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.378620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.378487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.479080 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.479035    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5r4zc\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-kube-api-access-5r4zc\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.479080 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.479087    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.479312 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.479137    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.479312 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.479262    2562 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:52:41.479312 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.479276    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:52:41.479312 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.479296    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks: references non-existent secret key: tls.crt
Jun 02 15:52:41.479467 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.479343    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates podName:ce11bd5f-004e-4f56-bbd8-8ba90f606b47 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:41.979330067 +0000 UTC m=+367.877050436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates") pod "keda-metrics-apiserver-7c9f485588-7ddks" (UID: "ce11bd5f-004e-4f56-bbd8-8ba90f606b47") : references non-existent secret key: tls.crt
Jun 02 15:52:41.479530 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.479510    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.488327 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.488295    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r4zc\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-kube-api-access-5r4zc\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.597042 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.596950    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-cf49989db-g6zsz"]
Jun 02 15:52:41.600511 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.600491    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.602598 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.602580    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jun 02 15:52:41.610324 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.610304    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-g6zsz"]
Jun 02 15:52:41.680433 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.680393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9e09065a-a79f-4f8f-ae99-abc58a84743e-certificates\") pod \"keda-admission-cf49989db-g6zsz\" (UID: \"9e09065a-a79f-4f8f-ae99-abc58a84743e\") " pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.680611 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.680491    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:41.680611 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.680573    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99qkd\" (UniqueName: \"kubernetes.io/projected/9e09065a-a79f-4f8f-ae99-abc58a84743e-kube-api-access-99qkd\") pod \"keda-admission-cf49989db-g6zsz\" (UID: \"9e09065a-a79f-4f8f-ae99-abc58a84743e\") " pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.680693 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.680644    2562 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:52:41.680693 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.680657    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:52:41.680693 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.680665    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-8ttld: references non-existent secret key: ca.crt
Jun 02 15:52:41.680798 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.680713    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates podName:fab182e6-1aba-470c-aeb1-114d1cf458c1 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:42.680698422 +0000 UTC m=+368.578418795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates") pod "keda-operator-ffbb595cb-8ttld" (UID: "fab182e6-1aba-470c-aeb1-114d1cf458c1") : references non-existent secret key: ca.crt
Jun 02 15:52:41.781051 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.781012    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-99qkd\" (UniqueName: \"kubernetes.io/projected/9e09065a-a79f-4f8f-ae99-abc58a84743e-kube-api-access-99qkd\") pod \"keda-admission-cf49989db-g6zsz\" (UID: \"9e09065a-a79f-4f8f-ae99-abc58a84743e\") " pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.781693 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.781663    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9e09065a-a79f-4f8f-ae99-abc58a84743e-certificates\") pod \"keda-admission-cf49989db-g6zsz\" (UID: \"9e09065a-a79f-4f8f-ae99-abc58a84743e\") " pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.784102 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.784082    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9e09065a-a79f-4f8f-ae99-abc58a84743e-certificates\") pod \"keda-admission-cf49989db-g6zsz\" (UID: \"9e09065a-a79f-4f8f-ae99-abc58a84743e\") " pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.791555 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.791527    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99qkd\" (UniqueName: \"kubernetes.io/projected/9e09065a-a79f-4f8f-ae99-abc58a84743e-kube-api-access-99qkd\") pod \"keda-admission-cf49989db-g6zsz\" (UID: \"9e09065a-a79f-4f8f-ae99-abc58a84743e\") " pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.912696 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.912618    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:41.984712 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:41.984675    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:41.984896 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.984873    2562 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:52:41.984896 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.984885    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:52:41.985024 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.984901    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks: references non-existent secret key: tls.crt
Jun 02 15:52:41.985024 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:41.984965    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates podName:ce11bd5f-004e-4f56-bbd8-8ba90f606b47 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:42.984951583 +0000 UTC m=+368.882671949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates") pod "keda-metrics-apiserver-7c9f485588-7ddks" (UID: "ce11bd5f-004e-4f56-bbd8-8ba90f606b47") : references non-existent secret key: tls.crt
Jun 02 15:52:42.060533 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:42.060509    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-g6zsz"]
Jun 02 15:52:42.063646 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:52:42.063618    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e09065a_a79f_4f8f_ae99_abc58a84743e.slice/crio-49e44d0850a40ff8972f06aab3284414db3e6bc8b4d37babcc57a72be9d7a123 WatchSource:0}: Error finding container 49e44d0850a40ff8972f06aab3284414db3e6bc8b4d37babcc57a72be9d7a123: Status 404 returned error can't find the container with id 49e44d0850a40ff8972f06aab3284414db3e6bc8b4d37babcc57a72be9d7a123
Jun 02 15:52:42.691071 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:42.691029    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:42.691274 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.691220    2562 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:52:42.691274 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.691241    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:52:42.691274 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.691253    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-8ttld: references non-existent secret key: ca.crt
Jun 02 15:52:42.691424 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.691320    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates podName:fab182e6-1aba-470c-aeb1-114d1cf458c1 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:44.691298025 +0000 UTC m=+370.589018412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates") pod "keda-operator-ffbb595cb-8ttld" (UID: "fab182e6-1aba-470c-aeb1-114d1cf458c1") : references non-existent secret key: ca.crt
Jun 02 15:52:42.802867 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:42.802829    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-g6zsz" event={"ID":"9e09065a-a79f-4f8f-ae99-abc58a84743e","Type":"ContainerStarted","Data":"49e44d0850a40ff8972f06aab3284414db3e6bc8b4d37babcc57a72be9d7a123"}
Jun 02 15:52:42.994509 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:42.994418    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:42.994867 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.994575    2562 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:52:42.994867 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.994596    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:52:42.994867 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.994614    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks: references non-existent secret key: tls.crt
Jun 02 15:52:42.994867 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:42.994680    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates podName:ce11bd5f-004e-4f56-bbd8-8ba90f606b47 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:44.994662414 +0000 UTC m=+370.892382782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates") pod "keda-metrics-apiserver-7c9f485588-7ddks" (UID: "ce11bd5f-004e-4f56-bbd8-8ba90f606b47") : references non-existent secret key: tls.crt
Jun 02 15:52:43.807141 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:43.807105    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-g6zsz" event={"ID":"9e09065a-a79f-4f8f-ae99-abc58a84743e","Type":"ContainerStarted","Data":"1a071790740a7c48fde10d3b04d48f881aa7d9f70815ea6b7b88a7e90bc4747f"}
Jun 02 15:52:43.807325 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:43.807195    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:52:43.829990 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:43.829218    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-admission-cf49989db-g6zsz" podStartSLOduration=1.291039192 podStartE2EDuration="2.829197102s" podCreationTimestamp="2026-06-02 15:52:41 +0000 UTC" firstStartedPulling="2026-06-02 15:52:42.065041982 +0000 UTC m=+367.962762347" lastFinishedPulling="2026-06-02 15:52:43.603199892 +0000 UTC m=+369.500920257" observedRunningTime="2026-06-02 15:52:43.828987248 +0000 UTC m=+369.726707650" watchObservedRunningTime="2026-06-02 15:52:43.829197102 +0000 UTC m=+369.726917491"
Jun 02 15:52:44.708217 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:44.708179    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:44.708591 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:44.708317    2562 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:52:44.708591 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:44.708335    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:52:44.708591 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:44.708344    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-8ttld: references non-existent secret key: ca.crt
Jun 02 15:52:44.708591 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:44.708396    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates podName:fab182e6-1aba-470c-aeb1-114d1cf458c1 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:48.70838249 +0000 UTC m=+374.606102857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates") pod "keda-operator-ffbb595cb-8ttld" (UID: "fab182e6-1aba-470c-aeb1-114d1cf458c1") : references non-existent secret key: ca.crt
Jun 02 15:52:45.010582 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:45.010488    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:45.010729 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:45.010632    2562 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:52:45.010729 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:45.010657    2562 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:52:45.010729 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:45.010679    2562 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks: references non-existent secret key: tls.crt
Jun 02 15:52:45.010822 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:52:45.010736    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates podName:ce11bd5f-004e-4f56-bbd8-8ba90f606b47 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:49.010717807 +0000 UTC m=+374.908438173 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates") pod "keda-metrics-apiserver-7c9f485588-7ddks" (UID: "ce11bd5f-004e-4f56-bbd8-8ba90f606b47") : references non-existent secret key: tls.crt
Jun 02 15:52:48.740418 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:48.740381    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:48.742881 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:48.742855    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/fab182e6-1aba-470c-aeb1-114d1cf458c1-certificates\") pod \"keda-operator-ffbb595cb-8ttld\" (UID: \"fab182e6-1aba-470c-aeb1-114d1cf458c1\") " pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:48.787671 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:48.787634    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:48.937948 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:48.937897    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-8ttld"]
Jun 02 15:52:48.941680 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:52:48.941641    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab182e6_1aba_470c_aeb1_114d1cf458c1.slice/crio-1da0862b738b7bc38143698052ad75c5ae304a9ce6fdd0eed5a0994f24a00c47 WatchSource:0}: Error finding container 1da0862b738b7bc38143698052ad75c5ae304a9ce6fdd0eed5a0994f24a00c47: Status 404 returned error can't find the container with id 1da0862b738b7bc38143698052ad75c5ae304a9ce6fdd0eed5a0994f24a00c47
Jun 02 15:52:49.043813 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:49.043717    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:49.046130 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:49.046106    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ce11bd5f-004e-4f56-bbd8-8ba90f606b47-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7ddks\" (UID: \"ce11bd5f-004e-4f56-bbd8-8ba90f606b47\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:49.137607 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:49.137575    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:49.269677 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:49.269609    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"]
Jun 02 15:52:49.272287 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:52:49.272260    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce11bd5f_004e_4f56_bbd8_8ba90f606b47.slice/crio-daee4ec03507443712c0c34b5b44964742f4b3e87209eb5b0bf2c2ffc00aa86a WatchSource:0}: Error finding container daee4ec03507443712c0c34b5b44964742f4b3e87209eb5b0bf2c2ffc00aa86a: Status 404 returned error can't find the container with id daee4ec03507443712c0c34b5b44964742f4b3e87209eb5b0bf2c2ffc00aa86a
Jun 02 15:52:49.827765 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:49.827726    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks" event={"ID":"ce11bd5f-004e-4f56-bbd8-8ba90f606b47","Type":"ContainerStarted","Data":"daee4ec03507443712c0c34b5b44964742f4b3e87209eb5b0bf2c2ffc00aa86a"}
Jun 02 15:52:49.828866 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:49.828833    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-8ttld" event={"ID":"fab182e6-1aba-470c-aeb1-114d1cf458c1","Type":"ContainerStarted","Data":"1da0862b738b7bc38143698052ad75c5ae304a9ce6fdd0eed5a0994f24a00c47"}
Jun 02 15:52:51.837310 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:51.837273    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks" event={"ID":"ce11bd5f-004e-4f56-bbd8-8ba90f606b47","Type":"ContainerStarted","Data":"ee9238b77623e80dbdd3cbd34b74148fddf0d906b42c1d80b6b64285c8ba2002"}
Jun 02 15:52:51.837734 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:51.837403    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:52:51.854985 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:51.854928    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks" podStartSLOduration=8.421847508 podStartE2EDuration="10.854895668s" podCreationTimestamp="2026-06-02 15:52:41 +0000 UTC" firstStartedPulling="2026-06-02 15:52:49.273553588 +0000 UTC m=+375.171273958" lastFinishedPulling="2026-06-02 15:52:51.706601736 +0000 UTC m=+377.604322118" observedRunningTime="2026-06-02 15:52:51.85415112 +0000 UTC m=+377.751871507" watchObservedRunningTime="2026-06-02 15:52:51.854895668 +0000 UTC m=+377.752616056"
Jun 02 15:52:53.844548 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:53.844507    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-8ttld" event={"ID":"fab182e6-1aba-470c-aeb1-114d1cf458c1","Type":"ContainerStarted","Data":"7e5c460c9e647522c091328076f43e428d1322bbed35d6cec844653b88164b2f"}
Jun 02 15:52:53.844937 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:53.844610    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:52:53.868106 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:52:53.868039    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-8ttld" podStartSLOduration=9.185309033 podStartE2EDuration="13.868019554s" podCreationTimestamp="2026-06-02 15:52:40 +0000 UTC" firstStartedPulling="2026-06-02 15:52:48.942929674 +0000 UTC m=+374.840650040" lastFinishedPulling="2026-06-02 15:52:53.625640192 +0000 UTC m=+379.523360561" observedRunningTime="2026-06-02 15:52:53.86661009 +0000 UTC m=+379.764330478" watchObservedRunningTime="2026-06-02 15:52:53.868019554 +0000 UTC m=+379.765739944"
Jun 02 15:53:01.801485 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:01.801452    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-sw7lk"
Jun 02 15:53:02.844570 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:02.844540    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7ddks"
Jun 02 15:53:04.812688 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:04.812656    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-cf49989db-g6zsz"
Jun 02 15:53:14.849043 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:14.849012    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-8ttld"
Jun 02 15:53:50.534087 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.533995    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-z9x6c"]
Jun 02 15:53:50.537595 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.537577    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"]
Jun 02 15:53:50.537744 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.537728    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:50.540537 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.540519    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.541905 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.541886    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jun 02 15:53:50.542043 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.541933    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jun 02 15:53:50.542174 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.542160    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jun 02 15:53:50.543838 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.543721    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-grqqk\""
Jun 02 15:53:50.543838 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.543734    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-4zpcm\""
Jun 02 15:53:50.543838 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.543782    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jun 02 15:53:50.550725 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.550701    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-z9x6c"]
Jun 02 15:53:50.573332 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.573306    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"]
Jun 02 15:53:50.631366 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.631327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd4w7\" (UniqueName: \"kubernetes.io/projected/e146d349-da82-494f-9bb6-48280b7350b8-kube-api-access-qd4w7\") pod \"llmisvc-controller-manager-6b94ff949c-lxvlv\" (UID: \"e146d349-da82-494f-9bb6-48280b7350b8\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.631565 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.631385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvjv\" (UniqueName: \"kubernetes.io/projected/a73e9ee4-0f44-461c-8064-d18cf204c4f9-kube-api-access-bgvjv\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:50.631565 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.631417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e146d349-da82-494f-9bb6-48280b7350b8-cert\") pod \"llmisvc-controller-manager-6b94ff949c-lxvlv\" (UID: \"e146d349-da82-494f-9bb6-48280b7350b8\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.631565 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.631458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:50.732290 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.732251    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:50.732461 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.732349    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qd4w7\" (UniqueName: \"kubernetes.io/projected/e146d349-da82-494f-9bb6-48280b7350b8-kube-api-access-qd4w7\") pod \"llmisvc-controller-manager-6b94ff949c-lxvlv\" (UID: \"e146d349-da82-494f-9bb6-48280b7350b8\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.732461 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.732389    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvjv\" (UniqueName: \"kubernetes.io/projected/a73e9ee4-0f44-461c-8064-d18cf204c4f9-kube-api-access-bgvjv\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:50.732461 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:53:50.732408    2562 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jun 02 15:53:50.732461 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.732416    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e146d349-da82-494f-9bb6-48280b7350b8-cert\") pod \"llmisvc-controller-manager-6b94ff949c-lxvlv\" (UID: \"e146d349-da82-494f-9bb6-48280b7350b8\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.732622 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:53:50.732483    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert podName:a73e9ee4-0f44-461c-8064-d18cf204c4f9 nodeName:}" failed. No retries permitted until 2026-06-02 15:53:51.2324612 +0000 UTC m=+437.130181580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert") pod "kserve-controller-manager-7b649f477f-z9x6c" (UID: "a73e9ee4-0f44-461c-8064-d18cf204c4f9") : secret "kserve-webhook-server-cert" not found
Jun 02 15:53:50.735008 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.734980    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e146d349-da82-494f-9bb6-48280b7350b8-cert\") pod \"llmisvc-controller-manager-6b94ff949c-lxvlv\" (UID: \"e146d349-da82-494f-9bb6-48280b7350b8\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.743136 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.743104    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvjv\" (UniqueName: \"kubernetes.io/projected/a73e9ee4-0f44-461c-8064-d18cf204c4f9-kube-api-access-bgvjv\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:50.743475 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.743456    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd4w7\" (UniqueName: \"kubernetes.io/projected/e146d349-da82-494f-9bb6-48280b7350b8-kube-api-access-qd4w7\") pod \"llmisvc-controller-manager-6b94ff949c-lxvlv\" (UID: \"e146d349-da82-494f-9bb6-48280b7350b8\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.857731 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.857652    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:50.987204 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:50.987048    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"]
Jun 02 15:53:50.990017 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:53:50.989984    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode146d349_da82_494f_9bb6_48280b7350b8.slice/crio-1a6f1fbbf58fc857b0a2fe86a049d2ab5634045b5f0bdaa47d8aeb260dfb6ccf WatchSource:0}: Error finding container 1a6f1fbbf58fc857b0a2fe86a049d2ab5634045b5f0bdaa47d8aeb260dfb6ccf: Status 404 returned error can't find the container with id 1a6f1fbbf58fc857b0a2fe86a049d2ab5634045b5f0bdaa47d8aeb260dfb6ccf
Jun 02 15:53:51.006291 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:51.006263    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv" event={"ID":"e146d349-da82-494f-9bb6-48280b7350b8","Type":"ContainerStarted","Data":"1a6f1fbbf58fc857b0a2fe86a049d2ab5634045b5f0bdaa47d8aeb260dfb6ccf"}
Jun 02 15:53:51.237374 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:51.237328    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:51.239712 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:51.239692    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert\") pod \"kserve-controller-manager-7b649f477f-z9x6c\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") " pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:51.452761 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:51.452719    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:51.582467 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:51.582433    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-z9x6c"]
Jun 02 15:53:51.585317 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:53:51.585287    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73e9ee4_0f44_461c_8064_d18cf204c4f9.slice/crio-0125533b0ef38631f2769a4f04b3df6d5be6af83b2104f134bb020d36910bf1d WatchSource:0}: Error finding container 0125533b0ef38631f2769a4f04b3df6d5be6af83b2104f134bb020d36910bf1d: Status 404 returned error can't find the container with id 0125533b0ef38631f2769a4f04b3df6d5be6af83b2104f134bb020d36910bf1d
Jun 02 15:53:52.010965 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:52.010904    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c" event={"ID":"a73e9ee4-0f44-461c-8064-d18cf204c4f9","Type":"ContainerStarted","Data":"0125533b0ef38631f2769a4f04b3df6d5be6af83b2104f134bb020d36910bf1d"}
Jun 02 15:53:55.022580 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:55.022529    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c" event={"ID":"a73e9ee4-0f44-461c-8064-d18cf204c4f9","Type":"ContainerStarted","Data":"9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5"}
Jun 02 15:53:55.023036 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:55.022716    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:53:55.023960 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:55.023937    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv" event={"ID":"e146d349-da82-494f-9bb6-48280b7350b8","Type":"ContainerStarted","Data":"17906fefd369d0e9d81696a915043652ca80b03add62e349207a63c2e7311f21"}
Jun 02 15:53:55.024076 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:55.024061    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:53:55.041062 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:53:55.041019    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c" podStartSLOduration=1.802144131 podStartE2EDuration="5.041007229s" podCreationTimestamp="2026-06-02 15:53:50 +0000 UTC" firstStartedPulling="2026-06-02 15:53:51.586770438 +0000 UTC m=+437.484490805" lastFinishedPulling="2026-06-02 15:53:54.825633534 +0000 UTC m=+440.723353903" observedRunningTime="2026-06-02 15:53:55.040489804 +0000 UTC m=+440.938210191" watchObservedRunningTime="2026-06-02 15:53:55.041007229 +0000 UTC m=+440.938727654"
Jun 02 15:54:26.030211 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:26.030181    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv"
Jun 02 15:54:26.033198 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:26.033178    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:54:26.049791 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:26.049716    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-6b94ff949c-lxvlv" podStartSLOduration=32.220633215 podStartE2EDuration="36.049698209s" podCreationTimestamp="2026-06-02 15:53:50 +0000 UTC" firstStartedPulling="2026-06-02 15:53:50.991266723 +0000 UTC m=+436.888987089" lastFinishedPulling="2026-06-02 15:53:54.820331707 +0000 UTC m=+440.718052083" observedRunningTime="2026-06-02 15:53:55.057011733 +0000 UTC m=+440.954732120" watchObservedRunningTime="2026-06-02 15:54:26.049698209 +0000 UTC m=+471.947418598"
Jun 02 15:54:27.279007 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.278973    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-z9x6c"]
Jun 02 15:54:27.279425 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.279214    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c" podUID="a73e9ee4-0f44-461c-8064-d18cf204c4f9" containerName="manager" containerID="cri-o://9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5" gracePeriod=10
Jun 02 15:54:27.304287 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.304261    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-m9459"]
Jun 02 15:54:27.368658 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.368631    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-m9459"]
Jun 02 15:54:27.368776 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.368754    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.526391 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.526365    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:54:27.543566 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.543495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef893910-fee3-4249-88a9-b521442ea7d8-cert\") pod \"kserve-controller-manager-7b649f477f-m9459\" (UID: \"ef893910-fee3-4249-88a9-b521442ea7d8\") " pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.543678 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.543560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f28pn\" (UniqueName: \"kubernetes.io/projected/ef893910-fee3-4249-88a9-b521442ea7d8-kube-api-access-f28pn\") pod \"kserve-controller-manager-7b649f477f-m9459\" (UID: \"ef893910-fee3-4249-88a9-b521442ea7d8\") " pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.644229 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.644188    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgvjv\" (UniqueName: \"kubernetes.io/projected/a73e9ee4-0f44-461c-8064-d18cf204c4f9-kube-api-access-bgvjv\") pod \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") "
Jun 02 15:54:27.644391 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.644271    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert\") pod \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\" (UID: \"a73e9ee4-0f44-461c-8064-d18cf204c4f9\") "
Jun 02 15:54:27.644464 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.644445    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef893910-fee3-4249-88a9-b521442ea7d8-cert\") pod \"kserve-controller-manager-7b649f477f-m9459\" (UID: \"ef893910-fee3-4249-88a9-b521442ea7d8\") " pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.644515 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.644498    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f28pn\" (UniqueName: \"kubernetes.io/projected/ef893910-fee3-4249-88a9-b521442ea7d8-kube-api-access-f28pn\") pod \"kserve-controller-manager-7b649f477f-m9459\" (UID: \"ef893910-fee3-4249-88a9-b521442ea7d8\") " pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.646438 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.646401    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert" (OuterVolumeSpecName: "cert") pod "a73e9ee4-0f44-461c-8064-d18cf204c4f9" (UID: "a73e9ee4-0f44-461c-8064-d18cf204c4f9"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:54:27.646560 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.646510    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73e9ee4-0f44-461c-8064-d18cf204c4f9-kube-api-access-bgvjv" (OuterVolumeSpecName: "kube-api-access-bgvjv") pod "a73e9ee4-0f44-461c-8064-d18cf204c4f9" (UID: "a73e9ee4-0f44-461c-8064-d18cf204c4f9"). InnerVolumeSpecName "kube-api-access-bgvjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:54:27.646736 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.646717    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef893910-fee3-4249-88a9-b521442ea7d8-cert\") pod \"kserve-controller-manager-7b649f477f-m9459\" (UID: \"ef893910-fee3-4249-88a9-b521442ea7d8\") " pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.653472 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.653444    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f28pn\" (UniqueName: \"kubernetes.io/projected/ef893910-fee3-4249-88a9-b521442ea7d8-kube-api-access-f28pn\") pod \"kserve-controller-manager-7b649f477f-m9459\" (UID: \"ef893910-fee3-4249-88a9-b521442ea7d8\") " pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.722881 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.722842    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:54:27.745394 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.745365    2562 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a73e9ee4-0f44-461c-8064-d18cf204c4f9-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:54:27.745394 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.745396    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgvjv\" (UniqueName: \"kubernetes.io/projected/a73e9ee4-0f44-461c-8064-d18cf204c4f9-kube-api-access-bgvjv\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:54:27.848641 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:27.848492    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-m9459"]
Jun 02 15:54:27.851294 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:54:27.851266    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef893910_fee3_4249_88a9_b521442ea7d8.slice/crio-4f728f6bc36c540f0d10e6212a985a02fb0074db131b9a8cee9b61dc11c85529 WatchSource:0}: Error finding container 4f728f6bc36c540f0d10e6212a985a02fb0074db131b9a8cee9b61dc11c85529: Status 404 returned error can't find the container with id 4f728f6bc36c540f0d10e6212a985a02fb0074db131b9a8cee9b61dc11c85529
Jun 02 15:54:28.129443 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.129357    2562 generic.go:358] "Generic (PLEG): container finished" podID="a73e9ee4-0f44-461c-8064-d18cf204c4f9" containerID="9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5" exitCode=0
Jun 02 15:54:28.129443 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.129419    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c"
Jun 02 15:54:28.129727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.129447    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c" event={"ID":"a73e9ee4-0f44-461c-8064-d18cf204c4f9","Type":"ContainerDied","Data":"9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5"}
Jun 02 15:54:28.129727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.129482    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-z9x6c" event={"ID":"a73e9ee4-0f44-461c-8064-d18cf204c4f9","Type":"ContainerDied","Data":"0125533b0ef38631f2769a4f04b3df6d5be6af83b2104f134bb020d36910bf1d"}
Jun 02 15:54:28.129727 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.129497    2562 scope.go:117] "RemoveContainer" containerID="9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5"
Jun 02 15:54:28.130643 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.130612    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-m9459" event={"ID":"ef893910-fee3-4249-88a9-b521442ea7d8","Type":"ContainerStarted","Data":"4f728f6bc36c540f0d10e6212a985a02fb0074db131b9a8cee9b61dc11c85529"}
Jun 02 15:54:28.137711 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.137688    2562 scope.go:117] "RemoveContainer" containerID="9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5"
Jun 02 15:54:28.137989 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:54:28.137967    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5\": container with ID starting with 9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5 not found: ID does not exist" containerID="9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5"
Jun 02 15:54:28.138047 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.138001    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5"} err="failed to get container status \"9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5\": rpc error: code = NotFound desc = could not find container \"9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5\": container with ID starting with 9a0d142678b31bd6212a2facb7ee25614fa07cfdc738ec0101aa4b72b43a26c5 not found: ID does not exist"
Jun 02 15:54:28.151201 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.151177    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-z9x6c"]
Jun 02 15:54:28.157800 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.157776    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-z9x6c"]
Jun 02 15:54:28.601403 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:28.601368    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73e9ee4-0f44-461c-8064-d18cf204c4f9" path="/var/lib/kubelet/pods/a73e9ee4-0f44-461c-8064-d18cf204c4f9/volumes"
Jun 02 15:54:29.136527 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:29.136484    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-m9459" event={"ID":"ef893910-fee3-4249-88a9-b521442ea7d8","Type":"ContainerStarted","Data":"20ff121f074b26900eec1a233d2ead1b1f8fa266063133710dd1ddaec17e59e3"}
Jun 02 15:54:29.136721 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:54:29.136608    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:55:00.143969 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:00.143937    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7b649f477f-m9459"
Jun 02 15:55:00.161877 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:00.161820    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7b649f477f-m9459" podStartSLOduration=32.625367758 podStartE2EDuration="33.161806386s" podCreationTimestamp="2026-06-02 15:54:27 +0000 UTC" firstStartedPulling="2026-06-02 15:54:27.852443741 +0000 UTC m=+473.750164106" lastFinishedPulling="2026-06-02 15:54:28.388882365 +0000 UTC m=+474.286602734" observedRunningTime="2026-06-02 15:54:29.156470417 +0000 UTC m=+475.054190805" watchObservedRunningTime="2026-06-02 15:55:00.161806386 +0000 UTC m=+506.059526774"
Jun 02 15:55:01.167772 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.167736    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-dcn2c"]
Jun 02 15:55:01.168201 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.168186    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73e9ee4-0f44-461c-8064-d18cf204c4f9" containerName="manager"
Jun 02 15:55:01.168245 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.168204    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73e9ee4-0f44-461c-8064-d18cf204c4f9" containerName="manager"
Jun 02 15:55:01.168306 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.168296    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="a73e9ee4-0f44-461c-8064-d18cf204c4f9" containerName="manager"
Jun 02 15:55:01.171572 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.171552    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.173935 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.173892    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jun 02 15:55:01.174068 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.173946    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-q7qf4\""
Jun 02 15:55:01.183272 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.183252    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-dcn2c"]
Jun 02 15:55:01.211172 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.211118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgrvs\" (UniqueName: \"kubernetes.io/projected/24489f3b-883f-4789-abec-21cfab38f508-kube-api-access-sgrvs\") pod \"odh-model-controller-696fc77849-dcn2c\" (UID: \"24489f3b-883f-4789-abec-21cfab38f508\") " pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.211320 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.211218    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24489f3b-883f-4789-abec-21cfab38f508-cert\") pod \"odh-model-controller-696fc77849-dcn2c\" (UID: \"24489f3b-883f-4789-abec-21cfab38f508\") " pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.312188 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.312148    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgrvs\" (UniqueName: \"kubernetes.io/projected/24489f3b-883f-4789-abec-21cfab38f508-kube-api-access-sgrvs\") pod \"odh-model-controller-696fc77849-dcn2c\" (UID: \"24489f3b-883f-4789-abec-21cfab38f508\") " pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.312188 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.312196    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24489f3b-883f-4789-abec-21cfab38f508-cert\") pod \"odh-model-controller-696fc77849-dcn2c\" (UID: \"24489f3b-883f-4789-abec-21cfab38f508\") " pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.314589 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.314568    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24489f3b-883f-4789-abec-21cfab38f508-cert\") pod \"odh-model-controller-696fc77849-dcn2c\" (UID: \"24489f3b-883f-4789-abec-21cfab38f508\") " pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.322429 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.322406    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgrvs\" (UniqueName: \"kubernetes.io/projected/24489f3b-883f-4789-abec-21cfab38f508-kube-api-access-sgrvs\") pod \"odh-model-controller-696fc77849-dcn2c\" (UID: \"24489f3b-883f-4789-abec-21cfab38f508\") " pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.482366 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.482324    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:01.612106 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:01.612077    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-dcn2c"]
Jun 02 15:55:01.615681 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:55:01.615654    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24489f3b_883f_4789_abec_21cfab38f508.slice/crio-16a7aae029561528600187e9029d406f1d44e8c5f6c4efba46f7bc9199e428f1 WatchSource:0}: Error finding container 16a7aae029561528600187e9029d406f1d44e8c5f6c4efba46f7bc9199e428f1: Status 404 returned error can't find the container with id 16a7aae029561528600187e9029d406f1d44e8c5f6c4efba46f7bc9199e428f1
Jun 02 15:55:02.247206 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:02.247159    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-dcn2c" event={"ID":"24489f3b-883f-4789-abec-21cfab38f508","Type":"ContainerStarted","Data":"16a7aae029561528600187e9029d406f1d44e8c5f6c4efba46f7bc9199e428f1"}
Jun 02 15:55:04.255285 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:04.255254    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-dcn2c" event={"ID":"24489f3b-883f-4789-abec-21cfab38f508","Type":"ContainerStarted","Data":"a1ab55edd8d67d7a37b0087903dc3101a3cc1ed67ff454d37ce210fe4ba855dd"}
Jun 02 15:55:04.255707 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:04.255385    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:04.272755 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:04.272708    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-dcn2c" podStartSLOduration=0.81087916 podStartE2EDuration="3.272694255s" podCreationTimestamp="2026-06-02 15:55:01 +0000 UTC" firstStartedPulling="2026-06-02 15:55:01.616834885 +0000 UTC m=+507.514555251" lastFinishedPulling="2026-06-02 15:55:04.078649966 +0000 UTC m=+509.976370346" observedRunningTime="2026-06-02 15:55:04.271024798 +0000 UTC m=+510.168745186" watchObservedRunningTime="2026-06-02 15:55:04.272694255 +0000 UTC m=+510.170414642"
Jun 02 15:55:15.260872 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:15.260788    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-dcn2c"
Jun 02 15:55:36.157562 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.157519    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"]
Jun 02 15:55:36.201895 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.201860    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"]
Jun 02 15:55:36.202114 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.202016    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.204639 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.204611    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g84l2\""
Jun 02 15:55:36.204639 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.204621    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-baff1-kube-rbac-proxy-sar-config\""
Jun 02 15:55:36.204805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.204622    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jun 02 15:55:36.204805 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.204659    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-baff1-predictor-serving-cert\""
Jun 02 15:55:36.205013 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.205000    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jun 02 15:55:36.313784 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.313753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf07e785-a5da-493e-83d3-e68225bd2fef-proxy-tls\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.313784 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.313789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nv7l\" (UniqueName: \"kubernetes.io/projected/cf07e785-a5da-493e-83d3-e68225bd2fef-kube-api-access-6nv7l\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.314022 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.313958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-baff1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf07e785-a5da-493e-83d3-e68225bd2fef-success-200-isvc-baff1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.415416 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.415324    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-baff1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf07e785-a5da-493e-83d3-e68225bd2fef-success-200-isvc-baff1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.415416 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.415397    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf07e785-a5da-493e-83d3-e68225bd2fef-proxy-tls\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.415615 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.415421    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nv7l\" (UniqueName: \"kubernetes.io/projected/cf07e785-a5da-493e-83d3-e68225bd2fef-kube-api-access-6nv7l\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.416039 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.416007    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-baff1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf07e785-a5da-493e-83d3-e68225bd2fef-success-200-isvc-baff1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.418244 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.418219    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf07e785-a5da-493e-83d3-e68225bd2fef-proxy-tls\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.424618 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.424584    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nv7l\" (UniqueName: \"kubernetes.io/projected/cf07e785-a5da-493e-83d3-e68225bd2fef-kube-api-access-6nv7l\") pod \"success-200-isvc-baff1-predictor-58687f74b9-vqxzz\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") " pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.513713 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.513676    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:36.674241 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:36.674146    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"]
Jun 02 15:55:36.677022 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:55:36.676990    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf07e785_a5da_493e_83d3_e68225bd2fef.slice/crio-f4fa02727c52bdb0a9fe9458477eb7b90a27f22397afde8fae18496e2b050f76 WatchSource:0}: Error finding container f4fa02727c52bdb0a9fe9458477eb7b90a27f22397afde8fae18496e2b050f76: Status 404 returned error can't find the container with id f4fa02727c52bdb0a9fe9458477eb7b90a27f22397afde8fae18496e2b050f76
Jun 02 15:55:37.361864 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:37.361829    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" event={"ID":"cf07e785-a5da-493e-83d3-e68225bd2fef","Type":"ContainerStarted","Data":"f4fa02727c52bdb0a9fe9458477eb7b90a27f22397afde8fae18496e2b050f76"}
Jun 02 15:55:49.414175 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:49.414087    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" event={"ID":"cf07e785-a5da-493e-83d3-e68225bd2fef","Type":"ContainerStarted","Data":"2dd370208ca8d13602f10be4702c99e9030097b3ff1e82b281ef66dbb0ab6d5c"}
Jun 02 15:55:52.425368 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:52.425323    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" event={"ID":"cf07e785-a5da-493e-83d3-e68225bd2fef","Type":"ContainerStarted","Data":"dab2f329e2f5d49ab2f6e458e087ce9039eaba5e895a48a4a7bac7f72c4813bf"}
Jun 02 15:55:52.425858 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:52.425635    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:52.425858 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:52.425747    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:52.427256 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:52.427221    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jun 02 15:55:52.450743 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:52.450695    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podStartSLOduration=1.75447062 podStartE2EDuration="16.450680027s" podCreationTimestamp="2026-06-02 15:55:36 +0000 UTC" firstStartedPulling="2026-06-02 15:55:36.679048407 +0000 UTC m=+542.576768775" lastFinishedPulling="2026-06-02 15:55:51.375257796 +0000 UTC m=+557.272978182" observedRunningTime="2026-06-02 15:55:52.449746991 +0000 UTC m=+558.347467378" watchObservedRunningTime="2026-06-02 15:55:52.450680027 +0000 UTC m=+558.348400414"
Jun 02 15:55:53.428584 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:53.428538    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jun 02 15:55:58.433145 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:58.433115    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:55:58.433686 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:55:58.433660    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jun 02 15:56:08.434038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:08.433996    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jun 02 15:56:18.433654 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:18.433609    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jun 02 15:56:28.433697 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:28.433653    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jun 02 15:56:34.503090 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:34.503053    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:56:34.503717 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:34.503587    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 15:56:38.434899 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:38.434871    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:56:56.220934 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.220832    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"]
Jun 02 15:56:56.224225 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.224207    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.226253 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.226229    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-baff1-serving-cert\""
Jun 02 15:56:56.226342 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.226229    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-baff1-kube-rbac-proxy-sar-config\""
Jun 02 15:56:56.232756 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.232735    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"]
Jun 02 15:56:56.319491 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.319443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd907f82-51af-4797-a2fc-e34dbb7fff73-openshift-service-ca-bundle\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.319491 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.319493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.420974 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.420929    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd907f82-51af-4797-a2fc-e34dbb7fff73-openshift-service-ca-bundle\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.421156 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.421003    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.421156 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:56:56.421144    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-baff1-serving-cert: secret "switch-graph-baff1-serving-cert" not found
Jun 02 15:56:56.421240 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:56:56.421208    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls podName:dd907f82-51af-4797-a2fc-e34dbb7fff73 nodeName:}" failed. No retries permitted until 2026-06-02 15:56:56.92119252 +0000 UTC m=+622.818912891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls") pod "switch-graph-baff1-5ddf668d85-fbqdl" (UID: "dd907f82-51af-4797-a2fc-e34dbb7fff73") : secret "switch-graph-baff1-serving-cert" not found
Jun 02 15:56:56.421719 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.421699    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd907f82-51af-4797-a2fc-e34dbb7fff73-openshift-service-ca-bundle\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.924776 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.924743    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:56.927262 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:56.927229    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls\") pod \"switch-graph-baff1-5ddf668d85-fbqdl\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") " pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:57.135282 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:57.135244    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:56:57.263753 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:57.263725    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"]
Jun 02 15:56:57.265852 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:56:57.265823    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd907f82_51af_4797_a2fc_e34dbb7fff73.slice/crio-1b2ac35a35372805c95d7bb5f62e0da3530dbf87716d2b486975f544224f9222 WatchSource:0}: Error finding container 1b2ac35a35372805c95d7bb5f62e0da3530dbf87716d2b486975f544224f9222: Status 404 returned error can't find the container with id 1b2ac35a35372805c95d7bb5f62e0da3530dbf87716d2b486975f544224f9222
Jun 02 15:56:57.627707 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:56:57.627665    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" event={"ID":"dd907f82-51af-4797-a2fc-e34dbb7fff73","Type":"ContainerStarted","Data":"1b2ac35a35372805c95d7bb5f62e0da3530dbf87716d2b486975f544224f9222"}
Jun 02 15:57:00.638706 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:00.638673    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" event={"ID":"dd907f82-51af-4797-a2fc-e34dbb7fff73","Type":"ContainerStarted","Data":"19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc"}
Jun 02 15:57:00.639128 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:00.638776    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:57:00.655886 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:00.655832    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podStartSLOduration=2.294134773 podStartE2EDuration="4.655815122s" podCreationTimestamp="2026-06-02 15:56:56 +0000 UTC" firstStartedPulling="2026-06-02 15:56:57.267673957 +0000 UTC m=+623.165394324" lastFinishedPulling="2026-06-02 15:56:59.629354295 +0000 UTC m=+625.527074673" observedRunningTime="2026-06-02 15:57:00.655514356 +0000 UTC m=+626.553234743" watchObservedRunningTime="2026-06-02 15:57:00.655815122 +0000 UTC m=+626.553535511"
Jun 02 15:57:06.647739 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:06.647710    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:57:10.363732 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.363695    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"]
Jun 02 15:57:10.366181 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.363938    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" containerID="cri-o://19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc" gracePeriod=30
Jun 02 15:57:10.509090 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.509056    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"]
Jun 02 15:57:10.509366 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.509340    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container" containerID="cri-o://2dd370208ca8d13602f10be4702c99e9030097b3ff1e82b281ef66dbb0ab6d5c" gracePeriod=30
Jun 02 15:57:10.509460 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.509379    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kube-rbac-proxy" containerID="cri-o://dab2f329e2f5d49ab2f6e458e087ce9039eaba5e895a48a4a7bac7f72c4813bf" gracePeriod=30
Jun 02 15:57:10.613485 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.613454    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"]
Jun 02 15:57:10.624867 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.623645    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.626437 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.626394    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c6170-kube-rbac-proxy-sar-config\""
Jun 02 15:57:10.626679 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.626392    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c6170-predictor-serving-cert\""
Jun 02 15:57:10.633429 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.633377    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"]
Jun 02 15:57:10.641093 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.640828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-proxy-tls\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.641093 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.640900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-c6170-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-success-200-isvc-c6170-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.641093 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.640983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9lvl\" (UniqueName: \"kubernetes.io/projected/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-kube-api-access-b9lvl\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.672038 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.672004    2562 generic.go:358] "Generic (PLEG): container finished" podID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerID="dab2f329e2f5d49ab2f6e458e087ce9039eaba5e895a48a4a7bac7f72c4813bf" exitCode=2
Jun 02 15:57:10.672207 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.672034    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" event={"ID":"cf07e785-a5da-493e-83d3-e68225bd2fef","Type":"ContainerDied","Data":"dab2f329e2f5d49ab2f6e458e087ce9039eaba5e895a48a4a7bac7f72c4813bf"}
Jun 02 15:57:10.741693 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.741649    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-proxy-tls\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.741885 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.741708    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-c6170-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-success-200-isvc-c6170-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.741885 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.741758    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b9lvl\" (UniqueName: \"kubernetes.io/projected/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-kube-api-access-b9lvl\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.742512 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.742485    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-c6170-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-success-200-isvc-c6170-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.744269 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.744241    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-proxy-tls\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.754532 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.754509    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9lvl\" (UniqueName: \"kubernetes.io/projected/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-kube-api-access-b9lvl\") pod \"success-200-isvc-c6170-predictor-76675d7967-6zrmk\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") " pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:10.946790 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:10.946762    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:11.073598 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.073565    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"]
Jun 02 15:57:11.077227 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:57:11.077197    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb278ebb_a0df_49cf_9398_b2e740dbc8e8.slice/crio-0c9ab9165b4ee6b7fd2ee606bd350d57231b7598ac3858f228f8b91c37acd5a5 WatchSource:0}: Error finding container 0c9ab9165b4ee6b7fd2ee606bd350d57231b7598ac3858f228f8b91c37acd5a5: Status 404 returned error can't find the container with id 0c9ab9165b4ee6b7fd2ee606bd350d57231b7598ac3858f228f8b91c37acd5a5
Jun 02 15:57:11.079036 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.079017    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 15:57:11.645381 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.645347    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:11.676985 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.676941    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" event={"ID":"eb278ebb-a0df-49cf-9398-b2e740dbc8e8","Type":"ContainerStarted","Data":"b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f"}
Jun 02 15:57:11.676985 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.676984    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" event={"ID":"eb278ebb-a0df-49cf-9398-b2e740dbc8e8","Type":"ContainerStarted","Data":"575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe"}
Jun 02 15:57:11.677182 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.677000    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" event={"ID":"eb278ebb-a0df-49cf-9398-b2e740dbc8e8","Type":"ContainerStarted","Data":"0c9ab9165b4ee6b7fd2ee606bd350d57231b7598ac3858f228f8b91c37acd5a5"}
Jun 02 15:57:11.677182 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.677095    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:11.697010 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:11.696961    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podStartSLOduration=1.6969321960000001 podStartE2EDuration="1.696932196s" podCreationTimestamp="2026-06-02 15:57:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:57:11.694581719 +0000 UTC m=+637.592302107" watchObservedRunningTime="2026-06-02 15:57:11.696932196 +0000 UTC m=+637.594652583"
Jun 02 15:57:12.680544 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:12.680509    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:12.681869 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:12.681838    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jun 02 15:57:13.429115 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.429075    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jun 02 15:57:13.687337 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.687263    2562 generic.go:358] "Generic (PLEG): container finished" podID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerID="2dd370208ca8d13602f10be4702c99e9030097b3ff1e82b281ef66dbb0ab6d5c" exitCode=0
Jun 02 15:57:13.687337 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.687304    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" event={"ID":"cf07e785-a5da-493e-83d3-e68225bd2fef","Type":"ContainerDied","Data":"2dd370208ca8d13602f10be4702c99e9030097b3ff1e82b281ef66dbb0ab6d5c"}
Jun 02 15:57:13.687797 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.687692    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jun 02 15:57:13.757771 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.757745    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:57:13.765070 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.765049    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-baff1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf07e785-a5da-493e-83d3-e68225bd2fef-success-200-isvc-baff1-kube-rbac-proxy-sar-config\") pod \"cf07e785-a5da-493e-83d3-e68225bd2fef\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") "
Jun 02 15:57:13.765157 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.765081    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf07e785-a5da-493e-83d3-e68225bd2fef-proxy-tls\") pod \"cf07e785-a5da-493e-83d3-e68225bd2fef\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") "
Jun 02 15:57:13.765214 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.765165    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nv7l\" (UniqueName: \"kubernetes.io/projected/cf07e785-a5da-493e-83d3-e68225bd2fef-kube-api-access-6nv7l\") pod \"cf07e785-a5da-493e-83d3-e68225bd2fef\" (UID: \"cf07e785-a5da-493e-83d3-e68225bd2fef\") "
Jun 02 15:57:13.765387 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.765356    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf07e785-a5da-493e-83d3-e68225bd2fef-success-200-isvc-baff1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-baff1-kube-rbac-proxy-sar-config") pod "cf07e785-a5da-493e-83d3-e68225bd2fef" (UID: "cf07e785-a5da-493e-83d3-e68225bd2fef"). InnerVolumeSpecName "success-200-isvc-baff1-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:57:13.767351 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.767326    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf07e785-a5da-493e-83d3-e68225bd2fef-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cf07e785-a5da-493e-83d3-e68225bd2fef" (UID: "cf07e785-a5da-493e-83d3-e68225bd2fef"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:57:13.767351 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.767338    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf07e785-a5da-493e-83d3-e68225bd2fef-kube-api-access-6nv7l" (OuterVolumeSpecName: "kube-api-access-6nv7l") pod "cf07e785-a5da-493e-83d3-e68225bd2fef" (UID: "cf07e785-a5da-493e-83d3-e68225bd2fef"). InnerVolumeSpecName "kube-api-access-6nv7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:57:13.866509 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.866477    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nv7l\" (UniqueName: \"kubernetes.io/projected/cf07e785-a5da-493e-83d3-e68225bd2fef-kube-api-access-6nv7l\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:57:13.866509 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.866504    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-baff1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf07e785-a5da-493e-83d3-e68225bd2fef-success-200-isvc-baff1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:57:13.866509 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:13.866515    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf07e785-a5da-493e-83d3-e68225bd2fef-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:57:14.691946 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:14.691892    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz" event={"ID":"cf07e785-a5da-493e-83d3-e68225bd2fef","Type":"ContainerDied","Data":"f4fa02727c52bdb0a9fe9458477eb7b90a27f22397afde8fae18496e2b050f76"}
Jun 02 15:57:14.692357 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:14.691960    2562 scope.go:117] "RemoveContainer" containerID="dab2f329e2f5d49ab2f6e458e087ce9039eaba5e895a48a4a7bac7f72c4813bf"
Jun 02 15:57:14.692357 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:14.691976    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"
Jun 02 15:57:14.699781 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:14.699762    2562 scope.go:117] "RemoveContainer" containerID="2dd370208ca8d13602f10be4702c99e9030097b3ff1e82b281ef66dbb0ab6d5c"
Jun 02 15:57:14.711878 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:14.711851    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"]
Jun 02 15:57:14.723369 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:14.723342    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-baff1-predictor-58687f74b9-vqxzz"]
Jun 02 15:57:16.601604 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:16.601568    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" path="/var/lib/kubelet/pods/cf07e785-a5da-493e-83d3-e68225bd2fef/volumes"
Jun 02 15:57:16.646229 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:16.646194    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:18.692307 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:18.692276    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:18.692852 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:18.692825    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jun 02 15:57:21.645466 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:21.645430    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:21.645852 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:21.645537    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:57:26.646168 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:26.646129    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:28.693056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:28.693018    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jun 02 15:57:31.645436 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:31.645397    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:36.645881 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:36.645843    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:38.692883 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:38.692838    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jun 02 15:57:40.392841 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:57:40.392799    2562 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd907f82_51af_4797_a2fc_e34dbb7fff73.slice/crio-conmon-19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc.scope\": RecentStats: unable to find data in memory cache]"
Jun 02 15:57:40.393276 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:57:40.392838    2562 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd907f82_51af_4797_a2fc_e34dbb7fff73.slice/crio-conmon-19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc.scope\": RecentStats: unable to find data in memory cache]"
Jun 02 15:57:40.774150 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:40.774115    2562 generic.go:358] "Generic (PLEG): container finished" podID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerID="19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc" exitCode=0
Jun 02 15:57:40.774330 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:40.774189    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" event={"ID":"dd907f82-51af-4797-a2fc-e34dbb7fff73","Type":"ContainerDied","Data":"19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc"}
Jun 02 15:57:41.014229 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.014202    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:57:41.103509 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.103409    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd907f82-51af-4797-a2fc-e34dbb7fff73-openshift-service-ca-bundle\") pod \"dd907f82-51af-4797-a2fc-e34dbb7fff73\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") "
Jun 02 15:57:41.103668 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.103513    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls\") pod \"dd907f82-51af-4797-a2fc-e34dbb7fff73\" (UID: \"dd907f82-51af-4797-a2fc-e34dbb7fff73\") "
Jun 02 15:57:41.103794 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.103767    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd907f82-51af-4797-a2fc-e34dbb7fff73-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "dd907f82-51af-4797-a2fc-e34dbb7fff73" (UID: "dd907f82-51af-4797-a2fc-e34dbb7fff73"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:57:41.105620 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.105587    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dd907f82-51af-4797-a2fc-e34dbb7fff73" (UID: "dd907f82-51af-4797-a2fc-e34dbb7fff73"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:57:41.204991 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.204954    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd907f82-51af-4797-a2fc-e34dbb7fff73-openshift-service-ca-bundle\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:57:41.204991 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.204983    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd907f82-51af-4797-a2fc-e34dbb7fff73-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:57:41.778488 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.778463    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"
Jun 02 15:57:41.778888 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.778459    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl" event={"ID":"dd907f82-51af-4797-a2fc-e34dbb7fff73","Type":"ContainerDied","Data":"1b2ac35a35372805c95d7bb5f62e0da3530dbf87716d2b486975f544224f9222"}
Jun 02 15:57:41.778888 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.778579    2562 scope.go:117] "RemoveContainer" containerID="19cd4ad1033444d1411a730d85ec0a2d0de9cb1d554bae5f94e5a5da310870cc"
Jun 02 15:57:41.800976 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.800952    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"]
Jun 02 15:57:41.805733 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:41.805713    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-baff1-5ddf668d85-fbqdl"]
Jun 02 15:57:42.600445 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:42.600411    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" path="/var/lib/kubelet/pods/dd907f82-51af-4797-a2fc-e34dbb7fff73/volumes"
Jun 02 15:57:46.281663 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.281629    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"]
Jun 02 15:57:46.282056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.281962    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container"
Jun 02 15:57:46.282056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.281974    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container"
Jun 02 15:57:46.282056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.281987    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kube-rbac-proxy"
Jun 02 15:57:46.282056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.281993    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kube-rbac-proxy"
Jun 02 15:57:46.282056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.282008    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1"
Jun 02 15:57:46.282056 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.282015    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1"
Jun 02 15:57:46.282275 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.282063    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="dd907f82-51af-4797-a2fc-e34dbb7fff73" containerName="switch-graph-baff1"
Jun 02 15:57:46.282275 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.282072    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kube-rbac-proxy"
Jun 02 15:57:46.282275 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.282079    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="cf07e785-a5da-493e-83d3-e68225bd2fef" containerName="kserve-container"
Jun 02 15:57:46.284985 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.284965    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.287238 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.286994    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Jun 02 15:57:46.287446 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.287430    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Jun 02 15:57:46.296347 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.296325    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"]
Jun 02 15:57:46.353222 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.353186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be060c2b-6892-420e-a7ea-9c058c571bc7-proxy-tls\") pod \"model-chainer-554844c84b-9c2tm\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") " pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.353404 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.353276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be060c2b-6892-420e-a7ea-9c058c571bc7-openshift-service-ca-bundle\") pod \"model-chainer-554844c84b-9c2tm\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") " pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.454067 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.454033    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be060c2b-6892-420e-a7ea-9c058c571bc7-proxy-tls\") pod \"model-chainer-554844c84b-9c2tm\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") " pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.454248 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.454224    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be060c2b-6892-420e-a7ea-9c058c571bc7-openshift-service-ca-bundle\") pod \"model-chainer-554844c84b-9c2tm\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") " pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.454830 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.454795    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be060c2b-6892-420e-a7ea-9c058c571bc7-openshift-service-ca-bundle\") pod \"model-chainer-554844c84b-9c2tm\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") " pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.456608 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.456572    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be060c2b-6892-420e-a7ea-9c058c571bc7-proxy-tls\") pod \"model-chainer-554844c84b-9c2tm\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") " pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.594989 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.594870    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.724483 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.724451    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"]
Jun 02 15:57:46.727048 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:57:46.727022    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe060c2b_6892_420e_a7ea_9c058c571bc7.slice/crio-bfff1e798a89bce6def6e1d10788fc738fe627430c9604b2e39f6c6f6c7aebb3 WatchSource:0}: Error finding container bfff1e798a89bce6def6e1d10788fc738fe627430c9604b2e39f6c6f6c7aebb3: Status 404 returned error can't find the container with id bfff1e798a89bce6def6e1d10788fc738fe627430c9604b2e39f6c6f6c7aebb3
Jun 02 15:57:46.797041 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.797008    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" event={"ID":"be060c2b-6892-420e-a7ea-9c058c571bc7","Type":"ContainerStarted","Data":"8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd"}
Jun 02 15:57:46.797188 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.797048    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" event={"ID":"be060c2b-6892-420e-a7ea-9c058c571bc7","Type":"ContainerStarted","Data":"bfff1e798a89bce6def6e1d10788fc738fe627430c9604b2e39f6c6f6c7aebb3"}
Jun 02 15:57:46.797188 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.797100    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:46.815987 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:46.815838    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podStartSLOduration=0.815821624 podStartE2EDuration="815.821624ms" podCreationTimestamp="2026-06-02 15:57:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:57:46.8158081 +0000 UTC m=+672.713528500" watchObservedRunningTime="2026-06-02 15:57:46.815821624 +0000 UTC m=+672.713542013"
Jun 02 15:57:48.692985 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:48.692945    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jun 02 15:57:52.806090 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:52.806060    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:57:56.363095 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.363056    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"]
Jun 02 15:57:56.363546 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.363368    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" containerID="cri-o://8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd" gracePeriod=30
Jun 02 15:57:56.588145 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.588111    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"]
Jun 02 15:57:56.592053 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.592029    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.594353 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.594334    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f26f3-kube-rbac-proxy-sar-config\""
Jun 02 15:57:56.594353 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.594348    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f26f3-predictor-serving-cert\""
Jun 02 15:57:56.605383 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.605359    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"]
Jun 02 15:57:56.745369 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.745331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nldlm\" (UniqueName: \"kubernetes.io/projected/741ee707-8c7d-4e5e-9bdd-9174d2213122-kube-api-access-nldlm\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.745540 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.745389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/741ee707-8c7d-4e5e-9bdd-9174d2213122-proxy-tls\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.745540 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.745452    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-f26f3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/741ee707-8c7d-4e5e-9bdd-9174d2213122-success-200-isvc-f26f3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.846249 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.846213    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nldlm\" (UniqueName: \"kubernetes.io/projected/741ee707-8c7d-4e5e-9bdd-9174d2213122-kube-api-access-nldlm\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.846416 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.846264    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/741ee707-8c7d-4e5e-9bdd-9174d2213122-proxy-tls\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.846416 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.846324    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-f26f3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/741ee707-8c7d-4e5e-9bdd-9174d2213122-success-200-isvc-f26f3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.847187 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.847158    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-f26f3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/741ee707-8c7d-4e5e-9bdd-9174d2213122-success-200-isvc-f26f3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.848992 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.848973    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/741ee707-8c7d-4e5e-9bdd-9174d2213122-proxy-tls\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.856875 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.856842    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nldlm\" (UniqueName: \"kubernetes.io/projected/741ee707-8c7d-4e5e-9bdd-9174d2213122-kube-api-access-nldlm\") pod \"success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") " pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:56.903863 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:56.903828    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:57.049354 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.049331    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"]
Jun 02 15:57:57.051353 ip-10-0-142-13 kubenswrapper[2562]: W0602 15:57:57.051317    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741ee707_8c7d_4e5e_9bdd_9174d2213122.slice/crio-da3a67cafadcb7ea0b0bb22d9b36583d996ad55ef0edeb390ae36b5916a2d0c5 WatchSource:0}: Error finding container da3a67cafadcb7ea0b0bb22d9b36583d996ad55ef0edeb390ae36b5916a2d0c5: Status 404 returned error can't find the container with id da3a67cafadcb7ea0b0bb22d9b36583d996ad55ef0edeb390ae36b5916a2d0c5
Jun 02 15:57:57.804646 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.804605    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:57.831569 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.831539    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" event={"ID":"741ee707-8c7d-4e5e-9bdd-9174d2213122","Type":"ContainerStarted","Data":"f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a"}
Jun 02 15:57:57.831569 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.831578    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" event={"ID":"741ee707-8c7d-4e5e-9bdd-9174d2213122","Type":"ContainerStarted","Data":"f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285"}
Jun 02 15:57:57.831801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.831590    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" event={"ID":"741ee707-8c7d-4e5e-9bdd-9174d2213122","Type":"ContainerStarted","Data":"da3a67cafadcb7ea0b0bb22d9b36583d996ad55ef0edeb390ae36b5916a2d0c5"}
Jun 02 15:57:57.831801 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.831694    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:57.853169 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:57.853113    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podStartSLOduration=1.853074634 podStartE2EDuration="1.853074634s" podCreationTimestamp="2026-06-02 15:57:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:57:57.851002945 +0000 UTC m=+683.748723333" watchObservedRunningTime="2026-06-02 15:57:57.853074634 +0000 UTC m=+683.750795023"
Jun 02 15:57:58.693075 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:58.693045    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 15:57:58.834506 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:58.834474    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:57:58.835874 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:58.835846    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jun 02 15:57:59.838443 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:57:59.838403    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jun 02 15:58:02.804503 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:02.804465    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:58:04.843711 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:04.843685    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 15:58:04.844268 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:04.844243    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jun 02 15:58:07.805085 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:07.805035    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:58:07.805474 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:07.805155    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:58:12.804975 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:12.804873    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:58:14.844871 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:14.844835    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jun 02 15:58:17.804445 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:17.804403    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:58:22.805046 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:22.805000    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:58:24.844350 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:24.844313    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jun 02 15:58:26.517933 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.517885    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:58:26.595517 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.595483    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be060c2b-6892-420e-a7ea-9c058c571bc7-proxy-tls\") pod \"be060c2b-6892-420e-a7ea-9c058c571bc7\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") "
Jun 02 15:58:26.595677 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.595538    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be060c2b-6892-420e-a7ea-9c058c571bc7-openshift-service-ca-bundle\") pod \"be060c2b-6892-420e-a7ea-9c058c571bc7\" (UID: \"be060c2b-6892-420e-a7ea-9c058c571bc7\") "
Jun 02 15:58:26.595931 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.595881    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be060c2b-6892-420e-a7ea-9c058c571bc7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "be060c2b-6892-420e-a7ea-9c058c571bc7" (UID: "be060c2b-6892-420e-a7ea-9c058c571bc7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:58:26.597903 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.597872    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be060c2b-6892-420e-a7ea-9c058c571bc7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "be060c2b-6892-420e-a7ea-9c058c571bc7" (UID: "be060c2b-6892-420e-a7ea-9c058c571bc7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:58:26.696791 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.696762    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be060c2b-6892-420e-a7ea-9c058c571bc7-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:58:26.696791 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.696790    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be060c2b-6892-420e-a7ea-9c058c571bc7-openshift-service-ca-bundle\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 15:58:26.927311 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.927275    2562 generic.go:358] "Generic (PLEG): container finished" podID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerID="8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd" exitCode=0
Jun 02 15:58:26.927472 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.927331    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"
Jun 02 15:58:26.927472 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.927363    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" event={"ID":"be060c2b-6892-420e-a7ea-9c058c571bc7","Type":"ContainerDied","Data":"8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd"}
Jun 02 15:58:26.927472 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.927398    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm" event={"ID":"be060c2b-6892-420e-a7ea-9c058c571bc7","Type":"ContainerDied","Data":"bfff1e798a89bce6def6e1d10788fc738fe627430c9604b2e39f6c6f6c7aebb3"}
Jun 02 15:58:26.927472 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.927414    2562 scope.go:117] "RemoveContainer" containerID="8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd"
Jun 02 15:58:26.935684 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.935662    2562 scope.go:117] "RemoveContainer" containerID="8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd"
Jun 02 15:58:26.935949 ip-10-0-142-13 kubenswrapper[2562]: E0602 15:58:26.935928    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd\": container with ID starting with 8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd not found: ID does not exist" containerID="8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd"
Jun 02 15:58:26.935997 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.935959    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd"} err="failed to get container status \"8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd\": rpc error: code = NotFound desc = could not find container \"8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd\": container with ID starting with 8efb45f6a469f89eb4600bbc21e63f890667daae08647ad0351f72ccf069f2fd not found: ID does not exist"
Jun 02 15:58:26.946852 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.946795    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"]
Jun 02 15:58:26.953119 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:26.953094    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-554844c84b-9c2tm"]
Jun 02 15:58:28.600977 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:28.600940    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" path="/var/lib/kubelet/pods/be060c2b-6892-420e-a7ea-9c058c571bc7/volumes"
Jun 02 15:58:34.844700 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:34.844614    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Jun 02 15:58:44.844690 ip-10-0-142-13 kubenswrapper[2562]: I0602 15:58:44.844661    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 16:01:34.526385 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:01:34.526354    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:01:34.528574 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:01:34.528554    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:06:25.366975 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.366941    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"]
Jun 02 16:06:25.367552 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.367326    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container" containerID="cri-o://575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe" gracePeriod=30
Jun 02 16:06:25.367552 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.367398    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kube-rbac-proxy" containerID="cri-o://b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f" gracePeriod=30
Jun 02 16:06:25.442646 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.442614    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"]
Jun 02 16:06:25.443010 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.442997    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer"
Jun 02 16:06:25.443066 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.443012    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer"
Jun 02 16:06:25.443104 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.443072    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="be060c2b-6892-420e-a7ea-9c058c571bc7" containerName="model-chainer"
Jun 02 16:06:25.451731 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.451709    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.453890 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.453864    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-d60e3-kube-rbac-proxy-sar-config\""
Jun 02 16:06:25.454033 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.453959    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-d60e3-predictor-serving-cert\""
Jun 02 16:06:25.464167 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.464137    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"]
Jun 02 16:06:25.502852 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.502827    2562 generic.go:358] "Generic (PLEG): container finished" podID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerID="b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f" exitCode=2
Jun 02 16:06:25.503042 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.502884    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" event={"ID":"eb278ebb-a0df-49cf-9398-b2e740dbc8e8","Type":"ContainerDied","Data":"b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f"}
Jun 02 16:06:25.553430 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.553395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncpb\" (UniqueName: \"kubernetes.io/projected/566f8d45-09ac-4243-9b55-7e110233b781-kube-api-access-lncpb\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.553430 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.553432    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/566f8d45-09ac-4243-9b55-7e110233b781-proxy-tls\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.553628 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.553532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-d60e3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/566f8d45-09ac-4243-9b55-7e110233b781-success-200-isvc-d60e3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.654755 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.654682    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-d60e3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/566f8d45-09ac-4243-9b55-7e110233b781-success-200-isvc-d60e3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.654755 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.654729    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lncpb\" (UniqueName: \"kubernetes.io/projected/566f8d45-09ac-4243-9b55-7e110233b781-kube-api-access-lncpb\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.654978 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.654848    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/566f8d45-09ac-4243-9b55-7e110233b781-proxy-tls\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.655351 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.655332    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-d60e3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/566f8d45-09ac-4243-9b55-7e110233b781-success-200-isvc-d60e3-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.657150 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.657131    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/566f8d45-09ac-4243-9b55-7e110233b781-proxy-tls\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.665564 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.665543    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncpb\" (UniqueName: \"kubernetes.io/projected/566f8d45-09ac-4243-9b55-7e110233b781-kube-api-access-lncpb\") pod \"success-200-isvc-d60e3-predictor-96cb78977-5txz5\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") " pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.764535 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.764501    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:25.893072 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.893019    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"]
Jun 02 16:06:25.896858 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:06:25.896829    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566f8d45_09ac_4243_9b55_7e110233b781.slice/crio-44fb553621b5be68c67a281acd32c7df8546483cbac9e02d7afe307c3233c254 WatchSource:0}: Error finding container 44fb553621b5be68c67a281acd32c7df8546483cbac9e02d7afe307c3233c254: Status 404 returned error can't find the container with id 44fb553621b5be68c67a281acd32c7df8546483cbac9e02d7afe307c3233c254
Jun 02 16:06:25.898644 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:25.898628    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 16:06:26.507745 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:26.507708    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" event={"ID":"566f8d45-09ac-4243-9b55-7e110233b781","Type":"ContainerStarted","Data":"0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906"}
Jun 02 16:06:26.507745 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:26.507747    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" event={"ID":"566f8d45-09ac-4243-9b55-7e110233b781","Type":"ContainerStarted","Data":"de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37"}
Jun 02 16:06:26.508152 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:26.507761    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" event={"ID":"566f8d45-09ac-4243-9b55-7e110233b781","Type":"ContainerStarted","Data":"44fb553621b5be68c67a281acd32c7df8546483cbac9e02d7afe307c3233c254"}
Jun 02 16:06:26.508152 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:26.507845    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:26.529694 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:26.529647    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podStartSLOduration=1.529634524 podStartE2EDuration="1.529634524s" podCreationTimestamp="2026-06-02 16:06:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:06:26.527304612 +0000 UTC m=+1192.425025000" watchObservedRunningTime="2026-06-02 16:06:26.529634524 +0000 UTC m=+1192.427354911"
Jun 02 16:06:27.510847 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:27.510812    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:27.512047 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:27.512017    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jun 02 16:06:28.407005 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.406983    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 16:06:28.479211 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.479133    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-proxy-tls\") pod \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") "
Jun 02 16:06:28.479211 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.479183    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9lvl\" (UniqueName: \"kubernetes.io/projected/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-kube-api-access-b9lvl\") pod \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") "
Jun 02 16:06:28.479407 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.479211    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-c6170-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-success-200-isvc-c6170-kube-rbac-proxy-sar-config\") pod \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\" (UID: \"eb278ebb-a0df-49cf-9398-b2e740dbc8e8\") "
Jun 02 16:06:28.479574 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.479549    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-success-200-isvc-c6170-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-c6170-kube-rbac-proxy-sar-config") pod "eb278ebb-a0df-49cf-9398-b2e740dbc8e8" (UID: "eb278ebb-a0df-49cf-9398-b2e740dbc8e8"). InnerVolumeSpecName "success-200-isvc-c6170-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:06:28.481239 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.481207    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-kube-api-access-b9lvl" (OuterVolumeSpecName: "kube-api-access-b9lvl") pod "eb278ebb-a0df-49cf-9398-b2e740dbc8e8" (UID: "eb278ebb-a0df-49cf-9398-b2e740dbc8e8"). InnerVolumeSpecName "kube-api-access-b9lvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:06:28.481330 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.481235    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "eb278ebb-a0df-49cf-9398-b2e740dbc8e8" (UID: "eb278ebb-a0df-49cf-9398-b2e740dbc8e8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:06:28.515132 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.515107    2562 generic.go:358] "Generic (PLEG): container finished" podID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerID="575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe" exitCode=0
Jun 02 16:06:28.515423 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.515204    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"
Jun 02 16:06:28.515423 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.515198    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" event={"ID":"eb278ebb-a0df-49cf-9398-b2e740dbc8e8","Type":"ContainerDied","Data":"575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe"}
Jun 02 16:06:28.515423 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.515304    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk" event={"ID":"eb278ebb-a0df-49cf-9398-b2e740dbc8e8","Type":"ContainerDied","Data":"0c9ab9165b4ee6b7fd2ee606bd350d57231b7598ac3858f228f8b91c37acd5a5"}
Jun 02 16:06:28.515423 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.515321    2562 scope.go:117] "RemoveContainer" containerID="b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f"
Jun 02 16:06:28.515661 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.515636    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jun 02 16:06:28.524099 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.524085    2562 scope.go:117] "RemoveContainer" containerID="575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe"
Jun 02 16:06:28.531005 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.530988    2562 scope.go:117] "RemoveContainer" containerID="b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f"
Jun 02 16:06:28.531250 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:06:28.531230    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f\": container with ID starting with b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f not found: ID does not exist" containerID="b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f"
Jun 02 16:06:28.531290 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.531260    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f"} err="failed to get container status \"b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f\": rpc error: code = NotFound desc = could not find container \"b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f\": container with ID starting with b4fdc0b89a8bc53f29d5f2e0d10ff965bee08c2a7ebb5f2531a771900237a78f not found: ID does not exist"
Jun 02 16:06:28.531290 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.531282    2562 scope.go:117] "RemoveContainer" containerID="575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe"
Jun 02 16:06:28.531486 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:06:28.531472    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe\": container with ID starting with 575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe not found: ID does not exist" containerID="575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe"
Jun 02 16:06:28.531534 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.531488    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe"} err="failed to get container status \"575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe\": rpc error: code = NotFound desc = could not find container \"575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe\": container with ID starting with 575d86910ad956c348eaed575b2bbc711d5c181209d3712055d8a1191ac998fe not found: ID does not exist"
Jun 02 16:06:28.539994 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.539974    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"]
Jun 02 16:06:28.548589 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.548569    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6170-predictor-76675d7967-6zrmk"]
Jun 02 16:06:28.579896 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.579874    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:06:28.580009 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.579898    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9lvl\" (UniqueName: \"kubernetes.io/projected/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-kube-api-access-b9lvl\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:06:28.580009 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.579935    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-c6170-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eb278ebb-a0df-49cf-9398-b2e740dbc8e8-success-200-isvc-c6170-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:06:28.601111 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:28.601089    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" path="/var/lib/kubelet/pods/eb278ebb-a0df-49cf-9398-b2e740dbc8e8/volumes"
Jun 02 16:06:33.520811 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:33.520782    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:06:33.521400 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:33.521372    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jun 02 16:06:34.547363 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:34.547337    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:06:34.550405 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:34.550385    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:06:43.521952 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:43.521888    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jun 02 16:06:53.521753 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:06:53.521709    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jun 02 16:07:03.521300 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:03.521260    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jun 02 16:07:11.634250 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.634214    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"]
Jun 02 16:07:11.634671 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.634570    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container" containerID="cri-o://f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285" gracePeriod=30
Jun 02 16:07:11.634744 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.634658    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kube-rbac-proxy" containerID="cri-o://f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a" gracePeriod=30
Jun 02 16:07:11.676127 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676100    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"]
Jun 02 16:07:11.676461 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676448    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container"
Jun 02 16:07:11.676522 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676464    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container"
Jun 02 16:07:11.676522 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676485    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kube-rbac-proxy"
Jun 02 16:07:11.676522 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676490    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kube-rbac-proxy"
Jun 02 16:07:11.676661 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676551    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kserve-container"
Jun 02 16:07:11.676661 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.676561    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb278ebb-a0df-49cf-9398-b2e740dbc8e8" containerName="kube-rbac-proxy"
Jun 02 16:07:11.679676 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.679660    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.682257 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.682232    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5da44-predictor-serving-cert\""
Jun 02 16:07:11.682353 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.682268    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-5da44-kube-rbac-proxy-sar-config\""
Jun 02 16:07:11.695943 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.695904    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"]
Jun 02 16:07:11.840404 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.840166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-proxy-tls\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.840404 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.840225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-5da44-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-success-200-isvc-5da44-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.840404 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.840319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdfx2\" (UniqueName: \"kubernetes.io/projected/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-kube-api-access-kdfx2\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.941602 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.941574    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-proxy-tls\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.941775 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.941613    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-5da44-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-success-200-isvc-5da44-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.941775 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.941675    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kdfx2\" (UniqueName: \"kubernetes.io/projected/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-kube-api-access-kdfx2\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.942357 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.942333    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-5da44-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-success-200-isvc-5da44-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.944026 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.944005    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-proxy-tls\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.952110 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.952089    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdfx2\" (UniqueName: \"kubernetes.io/projected/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-kube-api-access-kdfx2\") pod \"success-200-isvc-5da44-predictor-77d9c58457-8lswh\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") " pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:11.991629 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:11.991595    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:12.123902 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.123858    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"]
Jun 02 16:07:12.127598 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:07:12.127572    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb3dab4_e76c_4ec0_aa1c_a6943743cc6d.slice/crio-3b26e666c2e33a86564ce9112ee80667551c86c3cc9a4e336945801dd23d86c7 WatchSource:0}: Error finding container 3b26e666c2e33a86564ce9112ee80667551c86c3cc9a4e336945801dd23d86c7: Status 404 returned error can't find the container with id 3b26e666c2e33a86564ce9112ee80667551c86c3cc9a4e336945801dd23d86c7
Jun 02 16:07:12.663539 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.663424    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" event={"ID":"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d","Type":"ContainerStarted","Data":"97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49"}
Jun 02 16:07:12.663539 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.663465    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" event={"ID":"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d","Type":"ContainerStarted","Data":"a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c"}
Jun 02 16:07:12.663539 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.663480    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" event={"ID":"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d","Type":"ContainerStarted","Data":"3b26e666c2e33a86564ce9112ee80667551c86c3cc9a4e336945801dd23d86c7"}
Jun 02 16:07:12.664034 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.663552    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:12.664957 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.664934    2562 generic.go:358] "Generic (PLEG): container finished" podID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerID="f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a" exitCode=2
Jun 02 16:07:12.665048 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.664999    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" event={"ID":"741ee707-8c7d-4e5e-9bdd-9174d2213122","Type":"ContainerDied","Data":"f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a"}
Jun 02 16:07:12.682137 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:12.682103    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podStartSLOduration=1.682090377 podStartE2EDuration="1.682090377s" podCreationTimestamp="2026-06-02 16:07:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:07:12.681189703 +0000 UTC m=+1238.578910091" watchObservedRunningTime="2026-06-02 16:07:12.682090377 +0000 UTC m=+1238.579810765"
Jun 02 16:07:13.521843 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:13.521810    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:07:13.668278 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:13.668244    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:13.669619 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:13.669595    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jun 02 16:07:14.581818 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.581798    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 16:07:14.672688 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.672660    2562 generic.go:358] "Generic (PLEG): container finished" podID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerID="f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285" exitCode=0
Jun 02 16:07:14.673058 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.672726    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"
Jun 02 16:07:14.673058 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.672746    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" event={"ID":"741ee707-8c7d-4e5e-9bdd-9174d2213122","Type":"ContainerDied","Data":"f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285"}
Jun 02 16:07:14.673058 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.672784    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq" event={"ID":"741ee707-8c7d-4e5e-9bdd-9174d2213122","Type":"ContainerDied","Data":"da3a67cafadcb7ea0b0bb22d9b36583d996ad55ef0edeb390ae36b5916a2d0c5"}
Jun 02 16:07:14.673058 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.672800    2562 scope.go:117] "RemoveContainer" containerID="f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a"
Jun 02 16:07:14.673253 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.673084    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jun 02 16:07:14.683593 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.683574    2562 scope.go:117] "RemoveContainer" containerID="f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285"
Jun 02 16:07:14.690336 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.690316    2562 scope.go:117] "RemoveContainer" containerID="f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a"
Jun 02 16:07:14.690658 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:07:14.690615    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a\": container with ID starting with f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a not found: ID does not exist" containerID="f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a"
Jun 02 16:07:14.690708 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.690673    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a"} err="failed to get container status \"f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a\": rpc error: code = NotFound desc = could not find container \"f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a\": container with ID starting with f189ef650ef7752f911f611899f563982b9e3436899fd2ef5080ab4eefb2cf0a not found: ID does not exist"
Jun 02 16:07:14.690750 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.690705    2562 scope.go:117] "RemoveContainer" containerID="f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285"
Jun 02 16:07:14.691015 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:07:14.690990    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285\": container with ID starting with f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285 not found: ID does not exist" containerID="f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285"
Jun 02 16:07:14.691061 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.691028    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285"} err="failed to get container status \"f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285\": rpc error: code = NotFound desc = could not find container \"f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285\": container with ID starting with f62a27d9eef17007253b9bdb6f6ac7ab438001ba6d7c545c908f775c117f1285 not found: ID does not exist"
Jun 02 16:07:14.766345 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.766322    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/741ee707-8c7d-4e5e-9bdd-9174d2213122-proxy-tls\") pod \"741ee707-8c7d-4e5e-9bdd-9174d2213122\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") "
Jun 02 16:07:14.766448 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.766352    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nldlm\" (UniqueName: \"kubernetes.io/projected/741ee707-8c7d-4e5e-9bdd-9174d2213122-kube-api-access-nldlm\") pod \"741ee707-8c7d-4e5e-9bdd-9174d2213122\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") "
Jun 02 16:07:14.766448 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.766429    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-f26f3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/741ee707-8c7d-4e5e-9bdd-9174d2213122-success-200-isvc-f26f3-kube-rbac-proxy-sar-config\") pod \"741ee707-8c7d-4e5e-9bdd-9174d2213122\" (UID: \"741ee707-8c7d-4e5e-9bdd-9174d2213122\") "
Jun 02 16:07:14.766823 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.766795    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/741ee707-8c7d-4e5e-9bdd-9174d2213122-success-200-isvc-f26f3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-f26f3-kube-rbac-proxy-sar-config") pod "741ee707-8c7d-4e5e-9bdd-9174d2213122" (UID: "741ee707-8c7d-4e5e-9bdd-9174d2213122"). InnerVolumeSpecName "success-200-isvc-f26f3-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:07:14.768312 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.768293    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/741ee707-8c7d-4e5e-9bdd-9174d2213122-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "741ee707-8c7d-4e5e-9bdd-9174d2213122" (UID: "741ee707-8c7d-4e5e-9bdd-9174d2213122"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:07:14.768393 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.768380    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/741ee707-8c7d-4e5e-9bdd-9174d2213122-kube-api-access-nldlm" (OuterVolumeSpecName: "kube-api-access-nldlm") pod "741ee707-8c7d-4e5e-9bdd-9174d2213122" (UID: "741ee707-8c7d-4e5e-9bdd-9174d2213122"). InnerVolumeSpecName "kube-api-access-nldlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:07:14.867977 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.867954    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/741ee707-8c7d-4e5e-9bdd-9174d2213122-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:07:14.867977 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.867975    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nldlm\" (UniqueName: \"kubernetes.io/projected/741ee707-8c7d-4e5e-9bdd-9174d2213122-kube-api-access-nldlm\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:07:14.868139 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.867986    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-f26f3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/741ee707-8c7d-4e5e-9bdd-9174d2213122-success-200-isvc-f26f3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:07:14.994923 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.994882    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"]
Jun 02 16:07:14.997120 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:14.997100    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f26f3-predictor-7cd54b6f9f-lvrzq"]
Jun 02 16:07:16.600748 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:16.600713    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" path="/var/lib/kubelet/pods/741ee707-8c7d-4e5e-9bdd-9174d2213122/volumes"
Jun 02 16:07:19.676814 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:19.676789    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:07:19.677302 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:19.677276    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jun 02 16:07:29.677411 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:29.677368    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jun 02 16:07:35.730503 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.730470    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"]
Jun 02 16:07:35.731007 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.730772    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container" containerID="cri-o://de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37" gracePeriod=30
Jun 02 16:07:35.731007 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.730796    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kube-rbac-proxy" containerID="cri-o://0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906" gracePeriod=30
Jun 02 16:07:35.778810 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.778780    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"]
Jun 02 16:07:35.779170 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.779155    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container"
Jun 02 16:07:35.779170 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.779171    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container"
Jun 02 16:07:35.779170 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.779188    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kube-rbac-proxy"
Jun 02 16:07:35.779170 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.779194    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kube-rbac-proxy"
Jun 02 16:07:35.779408 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.779242    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kube-rbac-proxy"
Jun 02 16:07:35.779408 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.779255    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="741ee707-8c7d-4e5e-9bdd-9174d2213122" containerName="kserve-container"
Jun 02 16:07:35.783505 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.783489    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.785616 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.785599    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-94f66-kube-rbac-proxy-sar-config\""
Jun 02 16:07:35.785701 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.785682    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-94f66-predictor-serving-cert\""
Jun 02 16:07:35.792786 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.792765    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"]
Jun 02 16:07:35.829215 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.829176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/24c19931-bedd-48ce-a058-86f85cf11d23-proxy-tls\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.829363 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.829227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgcbq\" (UniqueName: \"kubernetes.io/projected/24c19931-bedd-48ce-a058-86f85cf11d23-kube-api-access-sgcbq\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.829363 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.829312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-94f66-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/24c19931-bedd-48ce-a058-86f85cf11d23-success-200-isvc-94f66-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.930366 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.930332    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/24c19931-bedd-48ce-a058-86f85cf11d23-proxy-tls\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.930366 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.930371    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgcbq\" (UniqueName: \"kubernetes.io/projected/24c19931-bedd-48ce-a058-86f85cf11d23-kube-api-access-sgcbq\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.930569 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.930420    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-94f66-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/24c19931-bedd-48ce-a058-86f85cf11d23-success-200-isvc-94f66-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.931136 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.931113    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-94f66-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/24c19931-bedd-48ce-a058-86f85cf11d23-success-200-isvc-94f66-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.932840 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.932815    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/24c19931-bedd-48ce-a058-86f85cf11d23-proxy-tls\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:35.940688 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:35.940662    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgcbq\" (UniqueName: \"kubernetes.io/projected/24c19931-bedd-48ce-a058-86f85cf11d23-kube-api-access-sgcbq\") pod \"success-200-isvc-94f66-predictor-56bb84dc9f-x28j5\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") " pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:36.094452 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.094385    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:36.221146 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.221112    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"]
Jun 02 16:07:36.223960 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:07:36.223929    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c19931_bedd_48ce_a058_86f85cf11d23.slice/crio-762a2e66d0efb735218aacb6f036ab2de0258259be031a11ea75faba33f07620 WatchSource:0}: Error finding container 762a2e66d0efb735218aacb6f036ab2de0258259be031a11ea75faba33f07620: Status 404 returned error can't find the container with id 762a2e66d0efb735218aacb6f036ab2de0258259be031a11ea75faba33f07620
Jun 02 16:07:36.747569 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.747534    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" event={"ID":"24c19931-bedd-48ce-a058-86f85cf11d23","Type":"ContainerStarted","Data":"38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1"}
Jun 02 16:07:36.747977 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.747575    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" event={"ID":"24c19931-bedd-48ce-a058-86f85cf11d23","Type":"ContainerStarted","Data":"ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6"}
Jun 02 16:07:36.747977 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.747591    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" event={"ID":"24c19931-bedd-48ce-a058-86f85cf11d23","Type":"ContainerStarted","Data":"762a2e66d0efb735218aacb6f036ab2de0258259be031a11ea75faba33f07620"}
Jun 02 16:07:36.747977 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.747695    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:36.749113 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.749093    2562 generic.go:358] "Generic (PLEG): container finished" podID="566f8d45-09ac-4243-9b55-7e110233b781" containerID="0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906" exitCode=2
Jun 02 16:07:36.749200 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.749145    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" event={"ID":"566f8d45-09ac-4243-9b55-7e110233b781","Type":"ContainerDied","Data":"0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906"}
Jun 02 16:07:36.765242 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:36.765206    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podStartSLOduration=1.76519168 podStartE2EDuration="1.76519168s" podCreationTimestamp="2026-06-02 16:07:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:07:36.764106952 +0000 UTC m=+1262.661827341" watchObservedRunningTime="2026-06-02 16:07:36.76519168 +0000 UTC m=+1262.662912070"
Jun 02 16:07:37.752356 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:37.752331    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:37.753660 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:37.753631    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jun 02 16:07:38.516545 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:38.516496    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Jun 02 16:07:38.757132 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:38.757089    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jun 02 16:07:38.981884 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:38.981858    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:07:39.061165 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.061119    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/566f8d45-09ac-4243-9b55-7e110233b781-proxy-tls\") pod \"566f8d45-09ac-4243-9b55-7e110233b781\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") "
Jun 02 16:07:39.061347 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.061236    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lncpb\" (UniqueName: \"kubernetes.io/projected/566f8d45-09ac-4243-9b55-7e110233b781-kube-api-access-lncpb\") pod \"566f8d45-09ac-4243-9b55-7e110233b781\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") "
Jun 02 16:07:39.061347 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.061273    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-d60e3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/566f8d45-09ac-4243-9b55-7e110233b781-success-200-isvc-d60e3-kube-rbac-proxy-sar-config\") pod \"566f8d45-09ac-4243-9b55-7e110233b781\" (UID: \"566f8d45-09ac-4243-9b55-7e110233b781\") "
Jun 02 16:07:39.061651 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.061616    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/566f8d45-09ac-4243-9b55-7e110233b781-success-200-isvc-d60e3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-d60e3-kube-rbac-proxy-sar-config") pod "566f8d45-09ac-4243-9b55-7e110233b781" (UID: "566f8d45-09ac-4243-9b55-7e110233b781"). InnerVolumeSpecName "success-200-isvc-d60e3-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:07:39.063291 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.063265    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/566f8d45-09ac-4243-9b55-7e110233b781-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "566f8d45-09ac-4243-9b55-7e110233b781" (UID: "566f8d45-09ac-4243-9b55-7e110233b781"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:07:39.063386 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.063292    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/566f8d45-09ac-4243-9b55-7e110233b781-kube-api-access-lncpb" (OuterVolumeSpecName: "kube-api-access-lncpb") pod "566f8d45-09ac-4243-9b55-7e110233b781" (UID: "566f8d45-09ac-4243-9b55-7e110233b781"). InnerVolumeSpecName "kube-api-access-lncpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:07:39.162550 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.162464    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lncpb\" (UniqueName: \"kubernetes.io/projected/566f8d45-09ac-4243-9b55-7e110233b781-kube-api-access-lncpb\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:07:39.162550 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.162502    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-d60e3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/566f8d45-09ac-4243-9b55-7e110233b781-success-200-isvc-d60e3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:07:39.162550 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.162513    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/566f8d45-09ac-4243-9b55-7e110233b781-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:07:39.678001 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.677965    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jun 02 16:07:39.760663 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.760630    2562 generic.go:358] "Generic (PLEG): container finished" podID="566f8d45-09ac-4243-9b55-7e110233b781" containerID="de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37" exitCode=0
Jun 02 16:07:39.761096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.760713    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"
Jun 02 16:07:39.761096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.760722    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" event={"ID":"566f8d45-09ac-4243-9b55-7e110233b781","Type":"ContainerDied","Data":"de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37"}
Jun 02 16:07:39.761096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.760760    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5" event={"ID":"566f8d45-09ac-4243-9b55-7e110233b781","Type":"ContainerDied","Data":"44fb553621b5be68c67a281acd32c7df8546483cbac9e02d7afe307c3233c254"}
Jun 02 16:07:39.761096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.760776    2562 scope.go:117] "RemoveContainer" containerID="0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906"
Jun 02 16:07:39.769323 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.769252    2562 scope.go:117] "RemoveContainer" containerID="de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37"
Jun 02 16:07:39.776485 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.776468    2562 scope.go:117] "RemoveContainer" containerID="0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906"
Jun 02 16:07:39.776725 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:07:39.776706    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906\": container with ID starting with 0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906 not found: ID does not exist" containerID="0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906"
Jun 02 16:07:39.776772 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.776734    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906"} err="failed to get container status \"0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906\": rpc error: code = NotFound desc = could not find container \"0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906\": container with ID starting with 0ab214c01c68e5a1ff0c0b720f4eb3029cf3db7e7c4b2a194eb356e93a2cb906 not found: ID does not exist"
Jun 02 16:07:39.776772 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.776751    2562 scope.go:117] "RemoveContainer" containerID="de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37"
Jun 02 16:07:39.777023 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:07:39.777006    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37\": container with ID starting with de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37 not found: ID does not exist" containerID="de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37"
Jun 02 16:07:39.777085 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.777028    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37"} err="failed to get container status \"de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37\": rpc error: code = NotFound desc = could not find container \"de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37\": container with ID starting with de213c25f5e182bef042918adc8cb98a57222cebfcbcb8729562fa54ebe07d37 not found: ID does not exist"
Jun 02 16:07:39.783882 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.783859    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"]
Jun 02 16:07:39.787640 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:39.787617    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d60e3-predictor-96cb78977-5txz5"]
Jun 02 16:07:40.600927 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:40.600879    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="566f8d45-09ac-4243-9b55-7e110233b781" path="/var/lib/kubelet/pods/566f8d45-09ac-4243-9b55-7e110233b781/volumes"
Jun 02 16:07:43.761791 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:43.761761    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:07:43.762411 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:43.762383    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jun 02 16:07:49.677306 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:49.677268    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jun 02 16:07:53.763100 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:53.763060    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jun 02 16:07:59.677995 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:07:59.677962    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:08:03.763311 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:03.763271    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jun 02 16:08:13.762591 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:13.762553    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jun 02 16:08:21.820540 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.820505    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"]
Jun 02 16:08:21.821021 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.820857    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container" containerID="cri-o://a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c" gracePeriod=30
Jun 02 16:08:21.821021 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.820883    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kube-rbac-proxy" containerID="cri-o://97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49" gracePeriod=30
Jun 02 16:08:21.848598 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.848572    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"]
Jun 02 16:08:21.848988 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.848971    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kube-rbac-proxy"
Jun 02 16:08:21.848988 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.848988    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kube-rbac-proxy"
Jun 02 16:08:21.849103 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.849000    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container"
Jun 02 16:08:21.849103 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.849006    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container"
Jun 02 16:08:21.849103 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.849057    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kserve-container"
Jun 02 16:08:21.849103 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.849067    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="566f8d45-09ac-4243-9b55-7e110233b781" containerName="kube-rbac-proxy"
Jun 02 16:08:21.852083 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.852068    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:21.853898 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.853870    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-411bf-kube-rbac-proxy-sar-config\""
Jun 02 16:08:21.853898 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.853884    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-411bf-predictor-serving-cert\""
Jun 02 16:08:21.861441 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.861419    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"]
Jun 02 16:08:21.912265 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.912234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f9jm\" (UniqueName: \"kubernetes.io/projected/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-kube-api-access-9f9jm\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:21.912265 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.912271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-proxy-tls\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:21.912529 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:21.912304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-411bf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-success-200-isvc-411bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.013266 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.013225    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9f9jm\" (UniqueName: \"kubernetes.io/projected/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-kube-api-access-9f9jm\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.013438 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.013271    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-proxy-tls\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.013438 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.013316    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-411bf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-success-200-isvc-411bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.014013 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.013989    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-411bf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-success-200-isvc-411bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.015631 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.015614    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-proxy-tls\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.027595 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.027555    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f9jm\" (UniqueName: \"kubernetes.io/projected/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-kube-api-access-9f9jm\") pod \"success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") " pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.164098 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.163975    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.294257 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.294217    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"]
Jun 02 16:08:22.296889 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:08:22.296865    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00b08e2_b81a_4a43_9fd5_e4f0462ea058.slice/crio-2a4be52e1dbc6f433ceb273ffde936cdacb71c0978a52f52dbb616a7db57499d WatchSource:0}: Error finding container 2a4be52e1dbc6f433ceb273ffde936cdacb71c0978a52f52dbb616a7db57499d: Status 404 returned error can't find the container with id 2a4be52e1dbc6f433ceb273ffde936cdacb71c0978a52f52dbb616a7db57499d
Jun 02 16:08:22.896940 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.896887    2562 generic.go:358] "Generic (PLEG): container finished" podID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerID="97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49" exitCode=2
Jun 02 16:08:22.897292 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.896970    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" event={"ID":"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d","Type":"ContainerDied","Data":"97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49"}
Jun 02 16:08:22.898382 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.898353    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" event={"ID":"a00b08e2-b81a-4a43-9fd5-e4f0462ea058","Type":"ContainerStarted","Data":"460565725c0200a9612d8a31bb7e227a2d2120836555f007b970a7fafbca89d1"}
Jun 02 16:08:22.898382 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.898380    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" event={"ID":"a00b08e2-b81a-4a43-9fd5-e4f0462ea058","Type":"ContainerStarted","Data":"013575caf6b063cc5496965e886fb09a00569e3d3021283b720f1c6fa5fc8eb2"}
Jun 02 16:08:22.898599 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.898389    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" event={"ID":"a00b08e2-b81a-4a43-9fd5-e4f0462ea058","Type":"ContainerStarted","Data":"2a4be52e1dbc6f433ceb273ffde936cdacb71c0978a52f52dbb616a7db57499d"}
Jun 02 16:08:22.898599 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.898474    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:22.917162 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:22.917104    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podStartSLOduration=1.9170892959999999 podStartE2EDuration="1.917089296s" podCreationTimestamp="2026-06-02 16:08:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:08:22.915408087 +0000 UTC m=+1308.813128475" watchObservedRunningTime="2026-06-02 16:08:22.917089296 +0000 UTC m=+1308.814809684"
Jun 02 16:08:23.763565 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:23.763540    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:08:23.902047 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:23.902018    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:23.903276 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:23.903248    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jun 02 16:08:24.673585 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.673546    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Jun 02 16:08:24.870845 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.870826    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:08:24.906206 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.906173    2562 generic.go:358] "Generic (PLEG): container finished" podID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerID="a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c" exitCode=0
Jun 02 16:08:24.906576 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.906248    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" event={"ID":"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d","Type":"ContainerDied","Data":"a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c"}
Jun 02 16:08:24.906576 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.906266    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"
Jun 02 16:08:24.906576 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.906286    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh" event={"ID":"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d","Type":"ContainerDied","Data":"3b26e666c2e33a86564ce9112ee80667551c86c3cc9a4e336945801dd23d86c7"}
Jun 02 16:08:24.906576 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.906302    2562 scope.go:117] "RemoveContainer" containerID="97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49"
Jun 02 16:08:24.906576 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.906563    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jun 02 16:08:24.914053 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.914036    2562 scope.go:117] "RemoveContainer" containerID="a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c"
Jun 02 16:08:24.921162 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.921146    2562 scope.go:117] "RemoveContainer" containerID="97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49"
Jun 02 16:08:24.921418 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:08:24.921398    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49\": container with ID starting with 97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49 not found: ID does not exist" containerID="97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49"
Jun 02 16:08:24.921467 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.921427    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49"} err="failed to get container status \"97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49\": rpc error: code = NotFound desc = could not find container \"97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49\": container with ID starting with 97828ba504f42534f2b50ec4bb107e748bd000652c0e9af858c8876e3aeb4a49 not found: ID does not exist"
Jun 02 16:08:24.921467 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.921444    2562 scope.go:117] "RemoveContainer" containerID="a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c"
Jun 02 16:08:24.921697 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:08:24.921681    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c\": container with ID starting with a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c not found: ID does not exist" containerID="a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c"
Jun 02 16:08:24.921744 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.921702    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c"} err="failed to get container status \"a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c\": rpc error: code = NotFound desc = could not find container \"a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c\": container with ID starting with a67f80297c5036ca27e68297a07b23f62983f648dc496f38d28360f9c2c31b7c not found: ID does not exist"
Jun 02 16:08:24.941466 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.941444    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-5da44-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-success-200-isvc-5da44-kube-rbac-proxy-sar-config\") pod \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") "
Jun 02 16:08:24.941548 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.941477    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdfx2\" (UniqueName: \"kubernetes.io/projected/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-kube-api-access-kdfx2\") pod \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") "
Jun 02 16:08:24.941592 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.941555    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-proxy-tls\") pod \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\" (UID: \"aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d\") "
Jun 02 16:08:24.941777 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.941753    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-success-200-isvc-5da44-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-5da44-kube-rbac-proxy-sar-config") pod "aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" (UID: "aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d"). InnerVolumeSpecName "success-200-isvc-5da44-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:08:24.943496 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.943474    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" (UID: "aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:08:24.943562 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:24.943507    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-kube-api-access-kdfx2" (OuterVolumeSpecName: "kube-api-access-kdfx2") pod "aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" (UID: "aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d"). InnerVolumeSpecName "kube-api-access-kdfx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:08:25.042812 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:25.042783    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:08:25.042812 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:25.042808    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-5da44-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-success-200-isvc-5da44-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:08:25.042989 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:25.042819    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdfx2\" (UniqueName: \"kubernetes.io/projected/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d-kube-api-access-kdfx2\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:08:25.230342 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:25.230298    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"]
Jun 02 16:08:25.232576 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:25.232554    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-5da44-predictor-77d9c58457-8lswh"]
Jun 02 16:08:26.600272 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:26.600238    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" path="/var/lib/kubelet/pods/aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d/volumes"
Jun 02 16:08:29.910431 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:29.910403    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:08:29.910852 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:29.910824    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jun 02 16:08:39.911299 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:39.911257    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jun 02 16:08:49.911746 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:49.911700    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jun 02 16:08:59.910887 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:08:59.910850    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jun 02 16:09:09.912097 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:09.912069    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:09:28.752716 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.752676    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-55fcd76598-g546m"]
Jun 02 16:09:28.753743 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.753700    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kube-rbac-proxy"
Jun 02 16:09:28.753743 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.753725    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kube-rbac-proxy"
Jun 02 16:09:28.753893 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.753760    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container"
Jun 02 16:09:28.753893 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.753770    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container"
Jun 02 16:09:28.754014 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.753940    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kserve-container"
Jun 02 16:09:28.754014 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.753957    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="aeb3dab4-e76c-4ec0-aa1c-a6943743cc6d" containerName="kube-rbac-proxy"
Jun 02 16:09:28.759862 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.759842    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.768096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.768072    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55fcd76598-g546m"]
Jun 02 16:09:28.777257 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-oauth-config\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.777371 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-trusted-ca-bundle\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.777371 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777285    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-serving-cert\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.777371 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbrx8\" (UniqueName: \"kubernetes.io/projected/2afa62c4-5f1e-4731-a38c-002aa3606ab3-kube-api-access-wbrx8\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.777371 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-service-ca\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.777598 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-config\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.777598 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.777400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-oauth-serving-cert\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.877730 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.877696    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-config\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.877945 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.877741    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-oauth-serving-cert\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.877945 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.877873    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-oauth-config\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878083 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.877996    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-trusted-ca-bundle\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878083 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878024    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-serving-cert\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878083 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878052    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrx8\" (UniqueName: \"kubernetes.io/projected/2afa62c4-5f1e-4731-a38c-002aa3606ab3-kube-api-access-wbrx8\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878236 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878084    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-service-ca\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878567 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878546    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-config\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878670 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878610    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-oauth-serving-cert\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878757 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878736    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-service-ca\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.878852 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.878836    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa62c4-5f1e-4731-a38c-002aa3606ab3-trusted-ca-bundle\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.880392 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.880361    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-serving-cert\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.880500 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.880392    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa62c4-5f1e-4731-a38c-002aa3606ab3-console-oauth-config\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:28.895115 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:28.895094    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrx8\" (UniqueName: \"kubernetes.io/projected/2afa62c4-5f1e-4731-a38c-002aa3606ab3-kube-api-access-wbrx8\") pod \"console-55fcd76598-g546m\" (UID: \"2afa62c4-5f1e-4731-a38c-002aa3606ab3\") " pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:29.069809 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:29.069717    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:29.197390 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:29.197358    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55fcd76598-g546m"]
Jun 02 16:09:29.200280 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:09:29.200256    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2afa62c4_5f1e_4731_a38c_002aa3606ab3.slice/crio-45c28aaf449fb37b8e980957f64145688760a895d8f31b688565373d23f9072a WatchSource:0}: Error finding container 45c28aaf449fb37b8e980957f64145688760a895d8f31b688565373d23f9072a: Status 404 returned error can't find the container with id 45c28aaf449fb37b8e980957f64145688760a895d8f31b688565373d23f9072a
Jun 02 16:09:30.113241 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:30.113208    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fcd76598-g546m" event={"ID":"2afa62c4-5f1e-4731-a38c-002aa3606ab3","Type":"ContainerStarted","Data":"5e8bbe873336c93fa3b0b4dac058dc063c29cf046611e11d3ea2eb4b89849de0"}
Jun 02 16:09:30.113241 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:30.113243    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fcd76598-g546m" event={"ID":"2afa62c4-5f1e-4731-a38c-002aa3606ab3","Type":"ContainerStarted","Data":"45c28aaf449fb37b8e980957f64145688760a895d8f31b688565373d23f9072a"}
Jun 02 16:09:30.131070 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:30.131030    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-55fcd76598-g546m" podStartSLOduration=2.131016238 podStartE2EDuration="2.131016238s" podCreationTimestamp="2026-06-02 16:09:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:09:30.130179688 +0000 UTC m=+1376.027900079" watchObservedRunningTime="2026-06-02 16:09:30.131016238 +0000 UTC m=+1376.028736627"
Jun 02 16:09:39.070341 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:39.070298    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:39.070341 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:39.070350    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:39.074989 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:39.074961    2562 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:39.144874 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:39.144844    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55fcd76598-g546m"
Jun 02 16:09:39.194648 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:09:39.194615    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-665f8df445-n5mdj"]
Jun 02 16:10:04.220777 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.220711    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-665f8df445-n5mdj" podUID="279064f6-c588-4a08-ba85-4ce349a2c8d9" containerName="console" containerID="cri-o://35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6" gracePeriod=15
Jun 02 16:10:04.467858 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.467837    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-665f8df445-n5mdj_279064f6-c588-4a08-ba85-4ce349a2c8d9/console/0.log"
Jun 02 16:10:04.468010 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.467896    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 16:10:04.574823 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574755    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-serving-cert\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.574823 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574795    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-config\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.574823 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574817    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-oauth-serving-cert\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.575096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574844    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-oauth-config\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.575096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574861    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-service-ca\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.575096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574904    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6j46\" (UniqueName: \"kubernetes.io/projected/279064f6-c588-4a08-ba85-4ce349a2c8d9-kube-api-access-l6j46\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.575096 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.574954    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-trusted-ca-bundle\") pod \"279064f6-c588-4a08-ba85-4ce349a2c8d9\" (UID: \"279064f6-c588-4a08-ba85-4ce349a2c8d9\") "
Jun 02 16:10:04.575313 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.575275    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-config" (OuterVolumeSpecName: "console-config") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:10:04.575376 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.575306    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:10:04.575431 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.575396    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:10:04.575485 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.575456    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:10:04.576872 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.576846    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:10:04.577011 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.576993    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:10:04.577059 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.576997    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/279064f6-c588-4a08-ba85-4ce349a2c8d9-kube-api-access-l6j46" (OuterVolumeSpecName: "kube-api-access-l6j46") pod "279064f6-c588-4a08-ba85-4ce349a2c8d9" (UID: "279064f6-c588-4a08-ba85-4ce349a2c8d9"). InnerVolumeSpecName "kube-api-access-l6j46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:10:04.676316 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676283    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:04.676316 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676312    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:04.676475 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676326    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-oauth-serving-cert\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:04.676475 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676338    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/279064f6-c588-4a08-ba85-4ce349a2c8d9-console-oauth-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:04.676475 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676351    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-service-ca\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:04.676475 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676364    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6j46\" (UniqueName: \"kubernetes.io/projected/279064f6-c588-4a08-ba85-4ce349a2c8d9-kube-api-access-l6j46\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:04.676475 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:04.676376    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/279064f6-c588-4a08-ba85-4ce349a2c8d9-trusted-ca-bundle\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:10:05.234702 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.234674    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-665f8df445-n5mdj_279064f6-c588-4a08-ba85-4ce349a2c8d9/console/0.log"
Jun 02 16:10:05.235167 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.234714    2562 generic.go:358] "Generic (PLEG): container finished" podID="279064f6-c588-4a08-ba85-4ce349a2c8d9" containerID="35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6" exitCode=2
Jun 02 16:10:05.235167 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.234746    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665f8df445-n5mdj" event={"ID":"279064f6-c588-4a08-ba85-4ce349a2c8d9","Type":"ContainerDied","Data":"35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6"}
Jun 02 16:10:05.235167 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.234784    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665f8df445-n5mdj" event={"ID":"279064f6-c588-4a08-ba85-4ce349a2c8d9","Type":"ContainerDied","Data":"4232d2427e88e4bb837359e61395a6eab3ff2459e0700950e5077a85e35503a0"}
Jun 02 16:10:05.235167 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.234800    2562 scope.go:117] "RemoveContainer" containerID="35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6"
Jun 02 16:10:05.235167 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.234813    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665f8df445-n5mdj"
Jun 02 16:10:05.242837 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.242822    2562 scope.go:117] "RemoveContainer" containerID="35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6"
Jun 02 16:10:05.243081 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:10:05.243059    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6\": container with ID starting with 35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6 not found: ID does not exist" containerID="35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6"
Jun 02 16:10:05.243146 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.243089    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6"} err="failed to get container status \"35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6\": rpc error: code = NotFound desc = could not find container \"35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6\": container with ID starting with 35c701254dc9bac672def7e1392413e48736684fb7982a927404d21cde8acef6 not found: ID does not exist"
Jun 02 16:10:05.252461 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.252434    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-665f8df445-n5mdj"]
Jun 02 16:10:05.257497 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:05.257479    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-665f8df445-n5mdj"]
Jun 02 16:10:06.601080 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:10:06.601049    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="279064f6-c588-4a08-ba85-4ce349a2c8d9" path="/var/lib/kubelet/pods/279064f6-c588-4a08-ba85-4ce349a2c8d9/volumes"
Jun 02 16:11:34.571280 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:11:34.571254    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:11:34.575467 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:11:34.575445    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:16:34.595895 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:34.595853    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:16:34.599293 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:34.599263    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:16:50.675160 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.675118    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"]
Jun 02 16:16:50.676142 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.676105    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" containerID="cri-o://ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6" gracePeriod=30
Jun 02 16:16:50.676548 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.676492    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kube-rbac-proxy" containerID="cri-o://38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1" gracePeriod=30
Jun 02 16:16:50.741657 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.741618    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"]
Jun 02 16:16:50.742037 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.742023    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="279064f6-c588-4a08-ba85-4ce349a2c8d9" containerName="console"
Jun 02 16:16:50.742037 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.742038    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="279064f6-c588-4a08-ba85-4ce349a2c8d9" containerName="console"
Jun 02 16:16:50.742162 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.742090    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="279064f6-c588-4a08-ba85-4ce349a2c8d9" containerName="console"
Jun 02 16:16:50.745217 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.745202    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.750142 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.749874    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-022f7-predictor-serving-cert\""
Jun 02 16:16:50.750142 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.749890    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-022f7-kube-rbac-proxy-sar-config\""
Jun 02 16:16:50.771990 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.771963    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"]
Jun 02 16:16:50.805721 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.805687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-022f7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-success-200-isvc-022f7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.805866 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.805762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.805866 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.805799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4jt\" (UniqueName: \"kubernetes.io/projected/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-kube-api-access-4n4jt\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.906989 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.906960    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-022f7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-success-200-isvc-022f7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.907177 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.907003    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.907177 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.907030    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4jt\" (UniqueName: \"kubernetes.io/projected/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-kube-api-access-4n4jt\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.907177 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:16:50.907138    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-022f7-predictor-serving-cert: secret "success-200-isvc-022f7-predictor-serving-cert" not found
Jun 02 16:16:50.907317 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:16:50.907242    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls podName:324a24e6-38b9-4b9a-ab7d-6c8daa16e31a nodeName:}" failed. No retries permitted until 2026-06-02 16:16:51.407218161 +0000 UTC m=+1817.304938528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls") pod "success-200-isvc-022f7-predictor-744f9d8855-rcmvf" (UID: "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a") : secret "success-200-isvc-022f7-predictor-serving-cert" not found
Jun 02 16:16:50.907635 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.907617    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-022f7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-success-200-isvc-022f7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:50.915978 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:50.915953    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4jt\" (UniqueName: \"kubernetes.io/projected/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-kube-api-access-4n4jt\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:51.411786 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.411742    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:51.414105 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.414086    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls\") pod \"success-200-isvc-022f7-predictor-744f9d8855-rcmvf\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") " pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:51.544467 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.544434    2562 generic.go:358] "Generic (PLEG): container finished" podID="24c19931-bedd-48ce-a058-86f85cf11d23" containerID="38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1" exitCode=2
Jun 02 16:16:51.544616 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.544495    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" event={"ID":"24c19931-bedd-48ce-a058-86f85cf11d23","Type":"ContainerDied","Data":"38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1"}
Jun 02 16:16:51.659932 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.659886    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:51.787979 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.787953    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"]
Jun 02 16:16:51.790512 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:16:51.790486    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324a24e6_38b9_4b9a_ab7d_6c8daa16e31a.slice/crio-aa9b35c120ed6196ce03870f4ecc81faf0593b68c75a37f42afc16ba821c06fe WatchSource:0}: Error finding container aa9b35c120ed6196ce03870f4ecc81faf0593b68c75a37f42afc16ba821c06fe: Status 404 returned error can't find the container with id aa9b35c120ed6196ce03870f4ecc81faf0593b68c75a37f42afc16ba821c06fe
Jun 02 16:16:51.792343 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:51.792322    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 16:16:52.549102 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:52.549068    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" event={"ID":"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a","Type":"ContainerStarted","Data":"2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f"}
Jun 02 16:16:52.549102 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:52.549106    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" event={"ID":"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a","Type":"ContainerStarted","Data":"48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7"}
Jun 02 16:16:52.549318 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:52.549120    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" event={"ID":"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a","Type":"ContainerStarted","Data":"aa9b35c120ed6196ce03870f4ecc81faf0593b68c75a37f42afc16ba821c06fe"}
Jun 02 16:16:52.549318 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:52.549201    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:52.568969 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:52.568906    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podStartSLOduration=2.568894196 podStartE2EDuration="2.568894196s" podCreationTimestamp="2026-06-02 16:16:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:16:52.56749791 +0000 UTC m=+1818.465218299" watchObservedRunningTime="2026-06-02 16:16:52.568894196 +0000 UTC m=+1818.466614583"
Jun 02 16:16:53.552688 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.552654    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:53.554087 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.554058    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jun 02 16:16:53.832583 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.832557    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:16:53.935130 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.935103    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgcbq\" (UniqueName: \"kubernetes.io/projected/24c19931-bedd-48ce-a058-86f85cf11d23-kube-api-access-sgcbq\") pod \"24c19931-bedd-48ce-a058-86f85cf11d23\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") "
Jun 02 16:16:53.935313 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.935159    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-94f66-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/24c19931-bedd-48ce-a058-86f85cf11d23-success-200-isvc-94f66-kube-rbac-proxy-sar-config\") pod \"24c19931-bedd-48ce-a058-86f85cf11d23\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") "
Jun 02 16:16:53.935313 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.935224    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/24c19931-bedd-48ce-a058-86f85cf11d23-proxy-tls\") pod \"24c19931-bedd-48ce-a058-86f85cf11d23\" (UID: \"24c19931-bedd-48ce-a058-86f85cf11d23\") "
Jun 02 16:16:53.935581 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.935553    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c19931-bedd-48ce-a058-86f85cf11d23-success-200-isvc-94f66-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-94f66-kube-rbac-proxy-sar-config") pod "24c19931-bedd-48ce-a058-86f85cf11d23" (UID: "24c19931-bedd-48ce-a058-86f85cf11d23"). InnerVolumeSpecName "success-200-isvc-94f66-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:16:53.937261 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.937240    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c19931-bedd-48ce-a058-86f85cf11d23-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "24c19931-bedd-48ce-a058-86f85cf11d23" (UID: "24c19931-bedd-48ce-a058-86f85cf11d23"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:16:53.937327 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:53.937242    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c19931-bedd-48ce-a058-86f85cf11d23-kube-api-access-sgcbq" (OuterVolumeSpecName: "kube-api-access-sgcbq") pod "24c19931-bedd-48ce-a058-86f85cf11d23" (UID: "24c19931-bedd-48ce-a058-86f85cf11d23"). InnerVolumeSpecName "kube-api-access-sgcbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:16:54.036646 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.036607    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgcbq\" (UniqueName: \"kubernetes.io/projected/24c19931-bedd-48ce-a058-86f85cf11d23-kube-api-access-sgcbq\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:16:54.036646 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.036640    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-94f66-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/24c19931-bedd-48ce-a058-86f85cf11d23-success-200-isvc-94f66-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:16:54.036646 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.036652    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/24c19931-bedd-48ce-a058-86f85cf11d23-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:16:54.557570 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.557538    2562 generic.go:358] "Generic (PLEG): container finished" podID="24c19931-bedd-48ce-a058-86f85cf11d23" containerID="ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6" exitCode=0
Jun 02 16:16:54.557969 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.557606    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"
Jun 02 16:16:54.557969 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.557620    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" event={"ID":"24c19931-bedd-48ce-a058-86f85cf11d23","Type":"ContainerDied","Data":"ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6"}
Jun 02 16:16:54.557969 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.557653    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" event={"ID":"24c19931-bedd-48ce-a058-86f85cf11d23","Type":"ContainerDied","Data":"762a2e66d0efb735218aacb6f036ab2de0258259be031a11ea75faba33f07620"}
Jun 02 16:16:54.557969 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.557666    2562 scope.go:117] "RemoveContainer" containerID="38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1"
Jun 02 16:16:54.558147 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.558110    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jun 02 16:16:54.566102 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.566060    2562 scope.go:117] "RemoveContainer" containerID="ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6"
Jun 02 16:16:54.572986 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.572968    2562 scope.go:117] "RemoveContainer" containerID="38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1"
Jun 02 16:16:54.573236 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:16:54.573220    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1\": container with ID starting with 38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1 not found: ID does not exist" containerID="38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1"
Jun 02 16:16:54.573284 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.573243    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1"} err="failed to get container status \"38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1\": rpc error: code = NotFound desc = could not find container \"38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1\": container with ID starting with 38be3ea3b4aff13e3f3533df810037c4a415a754dd64890d5ef2dda8df5c2da1 not found: ID does not exist"
Jun 02 16:16:54.573284 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.573258    2562 scope.go:117] "RemoveContainer" containerID="ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6"
Jun 02 16:16:54.573458 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:16:54.573443    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6\": container with ID starting with ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6 not found: ID does not exist" containerID="ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6"
Jun 02 16:16:54.573493 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.573462    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6"} err="failed to get container status \"ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6\": rpc error: code = NotFound desc = could not find container \"ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6\": container with ID starting with ab77191a4466787e3e70809ca6a31a393c58fe4aaae0ce37439e07f09231f2f6 not found: ID does not exist"
Jun 02 16:16:54.579768 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.579744    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"]
Jun 02 16:16:54.584155 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.584135    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5"]
Jun 02 16:16:54.601305 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.601273    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" path="/var/lib/kubelet/pods/24c19931-bedd-48ce-a058-86f85cf11d23/volumes"
Jun 02 16:16:54.757444 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.757404    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": context deadline exceeded"
Jun 02 16:16:54.762649 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:54.762624    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-94f66-predictor-56bb84dc9f-x28j5" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: i/o timeout"
Jun 02 16:16:59.562477 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:59.562444    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:16:59.563070 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:16:59.563037    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jun 02 16:17:09.563531 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:09.563488    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jun 02 16:17:19.562992 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:19.562950    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jun 02 16:17:29.563792 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:29.563752    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jun 02 16:17:36.722263 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.722079    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"]
Jun 02 16:17:36.722709 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.722464    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container" containerID="cri-o://013575caf6b063cc5496965e886fb09a00569e3d3021283b720f1c6fa5fc8eb2" gracePeriod=30
Jun 02 16:17:36.722837 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.722809    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kube-rbac-proxy" containerID="cri-o://460565725c0200a9612d8a31bb7e227a2d2120836555f007b970a7fafbca89d1" gracePeriod=30
Jun 02 16:17:36.769192 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769166    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"]
Jun 02 16:17:36.769493 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769480    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kube-rbac-proxy"
Jun 02 16:17:36.769535 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769495    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kube-rbac-proxy"
Jun 02 16:17:36.769535 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769504    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container"
Jun 02 16:17:36.769535 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769509    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container"
Jun 02 16:17:36.769627 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769571    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kserve-container"
Jun 02 16:17:36.769627 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.769580    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="24c19931-bedd-48ce-a058-86f85cf11d23" containerName="kube-rbac-proxy"
Jun 02 16:17:36.773701 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.773677    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.775393 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.775369    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-78d4f-predictor-serving-cert\""
Jun 02 16:17:36.775496 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.775389    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-78d4f-kube-rbac-proxy-sar-config\""
Jun 02 16:17:36.783133 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.783110    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"]
Jun 02 16:17:36.795560 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.795535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-78d4f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/170be138-1cf6-4494-953f-c87f63016e9a-success-200-isvc-78d4f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.795666 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.795594    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m55vj\" (UniqueName: \"kubernetes.io/projected/170be138-1cf6-4494-953f-c87f63016e9a-kube-api-access-m55vj\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.795731 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.795683    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/170be138-1cf6-4494-953f-c87f63016e9a-proxy-tls\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.896459 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.896428    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-78d4f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/170be138-1cf6-4494-953f-c87f63016e9a-success-200-isvc-78d4f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.896636 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.896487    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m55vj\" (UniqueName: \"kubernetes.io/projected/170be138-1cf6-4494-953f-c87f63016e9a-kube-api-access-m55vj\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.896636 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.896544    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/170be138-1cf6-4494-953f-c87f63016e9a-proxy-tls\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.897145 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.897123    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-78d4f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/170be138-1cf6-4494-953f-c87f63016e9a-success-200-isvc-78d4f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.899203 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.899164    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/170be138-1cf6-4494-953f-c87f63016e9a-proxy-tls\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:36.905902 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:36.905878    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m55vj\" (UniqueName: \"kubernetes.io/projected/170be138-1cf6-4494-953f-c87f63016e9a-kube-api-access-m55vj\") pod \"success-200-isvc-78d4f-predictor-599d66c69f-fkpzj\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") " pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:37.085775 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.085689    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:37.210505 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.210478    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"]
Jun 02 16:17:37.213999 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:17:37.213969    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170be138_1cf6_4494_953f_c87f63016e9a.slice/crio-bf28857ee4cb25cc5e55c42d59c096fb8aba7a842931bc0b8ab1ba086d1bb707 WatchSource:0}: Error finding container bf28857ee4cb25cc5e55c42d59c096fb8aba7a842931bc0b8ab1ba086d1bb707: Status 404 returned error can't find the container with id bf28857ee4cb25cc5e55c42d59c096fb8aba7a842931bc0b8ab1ba086d1bb707
Jun 02 16:17:37.691021 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.690990    2562 generic.go:358] "Generic (PLEG): container finished" podID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerID="460565725c0200a9612d8a31bb7e227a2d2120836555f007b970a7fafbca89d1" exitCode=2
Jun 02 16:17:37.691194 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.691061    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" event={"ID":"a00b08e2-b81a-4a43-9fd5-e4f0462ea058","Type":"ContainerDied","Data":"460565725c0200a9612d8a31bb7e227a2d2120836555f007b970a7fafbca89d1"}
Jun 02 16:17:37.692500 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.692480    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" event={"ID":"170be138-1cf6-4494-953f-c87f63016e9a","Type":"ContainerStarted","Data":"01d922d5141f7d3a221226aefcd8cfacc307877af14f9a2cac7c8cbe17735284"}
Jun 02 16:17:37.692584 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.692504    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" event={"ID":"170be138-1cf6-4494-953f-c87f63016e9a","Type":"ContainerStarted","Data":"f80affa01b5b91fe9b20c944238e996b7b19239342ad98a442b4a234b628f022"}
Jun 02 16:17:37.692584 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.692515    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" event={"ID":"170be138-1cf6-4494-953f-c87f63016e9a","Type":"ContainerStarted","Data":"bf28857ee4cb25cc5e55c42d59c096fb8aba7a842931bc0b8ab1ba086d1bb707"}
Jun 02 16:17:37.692659 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.692639    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:37.712261 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:37.712219    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podStartSLOduration=1.712207869 podStartE2EDuration="1.712207869s" podCreationTimestamp="2026-06-02 16:17:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:17:37.709962591 +0000 UTC m=+1863.607682979" watchObservedRunningTime="2026-06-02 16:17:37.712207869 +0000 UTC m=+1863.609928254"
Jun 02 16:17:38.696352 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:38.696316    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:38.697431 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:38.697404    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jun 02 16:17:39.564074 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.564042    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:17:39.701533 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.701496    2562 generic.go:358] "Generic (PLEG): container finished" podID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerID="013575caf6b063cc5496965e886fb09a00569e3d3021283b720f1c6fa5fc8eb2" exitCode=0
Jun 02 16:17:39.701996 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.701572    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" event={"ID":"a00b08e2-b81a-4a43-9fd5-e4f0462ea058","Type":"ContainerDied","Data":"013575caf6b063cc5496965e886fb09a00569e3d3021283b720f1c6fa5fc8eb2"}
Jun 02 16:17:39.701996 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.701931    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jun 02 16:17:39.775630 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.775608    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:17:39.822230 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.822152    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-411bf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-success-200-isvc-411bf-kube-rbac-proxy-sar-config\") pod \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") "
Jun 02 16:17:39.822371 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.822231    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-proxy-tls\") pod \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") "
Jun 02 16:17:39.822371 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.822325    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f9jm\" (UniqueName: \"kubernetes.io/projected/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-kube-api-access-9f9jm\") pod \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\" (UID: \"a00b08e2-b81a-4a43-9fd5-e4f0462ea058\") "
Jun 02 16:17:39.822514 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.822485    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-success-200-isvc-411bf-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-411bf-kube-rbac-proxy-sar-config") pod "a00b08e2-b81a-4a43-9fd5-e4f0462ea058" (UID: "a00b08e2-b81a-4a43-9fd5-e4f0462ea058"). InnerVolumeSpecName "success-200-isvc-411bf-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:17:39.822675 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.822656    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-411bf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-success-200-isvc-411bf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:17:39.824362 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.824335    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a00b08e2-b81a-4a43-9fd5-e4f0462ea058" (UID: "a00b08e2-b81a-4a43-9fd5-e4f0462ea058"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:17:39.824362 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.824346    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-kube-api-access-9f9jm" (OuterVolumeSpecName: "kube-api-access-9f9jm") pod "a00b08e2-b81a-4a43-9fd5-e4f0462ea058" (UID: "a00b08e2-b81a-4a43-9fd5-e4f0462ea058"). InnerVolumeSpecName "kube-api-access-9f9jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:17:39.923599 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.923564    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f9jm\" (UniqueName: \"kubernetes.io/projected/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-kube-api-access-9f9jm\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:17:39.923599 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:39.923592    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a00b08e2-b81a-4a43-9fd5-e4f0462ea058-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:17:40.706243 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:40.706217    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"
Jun 02 16:17:40.706243 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:40.706234    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw" event={"ID":"a00b08e2-b81a-4a43-9fd5-e4f0462ea058","Type":"ContainerDied","Data":"2a4be52e1dbc6f433ceb273ffde936cdacb71c0978a52f52dbb616a7db57499d"}
Jun 02 16:17:40.706709 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:40.706280    2562 scope.go:117] "RemoveContainer" containerID="460565725c0200a9612d8a31bb7e227a2d2120836555f007b970a7fafbca89d1"
Jun 02 16:17:40.714605 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:40.714587    2562 scope.go:117] "RemoveContainer" containerID="013575caf6b063cc5496965e886fb09a00569e3d3021283b720f1c6fa5fc8eb2"
Jun 02 16:17:40.725405 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:40.725379    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"]
Jun 02 16:17:40.730317 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:40.730287    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-411bf-predictor-85c4db6bcc-rm7lw"]
Jun 02 16:17:42.601288 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:42.601251    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" path="/var/lib/kubelet/pods/a00b08e2-b81a-4a43-9fd5-e4f0462ea058/volumes"
Jun 02 16:17:44.707039 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:44.707007    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:17:44.707556 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:44.707532    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jun 02 16:17:54.708126 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:17:54.708085    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jun 02 16:18:01.038587 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.038555    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"]
Jun 02 16:18:01.039009 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.038840    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container" containerID="cri-o://48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7" gracePeriod=30
Jun 02 16:18:01.039009 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.038879    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kube-rbac-proxy" containerID="cri-o://2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f" gracePeriod=30
Jun 02 16:18:01.051926 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.051890    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"]
Jun 02 16:18:01.052244 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.052231    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container"
Jun 02 16:18:01.052302 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.052245    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container"
Jun 02 16:18:01.052302 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.052261    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kube-rbac-proxy"
Jun 02 16:18:01.052302 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.052268    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kube-rbac-proxy"
Jun 02 16:18:01.052397 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.052323    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kserve-container"
Jun 02 16:18:01.052397 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.052331    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="a00b08e2-b81a-4a43-9fd5-e4f0462ea058" containerName="kube-rbac-proxy"
Jun 02 16:18:01.055534 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.055515    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.057450 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.057425    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-cb302-predictor-serving-cert\""
Jun 02 16:18:01.057549 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.057487    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-cb302-kube-rbac-proxy-sar-config\""
Jun 02 16:18:01.064239 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.064216    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"]
Jun 02 16:18:01.215086 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.215053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.215285 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.215094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7249g\" (UniqueName: \"kubernetes.io/projected/9dec0225-d6eb-4027-8086-2e17a4652344-kube-api-access-7249g\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.215285 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.215133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"success-200-isvc-cb302-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9dec0225-d6eb-4027-8086-2e17a4652344-success-200-isvc-cb302-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.315954 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.315844    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.315954 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.315901    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7249g\" (UniqueName: \"kubernetes.io/projected/9dec0225-d6eb-4027-8086-2e17a4652344-kube-api-access-7249g\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.316187 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.315980    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"success-200-isvc-cb302-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9dec0225-d6eb-4027-8086-2e17a4652344-success-200-isvc-cb302-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.316187 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:18:01.316017    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-cb302-predictor-serving-cert: secret "success-200-isvc-cb302-predictor-serving-cert" not found
Jun 02 16:18:01.316187 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:18:01.316088    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls podName:9dec0225-d6eb-4027-8086-2e17a4652344 nodeName:}" failed. No retries permitted until 2026-06-02 16:18:01.816067415 +0000 UTC m=+1887.713787781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls") pod "success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" (UID: "9dec0225-d6eb-4027-8086-2e17a4652344") : secret "success-200-isvc-cb302-predictor-serving-cert" not found
Jun 02 16:18:01.316770 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.316743    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"success-200-isvc-cb302-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9dec0225-d6eb-4027-8086-2e17a4652344-success-200-isvc-cb302-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.324455 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.324425    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7249g\" (UniqueName: \"kubernetes.io/projected/9dec0225-d6eb-4027-8086-2e17a4652344-kube-api-access-7249g\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.782194 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.782154    2562 generic.go:358] "Generic (PLEG): container finished" podID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerID="2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f" exitCode=2
Jun 02 16:18:01.782372 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.782240    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" event={"ID":"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a","Type":"ContainerDied","Data":"2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f"}
Jun 02 16:18:01.820821 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.820782    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.823375 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.823347    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls\") pod \"success-200-isvc-cb302-predictor-9dcf7c765-pmb6t\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") " pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:01.968301 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:01.968264    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:02.102000 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:02.101967    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"]
Jun 02 16:18:02.105344 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:18:02.105318    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dec0225_d6eb_4027_8086_2e17a4652344.slice/crio-f76f9cd3e83015f18b50de5aaf98b3f74a17c98c1a0493cf37255013ad34cc84 WatchSource:0}: Error finding container f76f9cd3e83015f18b50de5aaf98b3f74a17c98c1a0493cf37255013ad34cc84: Status 404 returned error can't find the container with id f76f9cd3e83015f18b50de5aaf98b3f74a17c98c1a0493cf37255013ad34cc84
Jun 02 16:18:02.787350 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:02.787310    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" event={"ID":"9dec0225-d6eb-4027-8086-2e17a4652344","Type":"ContainerStarted","Data":"faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d"}
Jun 02 16:18:02.787350 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:02.787350    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" event={"ID":"9dec0225-d6eb-4027-8086-2e17a4652344","Type":"ContainerStarted","Data":"f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4"}
Jun 02 16:18:02.787350 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:02.787360    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" event={"ID":"9dec0225-d6eb-4027-8086-2e17a4652344","Type":"ContainerStarted","Data":"f76f9cd3e83015f18b50de5aaf98b3f74a17c98c1a0493cf37255013ad34cc84"}
Jun 02 16:18:02.787612 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:02.787524    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:02.804942 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:02.804874    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podStartSLOduration=1.8048579249999999 podStartE2EDuration="1.804857925s" podCreationTimestamp="2026-06-02 16:18:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:18:02.803950029 +0000 UTC m=+1888.701670419" watchObservedRunningTime="2026-06-02 16:18:02.804857925 +0000 UTC m=+1888.702578313"
Jun 02 16:18:03.790663 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:03.790627    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:03.792041 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:03.792016    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jun 02 16:18:04.382534 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.382513    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:18:04.546473 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.546341    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n4jt\" (UniqueName: \"kubernetes.io/projected/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-kube-api-access-4n4jt\") pod \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") "
Jun 02 16:18:04.546473 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.546442    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls\") pod \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") "
Jun 02 16:18:04.546709 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.546554    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-022f7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-success-200-isvc-022f7-kube-rbac-proxy-sar-config\") pod \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\" (UID: \"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a\") "
Jun 02 16:18:04.546875 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.546852    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-success-200-isvc-022f7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-022f7-kube-rbac-proxy-sar-config") pod "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" (UID: "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a"). InnerVolumeSpecName "success-200-isvc-022f7-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:18:04.548571 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.548537    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-kube-api-access-4n4jt" (OuterVolumeSpecName: "kube-api-access-4n4jt") pod "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" (UID: "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a"). InnerVolumeSpecName "kube-api-access-4n4jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:18:04.548571 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.548539    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" (UID: "324a24e6-38b9-4b9a-ab7d-6c8daa16e31a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:18:04.647805 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.647769    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n4jt\" (UniqueName: \"kubernetes.io/projected/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-kube-api-access-4n4jt\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:18:04.647805 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.647803    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:18:04.648054 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.647816    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-022f7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a-success-200-isvc-022f7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:18:04.708437 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.708401    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jun 02 16:18:04.794992 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.794953    2562 generic.go:358] "Generic (PLEG): container finished" podID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerID="48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7" exitCode=0
Jun 02 16:18:04.795441 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.795033    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"
Jun 02 16:18:04.795441 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.795035    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" event={"ID":"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a","Type":"ContainerDied","Data":"48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7"}
Jun 02 16:18:04.795441 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.795072    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf" event={"ID":"324a24e6-38b9-4b9a-ab7d-6c8daa16e31a","Type":"ContainerDied","Data":"aa9b35c120ed6196ce03870f4ecc81faf0593b68c75a37f42afc16ba821c06fe"}
Jun 02 16:18:04.795441 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.795091    2562 scope.go:117] "RemoveContainer" containerID="2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f"
Jun 02 16:18:04.795698 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.795669    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jun 02 16:18:04.803155 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.803139    2562 scope.go:117] "RemoveContainer" containerID="48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7"
Jun 02 16:18:04.810534 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.810500    2562 scope.go:117] "RemoveContainer" containerID="2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f"
Jun 02 16:18:04.810856 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:18:04.810834    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f\": container with ID starting with 2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f not found: ID does not exist" containerID="2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f"
Jun 02 16:18:04.810976 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.810875    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f"} err="failed to get container status \"2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f\": rpc error: code = NotFound desc = could not find container \"2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f\": container with ID starting with 2f22c2956b132cfb6eabd7940d5887e6a06d8796db52cf0e35e6c48b95e19f4f not found: ID does not exist"
Jun 02 16:18:04.810976 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.810893    2562 scope.go:117] "RemoveContainer" containerID="48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7"
Jun 02 16:18:04.811268 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:18:04.811248    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7\": container with ID starting with 48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7 not found: ID does not exist" containerID="48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7"
Jun 02 16:18:04.811330 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.811276    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7"} err="failed to get container status \"48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7\": rpc error: code = NotFound desc = could not find container \"48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7\": container with ID starting with 48085c64b879f65c6832419156ac8fbd0cbecca113d58215f6cd5f62477a3ad7 not found: ID does not exist"
Jun 02 16:18:04.811376 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.811349    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"]
Jun 02 16:18:04.815514 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:04.815491    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-022f7-predictor-744f9d8855-rcmvf"]
Jun 02 16:18:06.601208 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:06.601174    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" path="/var/lib/kubelet/pods/324a24e6-38b9-4b9a-ab7d-6c8daa16e31a/volumes"
Jun 02 16:18:09.800197 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:09.800167    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:18:09.800644 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:09.800617    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jun 02 16:18:14.707868 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:14.707823    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jun 02 16:18:19.800671 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:19.800625    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jun 02 16:18:24.708633 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:24.708601    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:18:29.800722 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:29.800684    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jun 02 16:18:39.801002 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:39.800957    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jun 02 16:18:49.801823 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:18:49.801796    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:21:34.618684 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:21:34.618654    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:21:34.625036 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:21:34.625016    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:26:34.640952 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:26:34.640817    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:26:34.646244 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:26:34.646225    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:27:15.913764 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:15.913687    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"]
Jun 02 16:27:15.914338 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:15.914075    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container" containerID="cri-o://f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4" gracePeriod=30
Jun 02 16:27:15.914338 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:15.914113    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kube-rbac-proxy" containerID="cri-o://faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d" gracePeriod=30
Jun 02 16:27:16.539672 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:16.539637    2562 generic.go:358] "Generic (PLEG): container finished" podID="9dec0225-d6eb-4027-8086-2e17a4652344" containerID="faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d" exitCode=2
Jun 02 16:27:16.539844 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:16.539678    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" event={"ID":"9dec0225-d6eb-4027-8086-2e17a4652344","Type":"ContainerDied","Data":"faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d"}
Jun 02 16:27:18.964264 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:18.964236    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:27:19.048522 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.048494    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-cb302-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9dec0225-d6eb-4027-8086-2e17a4652344-success-200-isvc-cb302-kube-rbac-proxy-sar-config\") pod \"9dec0225-d6eb-4027-8086-2e17a4652344\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") "
Jun 02 16:27:19.048689 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.048533    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7249g\" (UniqueName: \"kubernetes.io/projected/9dec0225-d6eb-4027-8086-2e17a4652344-kube-api-access-7249g\") pod \"9dec0225-d6eb-4027-8086-2e17a4652344\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") "
Jun 02 16:27:19.048689 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.048569    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls\") pod \"9dec0225-d6eb-4027-8086-2e17a4652344\" (UID: \"9dec0225-d6eb-4027-8086-2e17a4652344\") "
Jun 02 16:27:19.048892 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.048869    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dec0225-d6eb-4027-8086-2e17a4652344-success-200-isvc-cb302-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-cb302-kube-rbac-proxy-sar-config") pod "9dec0225-d6eb-4027-8086-2e17a4652344" (UID: "9dec0225-d6eb-4027-8086-2e17a4652344"). InnerVolumeSpecName "success-200-isvc-cb302-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:27:19.050602 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.050573    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dec0225-d6eb-4027-8086-2e17a4652344-kube-api-access-7249g" (OuterVolumeSpecName: "kube-api-access-7249g") pod "9dec0225-d6eb-4027-8086-2e17a4652344" (UID: "9dec0225-d6eb-4027-8086-2e17a4652344"). InnerVolumeSpecName "kube-api-access-7249g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:27:19.050706 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.050625    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9dec0225-d6eb-4027-8086-2e17a4652344" (UID: "9dec0225-d6eb-4027-8086-2e17a4652344"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:27:19.149417 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.149341    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-cb302-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9dec0225-d6eb-4027-8086-2e17a4652344-success-200-isvc-cb302-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:27:19.149417 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.149367    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7249g\" (UniqueName: \"kubernetes.io/projected/9dec0225-d6eb-4027-8086-2e17a4652344-kube-api-access-7249g\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:27:19.149417 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.149378    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9dec0225-d6eb-4027-8086-2e17a4652344-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:27:19.549940 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.549888    2562 generic.go:358] "Generic (PLEG): container finished" podID="9dec0225-d6eb-4027-8086-2e17a4652344" containerID="f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4" exitCode=0
Jun 02 16:27:19.550135 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.549985    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"
Jun 02 16:27:19.550135 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.549980    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" event={"ID":"9dec0225-d6eb-4027-8086-2e17a4652344","Type":"ContainerDied","Data":"f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4"}
Jun 02 16:27:19.550135 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.550095    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t" event={"ID":"9dec0225-d6eb-4027-8086-2e17a4652344","Type":"ContainerDied","Data":"f76f9cd3e83015f18b50de5aaf98b3f74a17c98c1a0493cf37255013ad34cc84"}
Jun 02 16:27:19.550135 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.550111    2562 scope.go:117] "RemoveContainer" containerID="faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d"
Jun 02 16:27:19.558536 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.558521    2562 scope.go:117] "RemoveContainer" containerID="f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4"
Jun 02 16:27:19.565671 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.565653    2562 scope.go:117] "RemoveContainer" containerID="faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d"
Jun 02 16:27:19.565925 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:27:19.565890    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d\": container with ID starting with faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d not found: ID does not exist" containerID="faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d"
Jun 02 16:27:19.565984 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.565943    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d"} err="failed to get container status \"faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d\": rpc error: code = NotFound desc = could not find container \"faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d\": container with ID starting with faac265c3730a2ed0f2d3d3713de7718eb08f944f03832c35c4f6fa4dff0109d not found: ID does not exist"
Jun 02 16:27:19.565984 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.565968    2562 scope.go:117] "RemoveContainer" containerID="f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4"
Jun 02 16:27:19.566204 ip-10-0-142-13 kubenswrapper[2562]: E0602 16:27:19.566184    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4\": container with ID starting with f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4 not found: ID does not exist" containerID="f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4"
Jun 02 16:27:19.566268 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.566208    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4"} err="failed to get container status \"f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4\": rpc error: code = NotFound desc = could not find container \"f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4\": container with ID starting with f789f3171745ee4a7f9081721ea154afd2dc619ed3507cb84f5168c234f4cdd4 not found: ID does not exist"
Jun 02 16:27:19.571433 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.571413    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"]
Jun 02 16:27:19.576070 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:19.576049    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb302-predictor-9dcf7c765-pmb6t"]
Jun 02 16:27:20.600693 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:27:20.600660    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" path="/var/lib/kubelet/pods/9dec0225-d6eb-4027-8086-2e17a4652344/volumes"
Jun 02 16:31:34.660449 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:31:34.660335    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:31:34.666579 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:31:34.666561    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:34:56.282865 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:56.282832    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"]
Jun 02 16:34:56.283480 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:56.283138    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container" containerID="cri-o://f80affa01b5b91fe9b20c944238e996b7b19239342ad98a442b4a234b628f022" gracePeriod=30
Jun 02 16:34:56.283480 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:56.283175    2562 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kube-rbac-proxy" containerID="cri-o://01d922d5141f7d3a221226aefcd8cfacc307877af14f9a2cac7c8cbe17735284" gracePeriod=30
Jun 02 16:34:56.999417 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:56.999385    2562 generic.go:358] "Generic (PLEG): container finished" podID="170be138-1cf6-4494-953f-c87f63016e9a" containerID="01d922d5141f7d3a221226aefcd8cfacc307877af14f9a2cac7c8cbe17735284" exitCode=2
Jun 02 16:34:56.999596 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:56.999460    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" event={"ID":"170be138-1cf6-4494-953f-c87f63016e9a","Type":"ContainerDied","Data":"01d922d5141f7d3a221226aefcd8cfacc307877af14f9a2cac7c8cbe17735284"}
Jun 02 16:34:59.007825 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.007791    2562 generic.go:358] "Generic (PLEG): container finished" podID="170be138-1cf6-4494-953f-c87f63016e9a" containerID="f80affa01b5b91fe9b20c944238e996b7b19239342ad98a442b4a234b628f022" exitCode=0
Jun 02 16:34:59.008232 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.007870    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" event={"ID":"170be138-1cf6-4494-953f-c87f63016e9a","Type":"ContainerDied","Data":"f80affa01b5b91fe9b20c944238e996b7b19239342ad98a442b4a234b628f022"}
Jun 02 16:34:59.024865 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.024846    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:34:59.120603 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.120517    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"success-200-isvc-78d4f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/170be138-1cf6-4494-953f-c87f63016e9a-success-200-isvc-78d4f-kube-rbac-proxy-sar-config\") pod \"170be138-1cf6-4494-953f-c87f63016e9a\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") "
Jun 02 16:34:59.120603 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.120565    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m55vj\" (UniqueName: \"kubernetes.io/projected/170be138-1cf6-4494-953f-c87f63016e9a-kube-api-access-m55vj\") pod \"170be138-1cf6-4494-953f-c87f63016e9a\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") "
Jun 02 16:34:59.120603 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.120600    2562 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/170be138-1cf6-4494-953f-c87f63016e9a-proxy-tls\") pod \"170be138-1cf6-4494-953f-c87f63016e9a\" (UID: \"170be138-1cf6-4494-953f-c87f63016e9a\") "
Jun 02 16:34:59.120973 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.120944    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170be138-1cf6-4494-953f-c87f63016e9a-success-200-isvc-78d4f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "success-200-isvc-78d4f-kube-rbac-proxy-sar-config") pod "170be138-1cf6-4494-953f-c87f63016e9a" (UID: "170be138-1cf6-4494-953f-c87f63016e9a"). InnerVolumeSpecName "success-200-isvc-78d4f-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 16:34:59.122727 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.122702    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170be138-1cf6-4494-953f-c87f63016e9a-kube-api-access-m55vj" (OuterVolumeSpecName: "kube-api-access-m55vj") pod "170be138-1cf6-4494-953f-c87f63016e9a" (UID: "170be138-1cf6-4494-953f-c87f63016e9a"). InnerVolumeSpecName "kube-api-access-m55vj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:34:59.122787 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.122729    2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170be138-1cf6-4494-953f-c87f63016e9a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "170be138-1cf6-4494-953f-c87f63016e9a" (UID: "170be138-1cf6-4494-953f-c87f63016e9a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:34:59.221243 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.221210    2562 reconciler_common.go:299] "Volume detached for volume \"success-200-isvc-78d4f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/170be138-1cf6-4494-953f-c87f63016e9a-success-200-isvc-78d4f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:34:59.221243 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.221237    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m55vj\" (UniqueName: \"kubernetes.io/projected/170be138-1cf6-4494-953f-c87f63016e9a-kube-api-access-m55vj\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:34:59.221243 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:34:59.221249    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/170be138-1cf6-4494-953f-c87f63016e9a-proxy-tls\") on node \"ip-10-0-142-13.ec2.internal\" DevicePath \"\""
Jun 02 16:35:00.012663 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.012635    2562 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"
Jun 02 16:35:00.013160 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.012634    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj" event={"ID":"170be138-1cf6-4494-953f-c87f63016e9a","Type":"ContainerDied","Data":"bf28857ee4cb25cc5e55c42d59c096fb8aba7a842931bc0b8ab1ba086d1bb707"}
Jun 02 16:35:00.013160 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.012760    2562 scope.go:117] "RemoveContainer" containerID="01d922d5141f7d3a221226aefcd8cfacc307877af14f9a2cac7c8cbe17735284"
Jun 02 16:35:00.020733 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.020459    2562 scope.go:117] "RemoveContainer" containerID="f80affa01b5b91fe9b20c944238e996b7b19239342ad98a442b4a234b628f022"
Jun 02 16:35:00.035224 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.035201    2562 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"]
Jun 02 16:35:00.039161 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.039140    2562 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-78d4f-predictor-599d66c69f-fkpzj"]
Jun 02 16:35:00.600602 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:00.600566    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="170be138-1cf6-4494-953f-c87f63016e9a" path="/var/lib/kubelet/pods/170be138-1cf6-4494-953f-c87f63016e9a/volumes"
Jun 02 16:35:30.824326 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:30.824286    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pzdmg_4a83d011-679c-4bab-a182-b89f20266181/global-pull-secret-syncer/0.log"
Jun 02 16:35:30.913958 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:30.913892    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9bdmf_d4a6f02e-2650-4045-8a5d-b3b0bad6280d/konnectivity-agent/0.log"
Jun 02 16:35:31.071395 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:31.071366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-13.ec2.internal_74ec08f42b5a0b4796702a35bcb3d051/haproxy/0.log"
Jun 02 16:35:35.114453 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.114352    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x8vhv_ccc88934-7c14-4802-aee3-cb92ed967f0b/node-exporter/0.log"
Jun 02 16:35:35.139770 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.139742    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x8vhv_ccc88934-7c14-4802-aee3-cb92ed967f0b/kube-rbac-proxy/0.log"
Jun 02 16:35:35.164656 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.164635    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x8vhv_ccc88934-7c14-4802-aee3-cb92ed967f0b/init-textfile/0.log"
Jun 02 16:35:35.478792 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.478745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-f9bql_b4ff345a-bfac-4494-a4ef-da9a56a5821d/prometheus-operator/0.log"
Jun 02 16:35:35.500121 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.500096    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-f9bql_b4ff345a-bfac-4494-a4ef-da9a56a5821d/kube-rbac-proxy/0.log"
Jun 02 16:35:35.704633 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.704605    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fdb99758b-wrqfg_dddfcd5b-00f9-4746-badc-0ccdfa2340a7/thanos-query/0.log"
Jun 02 16:35:35.732109 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.732039    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fdb99758b-wrqfg_dddfcd5b-00f9-4746-badc-0ccdfa2340a7/kube-rbac-proxy-web/0.log"
Jun 02 16:35:35.759949 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.759903    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fdb99758b-wrqfg_dddfcd5b-00f9-4746-badc-0ccdfa2340a7/kube-rbac-proxy/0.log"
Jun 02 16:35:35.787788 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.787766    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fdb99758b-wrqfg_dddfcd5b-00f9-4746-badc-0ccdfa2340a7/prom-label-proxy/0.log"
Jun 02 16:35:35.818517 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.818493    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fdb99758b-wrqfg_dddfcd5b-00f9-4746-badc-0ccdfa2340a7/kube-rbac-proxy-rules/0.log"
Jun 02 16:35:35.850633 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:35.850612    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fdb99758b-wrqfg_dddfcd5b-00f9-4746-badc-0ccdfa2340a7/kube-rbac-proxy-metrics/0.log"
Jun 02 16:35:37.813530 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:37.813500    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-55fcd76598-g546m_2afa62c4-5f1e-4731-a38c-002aa3606ab3/console/0.log"
Jun 02 16:35:38.637048 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637016    2562 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"]
Jun 02 16:35:38.637345 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637334    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637391 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637346    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637391 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637356    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container"
Jun 02 16:35:38.637391 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637362    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container"
Jun 02 16:35:38.637391 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637372    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container"
Jun 02 16:35:38.637391 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637379    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container"
Jun 02 16:35:38.637391 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637388    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637394    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637403    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637408    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637415    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637419    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637463    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kserve-container"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637470    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="324a24e6-38b9-4b9a-ab7d-6c8daa16e31a" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637478    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kserve-container"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637483    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kserve-container"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637489    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="170be138-1cf6-4494-953f-c87f63016e9a" containerName="kube-rbac-proxy"
Jun 02 16:35:38.637573 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.637497    2562 memory_manager.go:356] "RemoveStaleState removing state" podUID="9dec0225-d6eb-4027-8086-2e17a4652344" containerName="kube-rbac-proxy"
Jun 02 16:35:38.641815 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.641796    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.643771 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.643754    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pm69g\"/\"openshift-service-ca.crt\""
Jun 02 16:35:38.644245 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.644230    2562 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pm69g\"/\"default-dockercfg-sjn2p\""
Jun 02 16:35:38.644351 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.644338    2562 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pm69g\"/\"kube-root-ca.crt\""
Jun 02 16:35:38.649689 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.649668    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"]
Jun 02 16:35:38.737469 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.737430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-lib-modules\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.737645 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.737486    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-proc\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.737645 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.737507    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ctzg\" (UniqueName: \"kubernetes.io/projected/9e3e2128-608c-4eac-9c90-8a50d674b6b0-kube-api-access-2ctzg\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.737645 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.737553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-podres\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.737748 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.737646    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-sys\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839088 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839055    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-proc\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839096    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2ctzg\" (UniqueName: \"kubernetes.io/projected/9e3e2128-608c-4eac-9c90-8a50d674b6b0-kube-api-access-2ctzg\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839126    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-podres\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839172    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-sys\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839191    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-proc\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839210    2562 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-lib-modules\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839293    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-sys\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839323    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-podres\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.839443 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.839325    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e3e2128-608c-4eac-9c90-8a50d674b6b0-lib-modules\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.847420 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.847398    2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ctzg\" (UniqueName: \"kubernetes.io/projected/9e3e2128-608c-4eac-9c90-8a50d674b6b0-kube-api-access-2ctzg\") pod \"perf-node-gather-daemonset-zj457\" (UID: \"9e3e2128-608c-4eac-9c90-8a50d674b6b0\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:38.952065 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:38.952035    2562 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:39.088695 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.088658    2562 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"]
Jun 02 16:35:39.091958 ip-10-0-142-13 kubenswrapper[2562]: W0602 16:35:39.091931    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9e3e2128_608c_4eac_9c90_8a50d674b6b0.slice/crio-c854e56ac89757d8444be1e10f162a9df21c66ae107e58d2aaa75fd37bf4a5c7 WatchSource:0}: Error finding container c854e56ac89757d8444be1e10f162a9df21c66ae107e58d2aaa75fd37bf4a5c7: Status 404 returned error can't find the container with id c854e56ac89757d8444be1e10f162a9df21c66ae107e58d2aaa75fd37bf4a5c7
Jun 02 16:35:39.093655 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.093633    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 16:35:39.105792 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.105768    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bqm8z_a7be4dd9-3fd8-4a8f-9697-785ae760e14e/dns/0.log"
Jun 02 16:35:39.130546 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.130519    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bqm8z_a7be4dd9-3fd8-4a8f-9697-785ae760e14e/kube-rbac-proxy/0.log"
Jun 02 16:35:39.137734 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.137707    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457" event={"ID":"9e3e2128-608c-4eac-9c90-8a50d674b6b0","Type":"ContainerStarted","Data":"c854e56ac89757d8444be1e10f162a9df21c66ae107e58d2aaa75fd37bf4a5c7"}
Jun 02 16:35:39.260136 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.260113    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sdcph_0bb9f541-1297-48c3-87f1-765335e0b0eb/dns-node-resolver/0.log"
Jun 02 16:35:39.760568 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.760529    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c44974544-6nsl4_3389aba7-0fa7-43fe-b5b9-994861a49a21/registry/0.log"
Jun 02 16:35:39.839847 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:39.839815    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfwt7_5d858e0a-17c3-4bc8-b185-1c981b666127/node-ca/0.log"
Jun 02 16:35:40.143812 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:40.143724    2562 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457" event={"ID":"9e3e2128-608c-4eac-9c90-8a50d674b6b0","Type":"ContainerStarted","Data":"c7dde76bc37dc8c2e873e56c5a6ed99e7a8982aa8ff0b9cdd77938d56fdda055"}
Jun 02 16:35:40.143984 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:40.143819    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:40.160005 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:40.159956    2562 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457" podStartSLOduration=2.159939316 podStartE2EDuration="2.159939316s" podCreationTimestamp="2026-06-02 16:35:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:35:40.158295346 +0000 UTC m=+2946.056015734" watchObservedRunningTime="2026-06-02 16:35:40.159939316 +0000 UTC m=+2946.057659701"
Jun 02 16:35:41.049041 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:41.049006    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbb7q_eff42c24-b415-4b7e-b900-3a00e7183c53/serve-healthcheck-canary/0.log"
Jun 02 16:35:41.463317 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:41.463287    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jcv2c_bb5e1206-06b4-4509-89d2-e4b6cdc5d966/kube-rbac-proxy/0.log"
Jun 02 16:35:41.488690 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:41.488659    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jcv2c_bb5e1206-06b4-4509-89d2-e4b6cdc5d966/exporter/0.log"
Jun 02 16:35:41.512969 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:41.512941    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jcv2c_bb5e1206-06b4-4509-89d2-e4b6cdc5d966/extractor/0.log"
Jun 02 16:35:43.693605 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:43.693523    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7b649f477f-m9459_ef893910-fee3-4249-88a9-b521442ea7d8/manager/0.log"
Jun 02 16:35:43.716514 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:43.716485    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-6b94ff949c-lxvlv_e146d349-da82-494f-9bb6-48280b7350b8/manager/0.log"
Jun 02 16:35:43.889999 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:43.889963    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-dcn2c_24489f3b-883f-4789-abec-21cfab38f508/manager/0.log"
Jun 02 16:35:46.157722 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:46.157692    2562 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-zj457"
Jun 02 16:35:49.581964 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:49.581907    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6hj2t_941b237a-3596-4d10-a02e-73e02d93b836/kube-multus/0.log"
Jun 02 16:35:50.038878 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.038853    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/kube-multus-additional-cni-plugins/0.log"
Jun 02 16:35:50.079366 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.079335    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/egress-router-binary-copy/0.log"
Jun 02 16:35:50.118689 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.118660    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/cni-plugins/0.log"
Jun 02 16:35:50.154997 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.154964    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/bond-cni-plugin/0.log"
Jun 02 16:35:50.189581 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.189551    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/routeoverride-cni/0.log"
Jun 02 16:35:50.228286 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.228211    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/whereabouts-cni-bincopy/0.log"
Jun 02 16:35:50.255293 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.255265    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wtc7p_bf747868-df96-43d9-8e2b-e0b6cb56045b/whereabouts-cni/0.log"
Jun 02 16:35:50.401599 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.401570    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jjrxf_d54bcdc8-8c47-4ede-9def-196f262b8a29/network-metrics-daemon/0.log"
Jun 02 16:35:50.424486 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:50.424459    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jjrxf_d54bcdc8-8c47-4ede-9def-196f262b8a29/kube-rbac-proxy/0.log"
Jun 02 16:35:51.642487 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.642457    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-controller/0.log"
Jun 02 16:35:51.687772 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.687718    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/0.log"
Jun 02 16:35:51.713230 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.713197    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovn-acl-logging/1.log"
Jun 02 16:35:51.737154 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.737122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/kube-rbac-proxy-node/0.log"
Jun 02 16:35:51.790355 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.790320    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/kube-rbac-proxy-ovn-metrics/0.log"
Jun 02 16:35:51.847015 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.846984    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/northd/0.log"
Jun 02 16:35:51.872463 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.872439    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/nbdb/0.log"
Jun 02 16:35:51.900579 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:51.900499    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/sbdb/0.log"
Jun 02 16:35:52.078211 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:52.078181    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qvffn_06c22a5b-522f-484f-b0bf-09f37b5ac339/ovnkube-controller/0.log"
Jun 02 16:35:53.618842 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:53.618807    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-r7254_a6572f3a-8d66-4e30-9ec0-138977fcfaca/network-check-target-container/0.log"
Jun 02 16:35:54.661502 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:54.661474    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-9qt9d_dca09013-6eef-48e9-886f-49647c0b34fa/iptables-alerter/0.log"
Jun 02 16:35:55.424539 ip-10-0-142-13 kubenswrapper[2562]: I0602 16:35:55.424506    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-nptbl_6dd51254-d67e-4784-9cf7-65c6d1985e2b/tuned/0.log"