Jun 02 15:44:15.741292 ip-10-0-140-193 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jun 02 15:44:15.741305 ip-10-0-140-193 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jun 02 15:44:15.741315 ip-10-0-140-193 systemd[1]: kubelet.service: Failed with result 'resources'.
Jun 02 15:44:15.741804 ip-10-0-140-193 systemd[1]: Failed to start Kubernetes Kubelet.
Jun 02 15:44:25.934138 ip-10-0-140-193 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jun 02 15:44:25.934158 ip-10-0-140-193 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0e4ead3c4f554bf78dd5712c02297461 --
Jun 02 15:46:55.248333 ip-10-0-140-193 systemd[1]: Starting Kubernetes Kubelet...
Jun 02 15:46:55.716590 ip-10-0-140-193 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 15:46:55.716590 ip-10-0-140-193 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 02 15:46:55.716590 ip-10-0-140-193 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 15:46:55.716590 ip-10-0-140-193 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 02 15:46:55.716590 ip-10-0-140-193 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 15:46:55.718367 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.718275    2563 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 02 15:46:55.721180 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721163    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:55.721180 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721180    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721183    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721187    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721191    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721194    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721197    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721200    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721203    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721210    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721213    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721216    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721219    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721221    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721224    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721226    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721229    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721231    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721234    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721237    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721239    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:55.721247 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721242    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721245    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721247    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721250    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721253    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721256    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721258    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721261    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721263    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721266    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721268    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721271    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721273    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721275    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721280    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721284    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721287    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721290    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721292    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721295    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:55.721756 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721298    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721301    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721303    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721306    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721309    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721311    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721314    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721316    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721318    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721321    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721325    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721329    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721332    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721334    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721338    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721340    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721343    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721346    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721348    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721351    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:55.722289 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721353    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721355    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721358    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721360    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721363    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721365    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721368    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721371    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721373    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721376    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721378    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721380    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721384    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721386    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721389    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721391    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721395    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721397    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721400    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:55.722778 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721404    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721407    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721409    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721412    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721414    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.721417    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722509    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722517    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722520    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722523    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722526    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722528    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722531    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722533    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722537    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722540    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722543    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722545    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722549    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:55.723254 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722552    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722555    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722557    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722560    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722563    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722565    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722569    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722571    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722574    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722576    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722579    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722582    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722584    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722587    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722589    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722592    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722594    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722597    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722599    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722603    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:55.723712 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722606    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722608    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722610    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722613    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722616    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722618    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722621    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722623    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722626    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722628    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722631    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722635    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722638    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722640    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722643    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722646    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722648    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722651    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722654    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722659    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:55.724223 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722662    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722664    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722667    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722670    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722672    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722675    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722677    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722680    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722682    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722685    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722687    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722689    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722692    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722694    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722696    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722699    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722702    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722704    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722707    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722709    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:55.724731 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722712    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722714    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722717    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722719    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722721    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722724    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722726    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722728    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722731    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722733    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722736    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722738    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.722741    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722811    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722822    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722832    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722839    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722846    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722849    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722854    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722858    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 02 15:46:55.725230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722862    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722865    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722868    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722871    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722874    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722877    2563 flags.go:64] FLAG: --cgroup-root=""
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722880    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722883    2563 flags.go:64] FLAG: --client-ca-file=""
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722886    2563 flags.go:64] FLAG: --cloud-config=""
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722889    2563 flags.go:64] FLAG: --cloud-provider="external"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722891    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722897    2563 flags.go:64] FLAG: --cluster-domain=""
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722900    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722903    2563 flags.go:64] FLAG: --config-dir=""
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722905    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722909    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722913    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722916    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722919    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722922    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722925    2563 flags.go:64] FLAG: --contention-profiling="false"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722928    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722931    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722934    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722937    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 02 15:46:55.725758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722942    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722945    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722948    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722950    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722954    2563 flags.go:64] FLAG: --enable-server="true"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722957    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722961    2563 flags.go:64] FLAG: --event-burst="100"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722964    2563 flags.go:64] FLAG: --event-qps="50"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722967    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722970    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722973    2563 flags.go:64] FLAG: --eviction-hard=""
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722980    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722983    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722986    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722989    2563 flags.go:64] FLAG: --eviction-soft=""
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722992    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722995    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.722998    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723001    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723004    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723021    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723026    2563 flags.go:64] FLAG: --feature-gates=""
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723031    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723036    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723040    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 02 15:46:55.726398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723043    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723046    2563 flags.go:64] FLAG: --healthz-port="10248"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723049    2563 flags.go:64] FLAG: --help="false"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723052    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-140-193.ec2.internal"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723055    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723058    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723060    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723064    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723068    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723071    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723074    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723077    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723080    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723082    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723085    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723089    2563 flags.go:64] FLAG: --kube-reserved=""
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723091    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723094    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723097    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723099    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723102    2563 flags.go:64] FLAG: --lock-file=""
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723105    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723107    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723111    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 02 15:46:55.727106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723116    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723119    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723122    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723125    2563 flags.go:64] FLAG: --logging-format="text"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723128    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723132    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723135    2563 flags.go:64] FLAG: --manifest-url=""
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723137    2563 flags.go:64] FLAG: --manifest-url-header=""
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723141    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723144    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723148    2563 flags.go:64] FLAG: --max-pods="110"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723151    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723154    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723157    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723160    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723163    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723166    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723170    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723177    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723180    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723183    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723186    2563 flags.go:64] FLAG: --pod-cidr=""
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723189    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
Jun 02 15:46:55.727695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723196    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723198    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723201    2563 flags.go:64] FLAG: --pods-per-core="0"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723204    2563 flags.go:64] FLAG: --port="10250"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723208    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723213    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c87e4e6d12694d83"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723217    2563 flags.go:64] FLAG: --qos-reserved=""
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723219    2563 flags.go:64] FLAG: --read-only-port="10255"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723222    2563 flags.go:64] FLAG: --register-node="true"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723225    2563 flags.go:64] FLAG: --register-schedulable="true"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723228    2563 flags.go:64] FLAG: --register-with-taints=""
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723232    2563 flags.go:64] FLAG: --registry-burst="10"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723235    2563 flags.go:64] FLAG: --registry-qps="5"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723237    2563 flags.go:64] FLAG: --reserved-cpus=""
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723241    2563 flags.go:64] FLAG: --reserved-memory=""
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723244    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723247    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723250    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723253    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723256    2563 flags.go:64] FLAG: --runonce="false"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723259    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723262    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723265    2563 flags.go:64] FLAG: --seccomp-default="false"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723268    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723271    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723275    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 02 15:46:55.728304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723279    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723282    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723285    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723288    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723291    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723294    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723297    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723299    2563 flags.go:64] FLAG: --system-cgroups=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723302    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723308    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723310    2563 flags.go:64] FLAG: --tls-cert-file=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723314    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723319    2563 flags.go:64] FLAG: --tls-min-version=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723321    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723325    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723327    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723330    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723333    2563 flags.go:64] FLAG: --v="2"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723337    2563 flags.go:64] FLAG: --version="false"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723342    2563 flags.go:64] FLAG: --vmodule=""
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723346    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723349    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723438    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723441    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:55.728938 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723445    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723447    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723450    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723452    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723455    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723457    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723460    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723463    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723466    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723469    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723472    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723474    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723477    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723479    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723482    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723484    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723486    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723489    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723491    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723495    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:55.729826 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723497    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723500    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723502    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723505    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723507    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723510    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723514    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723517    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723520    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723523    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723526    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723529    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723532    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723534    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723537    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723539    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723542    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723544    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723547    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:55.730650 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723550    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723553    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723555    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723558    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723560    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723563    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723565    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723568    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723570    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723572    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723575    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723577    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723581    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723583    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723585    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723588    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723590    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723593    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723595    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723597    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:55.731381 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723600    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723602    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723605    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723607    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723609    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723612    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723614    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723617    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723619    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723621    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723624    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723626    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723630    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723632    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723635    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723637    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723640    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723642    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723645    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723648    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:55.731881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723650    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:55.732395 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723652    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:55.732395 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723655    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:55.732395 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723657    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:55.732395 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.723661    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:55.732395 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.723666    2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 15:46:55.733315 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.733293    2563 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
Jun 02 15:46:55.733361 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.733317    2563 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733368    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733374    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733377    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733381    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733384    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733387    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733390    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733393    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:55.733392 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733396    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733399    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733402    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733405    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733407    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733410    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733414    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733418    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733421    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733423    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733426    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733428    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733431    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733433    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733436    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733439    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733442    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733444    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733446    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733449    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:55.733620 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733451    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733454    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733457    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733459    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733461    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733464    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733467    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733469    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733472    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733474    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733477    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733479    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733482    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733484    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733487    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733490    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733492    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733495    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733498    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733500    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:55.734235 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733502    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733505    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733507    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733510    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733512    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733515    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733517    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733520    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733522    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733526    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733531    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733533    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733536    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733539    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733541    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733544    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733546    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733549    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733551    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:55.734792 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733554    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733557    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733559    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733562    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733564    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733567    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733569    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733572    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733575    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733578    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733581    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733583    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733586    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733588    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733591    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733593    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733596    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733598    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:55.735287 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733600    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.733606    2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733725    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733730    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733734    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733737    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733739    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733742    2563 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733745    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733747    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733750    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733753    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733756    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733759    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733762    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 15:46:55.735721 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733764    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733767    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733769    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733772    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733774    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733777    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733780    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733782    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733784    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733788    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733790    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733793    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733795    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733798    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733800    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733803    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733805    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733808    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733810    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733812    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 15:46:55.736119 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733815    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733817    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733819    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733822    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733824    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733826    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733829    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733831    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733834    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733836    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733839    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733842    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733845    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733847    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733849    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733852    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733854    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733856    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733859    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733861    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 15:46:55.736605 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733864    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733866    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733870    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733872    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733874    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733877    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733879    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733881    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733884    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733886    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733890    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733894    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733896    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733899    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733901    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733904    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733906    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733908    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733911    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 15:46:55.737151 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733913    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733916    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733919    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733922    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733924    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733926    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733929    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733931    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733934    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733936    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733940    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733944    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733947    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:55.733950    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.733954    2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 15:46:55.737629 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.734732    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 02 15:46:55.738048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.736880    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 02 15:46:55.738048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.737721    2563 server.go:1019] "Starting client certificate rotation"
Jun 02 15:46:55.738048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.737821    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 15:46:55.738048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.737853    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 15:46:55.761946 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.761925    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 15:46:55.763625 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.763605    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 15:46:55.782204 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.782184    2563 log.go:25] "Validated CRI v1 runtime API"
Jun 02 15:46:55.788720 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.788699    2563 log.go:25] "Validated CRI v1 image API"
Jun 02 15:46:55.792523 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.792505    2563 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 02 15:46:55.795676 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.795655    2563 fs.go:135] Filesystem UUIDs: map[4f7c0628-a8d2-4815-acfe-7f478aa1bfdb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c234ba4c-21cb-4a81-8689-b53f16feff30:/dev/nvme0n1p4]
Jun 02 15:46:55.795718 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.795677    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 02 15:46:55.799581 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.799564    2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 15:46:55.801593 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.801491    2563 manager.go:217] Machine: {Timestamp:2026-06-02 15:46:55.800885452 +0000 UTC m=+0.424674295 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100660 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2460ecb681c6d4061ea7cbc205e5ef SystemUUID:ec2460ec-b681-c6d4-061e-a7cbc205e5ef BootID:0e4ead3c-4f55-4bf7-8dd5-712c02297461 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:5e:51:cf:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:5e:51:cf:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:45:e7:1c:ac:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 02 15:46:55.802315 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.802304    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 02 15:46:55.802401 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.802388    2563 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 02 15:46:55.804383 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.804361    2563 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 02 15:46:55.804530 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.804388    2563 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-193.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 02 15:46:55.804576 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.804541    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jun 02 15:46:55.804576 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.804549    2563 container_manager_linux.go:306] "Creating device plugin manager"
Jun 02 15:46:55.804576 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.804557    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 15:46:55.805437 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.805426    2563 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 15:46:55.806969 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.806959    2563 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 15:46:55.807095 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.807086    2563 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Jun 02 15:46:55.809547 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.809538    2563 kubelet.go:491] "Attempting to sync node with API server"
Jun 02 15:46:55.809585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.809551    2563 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 02 15:46:55.809585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.809563    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 02 15:46:55.809585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.809572    2563 kubelet.go:397] "Adding apiserver pod source"
Jun 02 15:46:55.809585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.809583    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 02 15:46:55.810551 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.810538    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 15:46:55.810591 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.810557    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 15:46:55.818849 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.818827    2563 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
Jun 02 15:46:55.820605 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.820590    2563 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 02 15:46:55.823132 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823118    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823138    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823148    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823157    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823167    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823175    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823181    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823186    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 02 15:46:55.823195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823193    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 02 15:46:55.823397 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823201    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 02 15:46:55.823397 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823215    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 02 15:46:55.823397 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.823221    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 02 15:46:55.824909 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.824885    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 02 15:46:55.824909 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.824903    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 02 15:46:55.826150 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.826119    2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 02 15:46:55.826579 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.826114    2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 02 15:46:55.828358 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.828345    2563 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Jun 02 15:46:55.828396 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.828388    2563 server.go:1295] "Started kubelet"
Jun 02 15:46:55.828538 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.828483    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 02 15:46:55.828593 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.828474    2563 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 02 15:46:55.828643 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.828605    2563 server_v1.go:47] "podresources" method="list" useActivePods=true
Jun 02 15:46:55.829245 ip-10-0-140-193 systemd[1]: Started Kubernetes Kubelet.
Jun 02 15:46:55.829975 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.829961    2563 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 02 15:46:55.831240 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.831227    2563 server.go:317] "Adding debug handlers to kubelet server"
Jun 02 15:46:55.837506 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.837461    2563 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-193.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 15:46:55.839672 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.839642    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 02 15:46:55.839760 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.837448    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-193.ec2.internal.18b54e34ac265978  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-193.ec2.internal,UID:ip-10-0-140-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-193.ec2.internal,},FirstTimestamp:2026-06-02 15:46:55.828359544 +0000 UTC m=+0.452148381,LastTimestamp:2026-06-02 15:46:55.828359544 +0000 UTC m=+0.452148381,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-193.ec2.internal,}"
Jun 02 15:46:55.839931 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.839914    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 02 15:46:55.839999 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.839945    2563 volume_manager.go:295] "The desired_state_of_world populator starts"
Jun 02 15:46:55.839999 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.839954    2563 volume_manager.go:297] "Starting Kubelet Volume Manager"
Jun 02 15:46:55.840147 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.840132    2563 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Jun 02 15:46:55.840276 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.840224    2563 reconstruct.go:97] "Volume reconstruction finished"
Jun 02 15:46:55.840358 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.840347    2563 reconciler.go:26] "Reconciler: start to sync state"
Jun 02 15:46:55.840912 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.840895    2563 factory.go:55] Registering systemd factory
Jun 02 15:46:55.840999 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.840925    2563 factory.go:223] Registration of the systemd container factory successfully
Jun 02 15:46:55.841227 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.841197    2563 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 02 15:46:55.841227 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.841203    2563 factory.go:153] Registering CRI-O factory
Jun 02 15:46:55.841227 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.841228    2563 factory.go:223] Registration of the crio container factory successfully
Jun 02 15:46:55.841416 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.841281    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 02 15:46:55.841416 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.841306    2563 factory.go:103] Registering Raw factory
Jun 02 15:46:55.841416 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.841322    2563 manager.go:1196] Started watching for new ooms in manager
Jun 02 15:46:55.841617 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.841591    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:55.842153 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.842137    2563 manager.go:319] Starting recovery of all containers
Jun 02 15:46:55.846892 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.846859    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jun 02 15:46:55.847095 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.847072    2563 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jun 02 15:46:55.852316 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.852144    2563 manager.go:324] Recovery completed
Jun 02 15:46:55.853931 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.853905    2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jun 02 15:46:55.856940 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.856927    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:55.858779 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.858757    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:55.858868 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.858797    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:55.858868 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.858812    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:55.859342 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.859320    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 02 15:46:55.859342 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.859331    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 02 15:46:55.859452 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.859351    2563 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 15:46:55.861460 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.861393    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-193.ec2.internal.18b54e34adf685d8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-193.ec2.internal,UID:ip-10-0-140-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-193.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-193.ec2.internal,},FirstTimestamp:2026-06-02 15:46:55.858779608 +0000 UTC m=+0.482568451,LastTimestamp:2026-06-02 15:46:55.858779608 +0000 UTC m=+0.482568451,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-193.ec2.internal,}"
Jun 02 15:46:55.861788 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.861774    2563 policy_none.go:49] "None policy: Start"
Jun 02 15:46:55.861854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.861795    2563 memory_manager.go:186] "Starting memorymanager" policy="None"
Jun 02 15:46:55.861854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.861829    2563 state_mem.go:35] "Initializing new in-memory state store"
Jun 02 15:46:55.868909 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.868846    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-193.ec2.internal.18b54e34adf6e1cc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-193.ec2.internal,UID:ip-10-0-140-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-193.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-193.ec2.internal,},FirstTimestamp:2026-06-02 15:46:55.858803148 +0000 UTC m=+0.482591987,LastTimestamp:2026-06-02 15:46:55.858803148 +0000 UTC m=+0.482591987,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-193.ec2.internal,}"
Jun 02 15:46:55.878927 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.878851    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-193.ec2.internal.18b54e34adf71ab9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-193.ec2.internal,UID:ip-10-0-140-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-193.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-193.ec2.internal,},FirstTimestamp:2026-06-02 15:46:55.858817721 +0000 UTC m=+0.482606561,LastTimestamp:2026-06-02 15:46:55.858817721 +0000 UTC m=+0.482606561,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-193.ec2.internal,}"
Jun 02 15:46:55.898680 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.898663    2563 manager.go:341] "Starting Device Plugin manager"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.898703    2563 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.898717    2563 server.go:85] "Starting device plugin registration server"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.898972    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.898985    2563 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.899173    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.899241    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.899250    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.899683    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 02 15:46:55.914489 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.899724    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:55.916790 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.916716    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-193.ec2.internal.18b54e34b07d0f45  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-193.ec2.internal,UID:ip-10-0-140-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-140-193.ec2.internal,},FirstTimestamp:2026-06-02 15:46:55.901151045 +0000 UTC m=+0.524939874,LastTimestamp:2026-06-02 15:46:55.901151045 +0000 UTC m=+0.524939874,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-193.ec2.internal,}"
Jun 02 15:46:55.939970 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.939938    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w8ctv"
Jun 02 15:46:55.953440 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.953418    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w8ctv"
Jun 02 15:46:55.974756 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.974682    2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Jun 02 15:46:55.976022 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.975994    2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Jun 02 15:46:55.976098 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.976037    2563 status_manager.go:230] "Starting to sync pod status with apiserver"
Jun 02 15:46:55.976098 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.976062    2563 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 02 15:46:55.976098 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.976071    2563 kubelet.go:2451] "Starting kubelet main sync loop"
Jun 02 15:46:55.976225 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:55.976118    2563 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jun 02 15:46:55.990467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.990442    2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:55.999751 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:55.999740    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:56.000550 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.000531    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:56.000641 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.000563    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:56.000641 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.000573    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:56.000641 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.000595    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.017625 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.017604    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.017696 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.017627    2563 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-193.ec2.internal\": node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.039941 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.039911    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.076810 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.076764    2563 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"]
Jun 02 15:46:56.076911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.076861    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:56.078717 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.078699    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:56.078786 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.078729    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:56.078786 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.078739    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:56.080053 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080039    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:56.080227 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080210    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.080287 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080247    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:56.080840 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080825    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:56.080911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080827    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:56.080911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080851    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:56.080911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080867    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:56.080911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080875    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:56.080911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.080894    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:56.082033 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.082001    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.082085 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.082043    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 15:46:56.084219 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.084202    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 15:46:56.084294 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.084230    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 15:46:56.084294 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.084239    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeHasSufficientPID"
Jun 02 15:46:56.106834 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.106809    2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-193.ec2.internal\" not found" node="ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.111424 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.111406    2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-193.ec2.internal\" not found" node="ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.140329 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.140300    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.141040 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.141005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e230094760b42b9068c20c1fe959349e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal\" (UID: \"e230094760b42b9068c20c1fe959349e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.141096 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.141050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e230094760b42b9068c20c1fe959349e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal\" (UID: \"e230094760b42b9068c20c1fe959349e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.141096 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.141068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cc8efb4d7faafb9447182ed8c6f6fdb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-193.ec2.internal\" (UID: \"2cc8efb4d7faafb9447182ed8c6f6fdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.241051 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.240969    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.242119 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.242100    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cc8efb4d7faafb9447182ed8c6f6fdb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-193.ec2.internal\" (UID: \"2cc8efb4d7faafb9447182ed8c6f6fdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.242171 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.242128    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e230094760b42b9068c20c1fe959349e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal\" (UID: \"e230094760b42b9068c20c1fe959349e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.242171 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.242150    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e230094760b42b9068c20c1fe959349e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal\" (UID: \"e230094760b42b9068c20c1fe959349e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.242239 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.242204    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2cc8efb4d7faafb9447182ed8c6f6fdb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-193.ec2.internal\" (UID: \"2cc8efb4d7faafb9447182ed8c6f6fdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.242239 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.242204    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e230094760b42b9068c20c1fe959349e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal\" (UID: \"e230094760b42b9068c20c1fe959349e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.242239 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.242218    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e230094760b42b9068c20c1fe959349e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal\" (UID: \"e230094760b42b9068c20c1fe959349e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.341579 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.341529    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.409815 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.409785    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.415465 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.415443    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.442697 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.442668    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.543168 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.543068    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.643509 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.643461    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-193.ec2.internal\" not found"
Jun 02 15:46:56.655717 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.655695    2563 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:56.737860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.737813    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 02 15:46:56.738349 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.737998    2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 02 15:46:56.738349 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.738033    2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 02 15:46:56.741001 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.740986    2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.764521 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.764495    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 15:46:56.765837 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.765822    2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal"
Jun 02 15:46:56.776801 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.776782    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 15:46:56.810006 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.809980    2563 apiserver.go:52] "Watching apiserver"
Jun 02 15:46:56.817923 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.817899    2563 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 02 15:46:56.819231 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.819164    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-5r476","openshift-cluster-node-tuning-operator/tuned-qvdd2","openshift-multus/multus-vqrnf","openshift-network-diagnostics/network-check-target-kpzkn","openshift-network-operator/iptables-alerter-2jm4l","openshift-ovn-kubernetes/ovnkube-node-9p9h9","kube-system/konnectivity-agent-66b65","kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2","openshift-dns/node-resolver-2ghc2","openshift-image-registry/node-ca-6glkd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal","openshift-multus/multus-additional-cni-plugins-jvn72"]
Jun 02 15:46:56.820754 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.820732    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:56.820854 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.820822    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:46:56.823983 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.823958    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.825345 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.825324    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.826552 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.826531    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.826651 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.826560    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:46:56.826705 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.826660    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:46:56.826901 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.826887    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.826953 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.826938    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vsw8b\""
Jun 02 15:46:56.826992 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.826937    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.827728 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.827707    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.828287 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.828269    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 02 15:46:56.828466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.828451    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.828758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.828743    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.828928 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.828907    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.829133 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.829115    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 02 15:46:56.829203 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.829139    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.829701 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.829682    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s5vwk\""
Jun 02 15:46:56.829814 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.829789    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xcvq4\""
Jun 02 15:46:56.829922 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.829907    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.830235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.830219    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 02 15:46:56.830292 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.830249    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.830341 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.830298    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.833330 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.833310    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.833795 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.833756    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 02 15:46:56.833981 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.833800    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9w8fp\""
Jun 02 15:46:56.833981 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.833830    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 02 15:46:56.833981 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.833838    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 02 15:46:56.833981 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.833890    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 02 15:46:56.834221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834073    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.834221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834104    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 02 15:46:56.834221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834107    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.834221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834193    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 02 15:46:56.834221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834206    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.834221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834214    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2cnjs\""
Jun 02 15:46:56.834423 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834246    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gnmp6\""
Jun 02 15:46:56.834423 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.834258    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 02 15:46:56.835151 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.835136    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.835789 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.835776    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.836195 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.836179    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.836483 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.836461    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.836723 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.836708    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pn2ph\""
Jun 02 15:46:56.837596 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.837581    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 02 15:46:56.837844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.837824    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 02 15:46:56.837935 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.837889    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 02 15:46:56.837935 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.837892    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lqclj\""
Jun 02 15:46:56.838887 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.838869    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 02 15:46:56.838972 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.838875    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 02 15:46:56.838972 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.838950    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p77xz\""
Jun 02 15:46:56.840151 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.840136    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 02 15:46:56.841097 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.841082    2563 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Jun 02 15:46:56.844800 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.844880 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-modprobe-d\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.844880 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-lib-modules\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.844880 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-cnibin\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec80ae08-6054-4495-a85d-c88fc5f066a8-cni-binary-copy\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-netns\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-socket-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-system-cni-dir\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-systemd\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-tuned\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.845036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.844997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-etc-selinux\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-slash\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-var-lib-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-ovn\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/461355bd-6507-41f6-a7cf-0c62360f9a90-konnectivity-ca\") pod \"konnectivity-agent-66b65\" (UID: \"461355bd-6507-41f6-a7cf-0c62360f9a90\") " pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czpf9\" (UniqueName: \"kubernetes.io/projected/d60aee99-f527-46fb-8992-2945e473f6d4-kube-api-access-czpf9\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxl5v\" (UniqueName: \"kubernetes.io/projected/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-kube-api-access-lxl5v\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-k8s-cni-cncf-io\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-daemon-config\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp85n\" (UniqueName: \"kubernetes.io/projected/a411c79a-4777-4a0e-a335-572999714887-kube-api-access-hp85n\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovnkube-config\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysconfig\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysctl-d\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-socket-dir-parent\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-kubelet\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-env-overrides\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-kubernetes\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-sys-fs\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845581    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-node-log\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-var-lib-kubelet\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-conf-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-os-release\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4097c2c0-12e3-478d-8b5f-c0565f205645-iptables-alerter-script\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-log-socket\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.845794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-tmp\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-multus-certs\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-etc-kubernetes\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-run-netns\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysctl-conf\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-cni-bin\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.845994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-cni-netd\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1992ad1c-cbee-4529-a8a8-b05b61961292-hosts-file\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d60aee99-f527-46fb-8992-2945e473f6d4-serviceca\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22hhn\" (UniqueName: \"kubernetes.io/projected/ec80ae08-6054-4495-a85d-c88fc5f066a8-kube-api-access-22hhn\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-device-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-cnibin\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st48j\" (UniqueName: \"kubernetes.io/projected/4097c2c0-12e3-478d-8b5f-c0565f205645-kube-api-access-st48j\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovn-node-metrics-cert\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrvt\" (UniqueName: \"kubernetes.io/projected/560b50bf-edb4-4b38-950c-a898eb7bc4b9-kube-api-access-6wrvt\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m5n4\" (UniqueName: \"kubernetes.io/projected/1992ad1c-cbee-4529-a8a8-b05b61961292-kube-api-access-9m5n4\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-cni-multus\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4097c2c0-12e3-478d-8b5f-c0565f205645-host-slash\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1992ad1c-cbee-4529-a8a8-b05b61961292-tmp-dir\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/461355bd-6507-41f6-a7cf-0c62360f9a90-agent-certs\") pod \"konnectivity-agent-66b65\" (UID: \"461355bd-6507-41f6-a7cf-0c62360f9a90\") " pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-sys\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-host\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdpsx\" (UniqueName: \"kubernetes.io/projected/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-kube-api-access-fdpsx\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-cni-binary-copy\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-registration-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovnkube-script-lib\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d60aee99-f527-46fb-8992-2945e473f6d4-host\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.846910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-systemd\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-hostroot\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-cni-bin\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-run\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-os-release\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-systemd-units\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-etc-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-system-cni-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-cni-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-kubelet\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.847385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.846845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhnjv\" (UniqueName: \"kubernetes.io/projected/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-kube-api-access-bhnjv\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:56.858818 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.858796    2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 15:46:56.900263 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.900241    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kp2tg"
Jun 02 15:46:56.910126 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.910102    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kp2tg"
Jun 02 15:46:56.941244 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:56.941203    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode230094760b42b9068c20c1fe959349e.slice/crio-fcc4016368e8c8fbd0a0eb00781beab764ac14c8ee624f645fa73022d1894389 WatchSource:0}: Error finding container fcc4016368e8c8fbd0a0eb00781beab764ac14c8ee624f645fa73022d1894389: Status 404 returned error can't find the container with id fcc4016368e8c8fbd0a0eb00781beab764ac14c8ee624f645fa73022d1894389
Jun 02 15:46:56.941647 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:56.941622    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc8efb4d7faafb9447182ed8c6f6fdb.slice/crio-3bab80b4c4098560ea96ae3666cbd7bbd5b9b1d8d5916543033b94f75ca03ff7 WatchSource:0}: Error finding container 3bab80b4c4098560ea96ae3666cbd7bbd5b9b1d8d5916543033b94f75ca03ff7: Status 404 returned error can't find the container with id 3bab80b4c4098560ea96ae3666cbd7bbd5b9b1d8d5916543033b94f75ca03ff7
Jun 02 15:46:56.945145 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.945116    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 15:46:56.947123 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947100    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-daemon-config\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.947216 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947144    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hp85n\" (UniqueName: \"kubernetes.io/projected/a411c79a-4777-4a0e-a335-572999714887-kube-api-access-hp85n\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.947216 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947178    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovnkube-config\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.947216 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947208    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysconfig\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.947369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947232    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysctl-d\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.947369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947255    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-socket-dir-parent\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.947369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947279    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:46:56.947369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947306    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.947369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947334    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-kubelet\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.947369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947361    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-env-overrides\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947387    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-kubernetes\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947412    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-sys-fs\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947434    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-node-log\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947456    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-var-lib-kubelet\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947481    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-conf-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947495    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysctl-d\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947507    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947533    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-os-release\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947560    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4097c2c0-12e3-478d-8b5f-c0565f205645-iptables-alerter-script\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.947653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947589    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-daemon-config\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947673    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-sys-fs\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947712    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-socket-dir-parent\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947748    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysconfig\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947782    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-conf-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947808    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-node-log\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947858    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-var-lib-kubelet\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947932    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-os-release\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947962    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.947992    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4097c2c0-12e3-478d-8b5f-c0565f205645-iptables-alerter-script\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948066    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-kubernetes\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948097    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-kubelet\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948118    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-log-socket\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948133    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-tmp\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948136    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovnkube-config\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948155    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-multus-certs\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948176    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-etc-kubernetes\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948199    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.948235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948200    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-log-socket\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948215    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948231    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-run-netns\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948247    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-multus-certs\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948270    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysctl-conf\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948288    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-cni-bin\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948304    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-cni-netd\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948307    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-env-overrides\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948318    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1992ad1c-cbee-4529-a8a8-b05b61961292-hosts-file\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948333    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d60aee99-f527-46fb-8992-2945e473f6d4-serviceca\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948351    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-run-netns\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948353    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-22hhn\" (UniqueName: \"kubernetes.io/projected/ec80ae08-6054-4495-a85d-c88fc5f066a8-kube-api-access-22hhn\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948379    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-device-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948395    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-cnibin\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948411    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-st48j\" (UniqueName: \"kubernetes.io/projected/4097c2c0-12e3-478d-8b5f-c0565f205645-kube-api-access-st48j\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948427    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovn-node-metrics-cert\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948443    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrvt\" (UniqueName: \"kubernetes.io/projected/560b50bf-edb4-4b38-950c-a898eb7bc4b9-kube-api-access-6wrvt\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.948991 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948459    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9m5n4\" (UniqueName: \"kubernetes.io/projected/1992ad1c-cbee-4529-a8a8-b05b61961292-kube-api-access-9m5n4\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948464    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948475    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-cni-multus\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948491    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4097c2c0-12e3-478d-8b5f-c0565f205645-host-slash\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948508    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1992ad1c-cbee-4529-a8a8-b05b61961292-tmp-dir\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948510    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-cnibin\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948497    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-cni-bin\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948545    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-etc-kubernetes\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948578    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-cni-netd\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948601    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.948607    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.948655    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:46:57.448633969 +0000 UTC m=+2.072422793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948689    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-device-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1992ad1c-cbee-4529-a8a8-b05b61961292-hosts-file\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948965    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-cni-multus\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948990    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.949892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.948527    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949192    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d60aee99-f527-46fb-8992-2945e473f6d4-serviceca\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949217    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/461355bd-6507-41f6-a7cf-0c62360f9a90-agent-certs\") pod \"konnectivity-agent-66b65\" (UID: \"461355bd-6507-41f6-a7cf-0c62360f9a90\") " pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949244    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-sys\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949266    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-host\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949289    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fdpsx\" (UniqueName: \"kubernetes.io/projected/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-kube-api-access-fdpsx\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949314    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-cni-binary-copy\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949319    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-sysctl-conf\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949342    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949355    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4097c2c0-12e3-478d-8b5f-c0565f205645-host-slash\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949369    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-registration-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949395    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949419    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovnkube-script-lib\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949442    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d60aee99-f527-46fb-8992-2945e473f6d4-host\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949465    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-systemd\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949487    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-hostroot\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949511    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-cni-bin\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949573    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-run\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.950686 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949599    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-os-release\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949623    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-systemd-units\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949647    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-etc-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949673    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-system-cni-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949690    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1992ad1c-cbee-4529-a8a8-b05b61961292-tmp-dir\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949697    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-cni-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949728    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-kubelet\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949754    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bhnjv\" (UniqueName: \"kubernetes.io/projected/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-kube-api-access-bhnjv\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949777    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949785    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-sys\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949801    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-modprobe-d\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949825    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-lib-modules\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949853    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-cnibin\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949910    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec80ae08-6054-4495-a85d-c88fc5f066a8-cni-binary-copy\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949928    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-netns\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949944    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-socket-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949959    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-system-cni-dir\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949975    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-systemd\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.951500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949989    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-tuned\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950036    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-etc-selinux\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950252    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-slash\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950249    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a411c79a-4777-4a0e-a335-572999714887-cni-binary-copy\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950295    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950317    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-run\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950334    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-var-lib-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950365    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-ovn\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950381    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-os-release\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950389    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/461355bd-6507-41f6-a7cf-0c62360f9a90-konnectivity-ca\") pod \"konnectivity-agent-66b65\" (UID: \"461355bd-6507-41f6-a7cf-0c62360f9a90\") " pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950415    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-czpf9\" (UniqueName: \"kubernetes.io/projected/d60aee99-f527-46fb-8992-2945e473f6d4-kube-api-access-czpf9\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950424    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-systemd-units\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950440    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxl5v\" (UniqueName: \"kubernetes.io/projected/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-kube-api-access-lxl5v\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950465    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-etc-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950467    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-k8s-cni-cncf-io\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950511    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-system-cni-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950546    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-var-lib-kubelet\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952271 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950549    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-k8s-cni-cncf-io\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950612    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-registration-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950653    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950809    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-modprobe-d\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950903    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-lib-modules\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.949826    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-cni-bin\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.950964    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-cnibin\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951103    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-var-lib-openvswitch\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951190    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovnkube-script-lib\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951200    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d60aee99-f527-46fb-8992-2945e473f6d4-host\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951251    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-ovn\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951268    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-systemd\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951425    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec80ae08-6054-4495-a85d-c88fc5f066a8-cni-binary-copy\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951483    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-host-run-netns\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951591    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-socket-dir\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951639    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a411c79a-4777-4a0e-a335-572999714887-system-cni-dir\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951682    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-run-systemd\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.952945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-host\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951790    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-hostroot\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951803    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/461355bd-6507-41f6-a7cf-0c62360f9a90-konnectivity-ca\") pod \"konnectivity-agent-66b65\" (UID: \"461355bd-6507-41f6-a7cf-0c62360f9a90\") " pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951831    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/560b50bf-edb4-4b38-950c-a898eb7bc4b9-host-slash\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951878    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec80ae08-6054-4495-a85d-c88fc5f066a8-multus-cni-dir\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.951898    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-etc-selinux\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.952226    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-tmp\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.952334    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/560b50bf-edb4-4b38-950c-a898eb7bc4b9-ovn-node-metrics-cert\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.953467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.952788    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/461355bd-6507-41f6-a7cf-0c62360f9a90-agent-certs\") pod \"konnectivity-agent-66b65\" (UID: \"461355bd-6507-41f6-a7cf-0c62360f9a90\") " pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:56.953793 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.953708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-etc-tuned\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.955163 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.955140    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 15:41:55 +0000 UTC" deadline="2027-11-11 18:30:20.153273586 +0000 UTC"
Jun 02 15:46:56.955163 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.955162    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12650h43m23.198114642s"
Jun 02 15:46:56.961901 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.961871    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czpf9\" (UniqueName: \"kubernetes.io/projected/d60aee99-f527-46fb-8992-2945e473f6d4-kube-api-access-czpf9\") pod \"node-ca-6glkd\" (UID: \"d60aee99-f527-46fb-8992-2945e473f6d4\") " pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:56.962102 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.962084    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st48j\" (UniqueName: \"kubernetes.io/projected/4097c2c0-12e3-478d-8b5f-c0565f205645-kube-api-access-st48j\") pod \"iptables-alerter-2jm4l\" (UID: \"4097c2c0-12e3-478d-8b5f-c0565f205645\") " pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:56.962605 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.962423    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:56.962605 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.962446    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:56.962605 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.962460    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:56.962605 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:56.962516    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:46:57.46249779 +0000 UTC m=+2.086286632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:56.962605 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.962560    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m5n4\" (UniqueName: \"kubernetes.io/projected/1992ad1c-cbee-4529-a8a8-b05b61961292-kube-api-access-9m5n4\") pod \"node-resolver-2ghc2\" (UID: \"1992ad1c-cbee-4529-a8a8-b05b61961292\") " pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:56.962836 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.962662    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdpsx\" (UniqueName: \"kubernetes.io/projected/d7d593cb-a89b-40ad-99d9-b5ceb3c1991f-kube-api-access-fdpsx\") pod \"aws-ebs-csi-driver-node-swkm2\" (UID: \"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:56.963698 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.963661    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22hhn\" (UniqueName: \"kubernetes.io/projected/ec80ae08-6054-4495-a85d-c88fc5f066a8-kube-api-access-22hhn\") pod \"multus-vqrnf\" (UID: \"ec80ae08-6054-4495-a85d-c88fc5f066a8\") " pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:56.963958 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.963928    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp85n\" (UniqueName: \"kubernetes.io/projected/a411c79a-4777-4a0e-a335-572999714887-kube-api-access-hp85n\") pod \"multus-additional-cni-plugins-jvn72\" (UID: \"a411c79a-4777-4a0e-a335-572999714887\") " pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:56.964174 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.964155    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxl5v\" (UniqueName: \"kubernetes.io/projected/14e1649a-8b6e-4b9d-8fd0-a95f14cfa822-kube-api-access-lxl5v\") pod \"tuned-qvdd2\" (UID: \"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822\") " pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:56.964501 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.964481    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrvt\" (UniqueName: \"kubernetes.io/projected/560b50bf-edb4-4b38-950c-a898eb7bc4b9-kube-api-access-6wrvt\") pod \"ovnkube-node-9p9h9\" (UID: \"560b50bf-edb4-4b38-950c-a898eb7bc4b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:56.964867 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.964851    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhnjv\" (UniqueName: \"kubernetes.io/projected/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-kube-api-access-bhnjv\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:56.979675 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.979631    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal" event={"ID":"e230094760b42b9068c20c1fe959349e","Type":"ContainerStarted","Data":"fcc4016368e8c8fbd0a0eb00781beab764ac14c8ee624f645fa73022d1894389"}
Jun 02 15:46:56.980491 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:56.980468    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal" event={"ID":"2cc8efb4d7faafb9447182ed8c6f6fdb","Type":"ContainerStarted","Data":"3bab80b4c4098560ea96ae3666cbd7bbd5b9b1d8d5916543033b94f75ca03ff7"}
Jun 02 15:46:57.100119 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.100033    2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:57.153945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.153918    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qvdd2"
Jun 02 15:46:57.160948 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.160923    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e1649a_8b6e_4b9d_8fd0_a95f14cfa822.slice/crio-58332424654f8da8bfcd3b9d23434132f2676e07cc4da786353df052bcfb2c8f WatchSource:0}: Error finding container 58332424654f8da8bfcd3b9d23434132f2676e07cc4da786353df052bcfb2c8f: Status 404 returned error can't find the container with id 58332424654f8da8bfcd3b9d23434132f2676e07cc4da786353df052bcfb2c8f
Jun 02 15:46:57.174148 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.174125    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vqrnf"
Jun 02 15:46:57.180280 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.180257    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec80ae08_6054_4495_a85d_c88fc5f066a8.slice/crio-a4fac1f6003f9e3c91df9c6a6273e982f41361096e12c10ae488e809738985f4 WatchSource:0}: Error finding container a4fac1f6003f9e3c91df9c6a6273e982f41361096e12c10ae488e809738985f4: Status 404 returned error can't find the container with id a4fac1f6003f9e3c91df9c6a6273e982f41361096e12c10ae488e809738985f4
Jun 02 15:46:57.191309 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.191291    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2jm4l"
Jun 02 15:46:57.195915 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.195896    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:46:57.197308 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.197287    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4097c2c0_12e3_478d_8b5f_c0565f205645.slice/crio-30ce8d11a723d7a5f7926cbe9d24c52a4299c0b69aa72064cc966dc17a585f25 WatchSource:0}: Error finding container 30ce8d11a723d7a5f7926cbe9d24c52a4299c0b69aa72064cc966dc17a585f25: Status 404 returned error can't find the container with id 30ce8d11a723d7a5f7926cbe9d24c52a4299c0b69aa72064cc966dc17a585f25
Jun 02 15:46:57.201883 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.201863    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560b50bf_edb4_4b38_950c_a898eb7bc4b9.slice/crio-98673244a2dc2a9d4533e3cc83c2226cb06c15a8a3c76c547551d83e39802e31 WatchSource:0}: Error finding container 98673244a2dc2a9d4533e3cc83c2226cb06c15a8a3c76c547551d83e39802e31: Status 404 returned error can't find the container with id 98673244a2dc2a9d4533e3cc83c2226cb06c15a8a3c76c547551d83e39802e31
Jun 02 15:46:57.212948 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.212927    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:46:57.220847 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.220825    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod461355bd_6507_41f6_a7cf_0c62360f9a90.slice/crio-297766042d770d68aaef34ba51767de5a84c91f076cd8eda9ddbe36cbd9e6e1d WatchSource:0}: Error finding container 297766042d770d68aaef34ba51767de5a84c91f076cd8eda9ddbe36cbd9e6e1d: Status 404 returned error can't find the container with id 297766042d770d68aaef34ba51767de5a84c91f076cd8eda9ddbe36cbd9e6e1d
Jun 02 15:46:57.237938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.237906    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2"
Jun 02 15:46:57.241673 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.241653    2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:57.244784 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.244759    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d593cb_a89b_40ad_99d9_b5ceb3c1991f.slice/crio-6f766197e789a1b13c4e3c0f2534246ea53f196bfe2637efbf1ef637a5a396e5 WatchSource:0}: Error finding container 6f766197e789a1b13c4e3c0f2534246ea53f196bfe2637efbf1ef637a5a396e5: Status 404 returned error can't find the container with id 6f766197e789a1b13c4e3c0f2534246ea53f196bfe2637efbf1ef637a5a396e5
Jun 02 15:46:57.246544 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.246528    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2ghc2"
Jun 02 15:46:57.252448 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.252427    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1992ad1c_cbee_4529_a8a8_b05b61961292.slice/crio-84a910a7d76c1b054a8c1ab35144efb88dc574c2beb5fef9f2021cc7d75ddd0f WatchSource:0}: Error finding container 84a910a7d76c1b054a8c1ab35144efb88dc574c2beb5fef9f2021cc7d75ddd0f: Status 404 returned error can't find the container with id 84a910a7d76c1b054a8c1ab35144efb88dc574c2beb5fef9f2021cc7d75ddd0f
Jun 02 15:46:57.257355 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.257337    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6glkd"
Jun 02 15:46:57.262765 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.262745    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jvn72"
Jun 02 15:46:57.263335 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.263318    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60aee99_f527_46fb_8992_2945e473f6d4.slice/crio-e3e85884c196a722573ba2f0f46343a2c356b9964b3a7dfcf51d2d41bb1bf269 WatchSource:0}: Error finding container e3e85884c196a722573ba2f0f46343a2c356b9964b3a7dfcf51d2d41bb1bf269: Status 404 returned error can't find the container with id e3e85884c196a722573ba2f0f46343a2c356b9964b3a7dfcf51d2d41bb1bf269
Jun 02 15:46:57.269058 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:46:57.269036    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda411c79a_4777_4a0e_a335_572999714887.slice/crio-e44fed2ee1ef64bb8ecb3642b2662c3c591b557cddd8f7121e14e16c2a21de9c WatchSource:0}: Error finding container e44fed2ee1ef64bb8ecb3642b2662c3c591b557cddd8f7121e14e16c2a21de9c: Status 404 returned error can't find the container with id e44fed2ee1ef64bb8ecb3642b2662c3c591b557cddd8f7121e14e16c2a21de9c
Jun 02 15:46:57.453515 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.453430    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:57.453681 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:57.453576    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:57.453681 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:57.453645    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:46:58.453625145 +0000 UTC m=+3.077413984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:57.554357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.554316    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:46:57.554536 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:57.554450    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:57.554536 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:57.554476    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:57.554536 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:57.554489    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:57.554750 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:57.554550    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:46:58.554531073 +0000 UTC m=+3.178319899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:57.594508 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.594475    2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:46:57.911250 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.911099    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:41:56 +0000 UTC" deadline="2028-02-20 15:43:49.847129342 +0000 UTC"
Jun 02 15:46:57.911250 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:57.911135    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15071h56m51.935997972s"
Jun 02 15:46:58.011135 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.011064    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vqrnf" event={"ID":"ec80ae08-6054-4495-a85d-c88fc5f066a8","Type":"ContainerStarted","Data":"a4fac1f6003f9e3c91df9c6a6273e982f41361096e12c10ae488e809738985f4"}
Jun 02 15:46:58.012917 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.012887    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2ghc2" event={"ID":"1992ad1c-cbee-4529-a8a8-b05b61961292","Type":"ContainerStarted","Data":"84a910a7d76c1b054a8c1ab35144efb88dc574c2beb5fef9f2021cc7d75ddd0f"}
Jun 02 15:46:58.021187 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.021155    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-66b65" event={"ID":"461355bd-6507-41f6-a7cf-0c62360f9a90","Type":"ContainerStarted","Data":"297766042d770d68aaef34ba51767de5a84c91f076cd8eda9ddbe36cbd9e6e1d"}
Jun 02 15:46:58.040432 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.040378    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2jm4l" event={"ID":"4097c2c0-12e3-478d-8b5f-c0565f205645","Type":"ContainerStarted","Data":"30ce8d11a723d7a5f7926cbe9d24c52a4299c0b69aa72064cc966dc17a585f25"}
Jun 02 15:46:58.052300 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.052260    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qvdd2" event={"ID":"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822","Type":"ContainerStarted","Data":"58332424654f8da8bfcd3b9d23434132f2676e07cc4da786353df052bcfb2c8f"}
Jun 02 15:46:58.089411 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.089365    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerStarted","Data":"e44fed2ee1ef64bb8ecb3642b2662c3c591b557cddd8f7121e14e16c2a21de9c"}
Jun 02 15:46:58.101549 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.101503    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6glkd" event={"ID":"d60aee99-f527-46fb-8992-2945e473f6d4","Type":"ContainerStarted","Data":"e3e85884c196a722573ba2f0f46343a2c356b9964b3a7dfcf51d2d41bb1bf269"}
Jun 02 15:46:58.124312 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.124273    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2" event={"ID":"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f","Type":"ContainerStarted","Data":"6f766197e789a1b13c4e3c0f2534246ea53f196bfe2637efbf1ef637a5a396e5"}
Jun 02 15:46:58.129973 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.129938    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"98673244a2dc2a9d4533e3cc83c2226cb06c15a8a3c76c547551d83e39802e31"}
Jun 02 15:46:58.464507 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.464392    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:58.464678 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.464560    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:58.464678 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.464638    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:47:00.464617526 +0000 UTC m=+5.088406359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:46:58.565364 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.565276    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:46:58.565526 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.565475    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:46:58.565526 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.565494    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:46:58.565526 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.565506    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:58.565680 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.565563    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:47:00.565546132 +0000 UTC m=+5.189334966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:46:58.911907 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.911813    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:41:56 +0000 UTC" deadline="2028-01-08 01:26:55.268094996 +0000 UTC"
Jun 02 15:46:58.911907 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.911854    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14025h39m56.356243643s"
Jun 02 15:46:58.976610 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.976575    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:46:58.976772 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.976720    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:46:58.977137 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:58.977116    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:46:58.977231 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:46:58.977218    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:46:59.335706 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:46:59.335628    2563 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 15:47:00.481729 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:00.481689    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:00.482208 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.481851    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:00.482208 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.481928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.481908161 +0000 UTC m=+9.105697000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:00.582875 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:00.582824    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:00.583073 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.582987    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:47:00.583073 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.583005    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:47:00.583073 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.583039    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:00.583238 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.583099    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.583078386 +0000 UTC m=+9.206867217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:00.976905 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:00.976828    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:00.977091 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.976970    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:00.977091 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:00.977033    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:00.977208 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:00.977156    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:02.976778 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:02.976736    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:02.977274 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:02.976876    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:02.977274 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:02.976977    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:02.977274 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:02.977059    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:04.517308 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:04.516801    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:04.517308 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.516945    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:04.517308 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.517001    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:47:12.516985948 +0000 UTC m=+17.140774777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:04.617754 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:04.617242    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:04.617754 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.617400    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:47:04.617754 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.617417    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:47:04.617754 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.617426    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:04.617754 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.617466    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:47:12.617453892 +0000 UTC m=+17.241242717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:04.976628 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:04.976544    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:04.976797 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.976666    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:04.977054 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:04.977027    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:04.977185 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:04.977115    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:06.977053 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:06.977003    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:06.977448 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:06.977003    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:06.977448 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:06.977145    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:06.977448 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:06.977181    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:08.977084 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:08.977041    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:08.977537 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:08.977069    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:08.977537 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:08.977179    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:08.977537 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:08.977302    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:10.977206 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:10.977171    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:10.977614 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:10.977224    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:10.977614 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:10.977295    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:10.977614 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:10.977402    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:12.576222 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:12.576183    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:12.576826 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.576348    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:12.576826 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.576433    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:47:28.576411833 +0000 UTC m=+33.200200659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:12.677216 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:12.676964    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:12.677387 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.677140    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:47:12.677387 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.677276    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:47:12.677387 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.677289    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:12.677387 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.677344    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:47:28.677326895 +0000 UTC m=+33.301115721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:12.977174 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:12.977081    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:12.977174 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:12.977131    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:12.977364 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.977246    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:12.977440 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:12.977376    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:14.976282 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:14.976241    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:14.976732 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:14.976241    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:14.976732 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:14.976394    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:14.976732 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:14.976445    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:16.976588 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:16.976567    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:16.976897 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:16.976584    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:16.976897 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:16.976671    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:16.976897 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:16.976746    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:17.174844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.174810    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qvdd2" event={"ID":"14e1649a-8b6e-4b9d-8fd0-a95f14cfa822","Type":"ContainerStarted","Data":"baab66a4766610bd12786101c0f9fbd6e7d0f161f0b25ed31230e1ba4f9ddf38"}
Jun 02 15:47:17.176388 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.176308    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal" event={"ID":"2cc8efb4d7faafb9447182ed8c6f6fdb","Type":"ContainerStarted","Data":"6b92f0ad49adb73465539b20095eb8569bd3c30b6be5a48b89c65a4001764b20"}
Jun 02 15:47:17.179385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.179344    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"913d0a9421be0d8e7b557fa4bac3d837d9053f036a7ee8b10de4fc8b05fb9c61"}
Jun 02 15:47:17.179466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.179391    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"4874c538fcce4b08311744c046f292baccea19c04e7833d25d234ae59bb9c668"}
Jun 02 15:47:17.179466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.179407    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"425de937186373c8f4c17f3612073bba88cad5614cc4ab9ee9a87aa73316aa41"}
Jun 02 15:47:17.179466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.179420    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"b5e268a897e11cc1ffc36ea5c562594877a10c5fb47fa0be91b7f273ee7013ff"}
Jun 02 15:47:17.179466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.179431    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"5d95da8cc02203cc13822a2eb1e5b4068ebe75c6c13b0d4b5681e42a58d2ee12"}
Jun 02 15:47:17.180596 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.180572    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vqrnf" event={"ID":"ec80ae08-6054-4495-a85d-c88fc5f066a8","Type":"ContainerStarted","Data":"74c7889a27f01d0688575ace69548b4f71386ceb550d90c5eeacdfadcde1aba5"}
Jun 02 15:47:17.250342 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.250288    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qvdd2" podStartSLOduration=1.976229704 podStartE2EDuration="21.250274433s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.162575498 +0000 UTC m=+1.786364323" lastFinishedPulling="2026-06-02 15:47:16.436620222 +0000 UTC m=+21.060409052" observedRunningTime="2026-06-02 15:47:17.250182052 +0000 UTC m=+21.873970897" watchObservedRunningTime="2026-06-02 15:47:17.250274433 +0000 UTC m=+21.874063278"
Jun 02 15:47:17.281992 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:17.281950    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-193.ec2.internal" podStartSLOduration=21.281937513 podStartE2EDuration="21.281937513s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:17.281681026 +0000 UTC m=+21.905469874" watchObservedRunningTime="2026-06-02 15:47:17.281937513 +0000 UTC m=+21.905726359"
Jun 02 15:47:18.184022 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.183827    2563 generic.go:358] "Generic (PLEG): container finished" podID="e230094760b42b9068c20c1fe959349e" containerID="18e0aedba378dd26844af84cf8087355c5004ce4909703a14c8c0558f14921fe" exitCode=0
Jun 02 15:47:18.184022 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.183904    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal" event={"ID":"e230094760b42b9068c20c1fe959349e","Type":"ContainerDied","Data":"18e0aedba378dd26844af84cf8087355c5004ce4909703a14c8c0558f14921fe"}
Jun 02 15:47:18.185358 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.185339    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2ghc2" event={"ID":"1992ad1c-cbee-4529-a8a8-b05b61961292","Type":"ContainerStarted","Data":"f83b0313c0288fa222ed171fd313f548db976db4c3e5b36370dfbf80e0362908"}
Jun 02 15:47:18.186518 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.186491    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-66b65" event={"ID":"461355bd-6507-41f6-a7cf-0c62360f9a90","Type":"ContainerStarted","Data":"d21a6850af630f21c686c85d62552974b894283d9142beb7da7930c836dcbdd8"}
Jun 02 15:47:18.187700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.187674    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2jm4l" event={"ID":"4097c2c0-12e3-478d-8b5f-c0565f205645","Type":"ContainerStarted","Data":"4382289d378c0d321a5dcbe97b25f21727ccc70fc2bd7c84e80bbe4a7f942032"}
Jun 02 15:47:18.188964 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.188868    2563 generic.go:358] "Generic (PLEG): container finished" podID="a411c79a-4777-4a0e-a335-572999714887" containerID="3b24211ae241c2aadf7707557e1c4be53d921296973bdf9db448071e404c67cb" exitCode=0
Jun 02 15:47:18.188964 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.188943    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerDied","Data":"3b24211ae241c2aadf7707557e1c4be53d921296973bdf9db448071e404c67cb"}
Jun 02 15:47:18.190365 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.190348    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6glkd" event={"ID":"d60aee99-f527-46fb-8992-2945e473f6d4","Type":"ContainerStarted","Data":"700c2358b0d745a25a82f7970f3e9b0fa7b1ac585f3a7be48b119e2fb29657b9"}
Jun 02 15:47:18.191663 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.191633    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2" event={"ID":"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f","Type":"ContainerStarted","Data":"7031daf398794c49c400fe35da1b11cf860de20f89ce8048020d449e7d1f47e9"}
Jun 02 15:47:18.194245 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.194223    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"6cbbed853dcfeafd7e21fb680888dd51e8240fbfc8531772fea95b1a56f45e42"}
Jun 02 15:47:18.211316 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.211272    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-vqrnf" podStartSLOduration=2.956848818 podStartE2EDuration="22.211259805s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.181761524 +0000 UTC m=+1.805550350" lastFinishedPulling="2026-06-02 15:47:16.436172498 +0000 UTC m=+21.059961337" observedRunningTime="2026-06-02 15:47:17.318265534 +0000 UTC m=+21.942054382" watchObservedRunningTime="2026-06-02 15:47:18.211259805 +0000 UTC m=+22.835048712"
Jun 02 15:47:18.274539 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.274498    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2jm4l" podStartSLOduration=3.072178965 podStartE2EDuration="22.274482988s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.199059151 +0000 UTC m=+1.822847976" lastFinishedPulling="2026-06-02 15:47:16.401363173 +0000 UTC m=+21.025151999" observedRunningTime="2026-06-02 15:47:18.242607121 +0000 UTC m=+22.866395967" watchObservedRunningTime="2026-06-02 15:47:18.274482988 +0000 UTC m=+22.898271901"
Jun 02 15:47:18.274746 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.274721    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-66b65" podStartSLOduration=3.116388654 podStartE2EDuration="22.274714408s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.22226926 +0000 UTC m=+1.846058085" lastFinishedPulling="2026-06-02 15:47:16.380594998 +0000 UTC m=+21.004383839" observedRunningTime="2026-06-02 15:47:18.273952113 +0000 UTC m=+22.897740960" watchObservedRunningTime="2026-06-02 15:47:18.274714408 +0000 UTC m=+22.898503258"
Jun 02 15:47:18.298844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.298802    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6glkd" podStartSLOduration=3.213881864 podStartE2EDuration="22.29878898s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.265581567 +0000 UTC m=+1.889370395" lastFinishedPulling="2026-06-02 15:47:16.350488683 +0000 UTC m=+20.974277511" observedRunningTime="2026-06-02 15:47:18.298533279 +0000 UTC m=+22.922322125" watchObservedRunningTime="2026-06-02 15:47:18.29878898 +0000 UTC m=+22.922577826"
Jun 02 15:47:18.373223 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.373167    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-2ghc2" podStartSLOduration=3.190958619 podStartE2EDuration="22.373149977s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.253956757 +0000 UTC m=+1.877745582" lastFinishedPulling="2026-06-02 15:47:16.43614811 +0000 UTC m=+21.059936940" observedRunningTime="2026-06-02 15:47:18.325522728 +0000 UTC m=+22.949311575" watchObservedRunningTime="2026-06-02 15:47:18.373149977 +0000 UTC m=+22.996938873"
Jun 02 15:47:18.593703 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.593677    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 02 15:47:18.912979 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.912821    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T15:47:18.59369982Z","UUID":"35840b96-4934-45c9-bf8c-ee2990d972e7","Handler":null,"Name":"","Endpoint":""}
Jun 02 15:47:18.915865 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.915838    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 02 15:47:18.915865 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.915868    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 02 15:47:18.976274 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.976238    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:18.976446 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:18.976352    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:18.976446 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:18.976238    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:18.976572 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:18.976454    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:19.198857 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:19.198774    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2" event={"ID":"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f","Type":"ContainerStarted","Data":"c7b2da20a44148165921a11d5facc732d04d87aaa3c7e96fbd0588413cbe051f"}
Jun 02 15:47:19.201080 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:19.201049    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal" event={"ID":"e230094760b42b9068c20c1fe959349e","Type":"ContainerStarted","Data":"15a3dc3004801ce7238bd113076976eb8b6b747755aba08669fe7e5f764529d9"}
Jun 02 15:47:19.231431 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:19.231377    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-193.ec2.internal" podStartSLOduration=23.231363359 podStartE2EDuration="23.231363359s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:19.231150555 +0000 UTC m=+23.854939406" watchObservedRunningTime="2026-06-02 15:47:19.231363359 +0000 UTC m=+23.855152205"
Jun 02 15:47:20.205784 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:20.205701    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2" event={"ID":"d7d593cb-a89b-40ad-99d9-b5ceb3c1991f","Type":"ContainerStarted","Data":"4e7e51c02c4b9277bb50dfa3340e8e9d1fe2fd4bd15f788fd8c9fb3730c90593"}
Jun 02 15:47:20.209073 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:20.209045    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"c6e7ababdc56f47d37d758209afa69dfee693d04966568128558e8d653469169"}
Jun 02 15:47:20.976784 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:20.976751    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:20.976967 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:20.976757    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:20.976967 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:20.976887    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:20.976967 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:20.976923    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:22.216470 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.216246    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" event={"ID":"560b50bf-edb4-4b38-950c-a898eb7bc4b9","Type":"ContainerStarted","Data":"d3c984bd76968c5235f5e37274a991920ad4bb07d91d86d0cea7e8e579771551"}
Jun 02 15:47:22.216956 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.216578    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:47:22.216956 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.216603    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:47:22.235894 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.235860    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:47:22.336635 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.336571    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9" podStartSLOduration=6.810683071 podStartE2EDuration="26.336552175s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.203318121 +0000 UTC m=+1.827106947" lastFinishedPulling="2026-06-02 15:47:16.729187227 +0000 UTC m=+21.352976051" observedRunningTime="2026-06-02 15:47:22.336238731 +0000 UTC m=+26.960027585" watchObservedRunningTime="2026-06-02 15:47:22.336552175 +0000 UTC m=+26.960341025"
Jun 02 15:47:22.336862 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.336836    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swkm2" podStartSLOduration=3.9083781650000002 podStartE2EDuration="26.336828375s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.246307066 +0000 UTC m=+1.870095891" lastFinishedPulling="2026-06-02 15:47:19.674757276 +0000 UTC m=+24.298546101" observedRunningTime="2026-06-02 15:47:20.241306203 +0000 UTC m=+24.865095050" watchObservedRunningTime="2026-06-02 15:47:22.336828375 +0000 UTC m=+26.960617221"
Jun 02 15:47:22.787288 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.787249    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:47:22.787887 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.787857    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:47:22.977231 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.977198    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:22.977415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:22.977239    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:22.977415 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:22.977322    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:22.977525 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:22.977444    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:23.219105 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.219035    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:47:23.219105 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.219071    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:47:23.219105 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.219103    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-66b65"
Jun 02 15:47:23.234505 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.234467    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:47:23.811453 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.811217    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5r476"]
Jun 02 15:47:23.811600 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.811492    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:23.811600 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:23.811576    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:23.815529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.815504    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kpzkn"]
Jun 02 15:47:23.815647 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:23.815604    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:23.815697 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:23.815673    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:24.221861 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:24.221767    2563 generic.go:358] "Generic (PLEG): container finished" podID="a411c79a-4777-4a0e-a335-572999714887" containerID="d579545c5118b6f044596326071afdb1fa3e3149beb44ef498f9bd71809b70f4" exitCode=0
Jun 02 15:47:24.222656 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:24.221853    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerDied","Data":"d579545c5118b6f044596326071afdb1fa3e3149beb44ef498f9bd71809b70f4"}
Jun 02 15:47:24.976238 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:24.976207    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:24.976406 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:24.976312    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:25.225400 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:25.225315    2563 generic.go:358] "Generic (PLEG): container finished" podID="a411c79a-4777-4a0e-a335-572999714887" containerID="dc5cee89192ba924054494e1411aab357e93e454c3f70f98544c61a304d993ae" exitCode=0
Jun 02 15:47:25.225758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:25.225395    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerDied","Data":"dc5cee89192ba924054494e1411aab357e93e454c3f70f98544c61a304d993ae"}
Jun 02 15:47:25.979730 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:25.979644    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:25.979872 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:25.979756    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:26.229222 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:26.229192    2563 generic.go:358] "Generic (PLEG): container finished" podID="a411c79a-4777-4a0e-a335-572999714887" containerID="86eb4825096315ff509e7153c0905595d0e89ec9af8d1f533d7a833241757dc8" exitCode=0
Jun 02 15:47:26.229222 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:26.229227    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerDied","Data":"86eb4825096315ff509e7153c0905595d0e89ec9af8d1f533d7a833241757dc8"}
Jun 02 15:47:26.976304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:26.976268    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:26.976490 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:26.976385    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:27.976607 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:27.976575    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:27.977069 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:27.976694    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpzkn" podUID="73906112-b7ef-4e92-865a-6fbf041dd5bf"
Jun 02 15:47:28.592278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:28.592243    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:28.592443 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.592384    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:28.592503 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.592444    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:48:00.592428826 +0000 UTC m=+65.216217651 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 15:47:28.693062 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:28.692999    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:28.693234 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.693160    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 15:47:28.693234 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.693186    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 15:47:28.693234 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.693198    2563 projected.go:194] Error preparing data for projected volume kube-api-access-xp2sz for pod openshift-network-diagnostics/network-check-target-kpzkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:28.693394 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.693255    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz podName:73906112-b7ef-4e92-865a-6fbf041dd5bf nodeName:}" failed. No retries permitted until 2026-06-02 15:48:00.693240674 +0000 UTC m=+65.317029499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xp2sz" (UniqueName: "kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz") pod "network-check-target-kpzkn" (UID: "73906112-b7ef-4e92-865a-6fbf041dd5bf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 15:47:28.977222 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:28.977136    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:28.977646 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:28.977268    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:47:29.224389 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.224361    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-193.ec2.internal" event="NodeReady"
Jun 02 15:47:29.224545 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.224516    2563 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Jun 02 15:47:29.318067 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.317970    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7fbbj"]
Jun 02 15:47:29.343920 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.343886    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-88dbl"]
Jun 02 15:47:29.344120 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.344100    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:29.351588 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.351565    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 02 15:47:29.352145 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.351939    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 02 15:47:29.352440 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.352417    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 02 15:47:29.352440 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.352431    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gg4df\""
Jun 02 15:47:29.356731 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.356710    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7fbbj"]
Jun 02 15:47:29.356849 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.356836    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.358781 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.358760    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-88dbl"]
Jun 02 15:47:29.360318 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.360299    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 02 15:47:29.360431 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.360340    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-25zcr\""
Jun 02 15:47:29.360609 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.360587    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 02 15:47:29.499418 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.499376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14fc5c79-98d8-47a1-8c19-2cb07454b83e-tmp-dir\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.499599 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.499433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:29.499599 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.499533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.499599 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.499565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmswj\" (UniqueName: \"kubernetes.io/projected/14fc5c79-98d8-47a1-8c19-2cb07454b83e-kube-api-access-qmswj\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.499739 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.499661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv2ct\" (UniqueName: \"kubernetes.io/projected/09c32fc9-879b-4dd1-9b1a-6b16d9972168-kube-api-access-vv2ct\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:29.499739 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.499683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14fc5c79-98d8-47a1-8c19-2cb07454b83e-config-volume\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.600338 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600254    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14fc5c79-98d8-47a1-8c19-2cb07454b83e-tmp-dir\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.600338 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600299    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:29.600338 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600336    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600361    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qmswj\" (UniqueName: \"kubernetes.io/projected/14fc5c79-98d8-47a1-8c19-2cb07454b83e-kube-api-access-qmswj\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600400    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vv2ct\" (UniqueName: \"kubernetes.io/projected/09c32fc9-879b-4dd1-9b1a-6b16d9972168-kube-api-access-vv2ct\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600422    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14fc5c79-98d8-47a1-8c19-2cb07454b83e-config-volume\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:29.600424    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:29.600485    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:29.600503    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:30.100486146 +0000 UTC m=+34.724274971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:47:29.600620 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:29.600545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:30.100526973 +0000 UTC m=+34.724315798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:29.600863 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600643    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14fc5c79-98d8-47a1-8c19-2cb07454b83e-tmp-dir\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.600945 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.600929    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14fc5c79-98d8-47a1-8c19-2cb07454b83e-config-volume\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.613164 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.612984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmswj\" (UniqueName: \"kubernetes.io/projected/14fc5c79-98d8-47a1-8c19-2cb07454b83e-kube-api-access-qmswj\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:29.613288 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.612995    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv2ct\" (UniqueName: \"kubernetes.io/projected/09c32fc9-879b-4dd1-9b1a-6b16d9972168-kube-api-access-vv2ct\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:29.976986 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.976901    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:47:29.980336 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.980307    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 15:47:29.981436 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.981411    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m8f5r\""
Jun 02 15:47:29.982529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:29.982502    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 15:47:30.105669 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:30.105631    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:30.105859 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:30.105680    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:30.105859 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:30.105779    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:30.105859 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:30.105809    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:30.106039 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:30.105864    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:31.105840482 +0000 UTC m=+35.729629331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:47:30.106039 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:30.105886    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:31.105876168 +0000 UTC m=+35.729664993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:30.976739 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:30.976701    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:47:30.982837 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:30.982804    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 15:47:30.983279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:30.982927    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j9fj4\""
Jun 02 15:47:31.114762 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:31.114728    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:31.114950 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:31.114792    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:31.114950 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:31.114918    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:31.114950 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:31.114919    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:31.115134 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:31.114982    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:33.114960693 +0000 UTC m=+37.738749542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:31.115134 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:31.114999    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:33.114992275 +0000 UTC m=+37.738781100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:47:32.787992 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.787907    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"]
Jun 02 15:47:32.802688 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.802666    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:32.810175 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.810155    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 02 15:47:32.811398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.811220    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 02 15:47:32.811398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.811267    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jun 02 15:47:32.811398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.811274    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 02 15:47:32.812155 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.812113    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"]
Jun 02 15:47:32.926882 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.926848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-tmp\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:32.927069 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.926907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:32.927069 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:32.926967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpdwm\" (UniqueName: \"kubernetes.io/projected/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-kube-api-access-lpdwm\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.027281 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.027246    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-tmp\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.027436 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.027304    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.027436 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.027392    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lpdwm\" (UniqueName: \"kubernetes.io/projected/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-kube-api-access-lpdwm\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.027700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.027681    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-tmp\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.030429 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.030408    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.037365 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.037338    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpdwm\" (UniqueName: \"kubernetes.io/projected/a08cf0eb-b597-4156-b44f-a89f6d0b43f5-kube-api-access-lpdwm\") pod \"klusterlet-addon-workmgr-7ffd444b74-fjvn8\" (UID: \"a08cf0eb-b597-4156-b44f-a89f6d0b43f5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.113121 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.113081    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:33.128057 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.128003    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:33.128214 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.128088    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:33.128214 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:33.128142    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:33.128311 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:33.128222    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:37.128200998 +0000 UTC m=+41.751989846 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:47:33.128311 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:33.128229    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:33.128311 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:33.128284    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:37.128266414 +0000 UTC m=+41.752055239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:33.246396 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.246203    2563 generic.go:358] "Generic (PLEG): container finished" podID="a411c79a-4777-4a0e-a335-572999714887" containerID="31a2ccf5d8e0b077f937be7e5a4626b93136fe5cbc2da10875c784d303aa6ecb" exitCode=0
Jun 02 15:47:33.246577 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.246276    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerDied","Data":"31a2ccf5d8e0b077f937be7e5a4626b93136fe5cbc2da10875c784d303aa6ecb"}
Jun 02 15:47:33.281180 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:33.279699    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"]
Jun 02 15:47:33.287959 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:47:33.287888    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08cf0eb_b597_4156_b44f_a89f6d0b43f5.slice/crio-39e852d38baf53c26562ce466f0598c95fb3d3fd717d51034ab93def4a6c5e82 WatchSource:0}: Error finding container 39e852d38baf53c26562ce466f0598c95fb3d3fd717d51034ab93def4a6c5e82: Status 404 returned error can't find the container with id 39e852d38baf53c26562ce466f0598c95fb3d3fd717d51034ab93def4a6c5e82
Jun 02 15:47:34.251443 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:34.251403    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8" event={"ID":"a08cf0eb-b597-4156-b44f-a89f6d0b43f5","Type":"ContainerStarted","Data":"39e852d38baf53c26562ce466f0598c95fb3d3fd717d51034ab93def4a6c5e82"}
Jun 02 15:47:34.255894 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:34.255833    2563 generic.go:358] "Generic (PLEG): container finished" podID="a411c79a-4777-4a0e-a335-572999714887" containerID="bc8fde0cf0216cf7378763ea3c3cfd5f3407c8b1238558ec48d89b26d4340258" exitCode=0
Jun 02 15:47:34.256059 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:34.255907    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerDied","Data":"bc8fde0cf0216cf7378763ea3c3cfd5f3407c8b1238558ec48d89b26d4340258"}
Jun 02 15:47:35.261237 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:35.261199    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jvn72" event={"ID":"a411c79a-4777-4a0e-a335-572999714887","Type":"ContainerStarted","Data":"cf2e82e98c123d651de290badf7052d47238341ea05344090d04d9f33d0eb27d"}
Jun 02 15:47:35.294500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:35.294445    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jvn72" podStartSLOduration=4.404662415 podStartE2EDuration="39.294427209s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.270534092 +0000 UTC m=+1.894322918" lastFinishedPulling="2026-06-02 15:47:32.160298885 +0000 UTC m=+36.784087712" observedRunningTime="2026-06-02 15:47:35.294057627 +0000 UTC m=+39.917846472" watchObservedRunningTime="2026-06-02 15:47:35.294427209 +0000 UTC m=+39.918216059"
Jun 02 15:47:37.159848 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:37.159811    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:37.159848 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:37.159856    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:37.160323 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:37.159949    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:37.160323 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:37.159951    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:37.160323 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:37.159999    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:45.159986937 +0000 UTC m=+49.783775762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:37.160323 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:37.160033    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:45.16002477 +0000 UTC m=+49.783813608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:47:38.270259 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:38.269082    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8" event={"ID":"a08cf0eb-b597-4156-b44f-a89f6d0b43f5","Type":"ContainerStarted","Data":"08754197ac0436715cbbfe9bb9370ca07ce2dd5108de99a80d5b1ef6403c435c"}
Jun 02 15:47:38.270782 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:38.270760    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:38.273143 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:38.273116    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:47:38.289746 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:38.289701    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8" podStartSLOduration=2.004306113 podStartE2EDuration="6.289689851s" podCreationTimestamp="2026-06-02 15:47:32 +0000 UTC" firstStartedPulling="2026-06-02 15:47:33.290296396 +0000 UTC m=+37.914085221" lastFinishedPulling="2026-06-02 15:47:37.575680127 +0000 UTC m=+42.199468959" observedRunningTime="2026-06-02 15:47:38.28891297 +0000 UTC m=+42.912701828" watchObservedRunningTime="2026-06-02 15:47:38.289689851 +0000 UTC m=+42.913478697"
Jun 02 15:47:45.214795 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:45.214755    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:47:45.215267 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:45.214805    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:47:45.215267 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:45.214896    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:47:45.215267 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:45.214959    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:48:01.214945249 +0000 UTC m=+65.838734073 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:47:45.215267 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:45.214898    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:47:45.215267 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:47:45.215041    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:48:01.215029853 +0000 UTC m=+65.838818678 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:47:55.236541 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:47:55.236513    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p9h9"
Jun 02 15:48:00.626400 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.626358    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:48:00.629999 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.629974    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 15:48:00.636738 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:00.636720    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 15:48:00.636801 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:00.636791    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:49:04.636774784 +0000 UTC m=+129.260563609 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : secret "metrics-daemon-secret" not found
Jun 02 15:48:00.727676 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.727638    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:48:00.731404 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.731386    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 15:48:00.741681 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.741656    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 15:48:00.752302 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.752265    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp2sz\" (UniqueName: \"kubernetes.io/projected/73906112-b7ef-4e92-865a-6fbf041dd5bf-kube-api-access-xp2sz\") pod \"network-check-target-kpzkn\" (UID: \"73906112-b7ef-4e92-865a-6fbf041dd5bf\") " pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:48:00.893038 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.892939    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m8f5r\""
Jun 02 15:48:00.898906 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:00.898886    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:48:01.046513 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:01.046482    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kpzkn"]
Jun 02 15:48:01.049911 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:48:01.049884    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73906112_b7ef_4e92_865a_6fbf041dd5bf.slice/crio-afc8dfbd743e839831650f7bbc9267933b89dbcfa31d3b99222d3755cd992b5b WatchSource:0}: Error finding container afc8dfbd743e839831650f7bbc9267933b89dbcfa31d3b99222d3755cd992b5b: Status 404 returned error can't find the container with id afc8dfbd743e839831650f7bbc9267933b89dbcfa31d3b99222d3755cd992b5b
Jun 02 15:48:01.231697 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:01.231617    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:48:01.231697 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:01.231670    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:48:01.231867 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:01.231766    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:48:01.231867 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:01.231832    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:48:33.231817849 +0000 UTC m=+97.855606675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:48:01.231867 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:01.231772    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:48:01.231967 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:01.231908    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:48:33.231895749 +0000 UTC m=+97.855684575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:48:01.314706 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:01.314672    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kpzkn" event={"ID":"73906112-b7ef-4e92-865a-6fbf041dd5bf","Type":"ContainerStarted","Data":"afc8dfbd743e839831650f7bbc9267933b89dbcfa31d3b99222d3755cd992b5b"}
Jun 02 15:48:04.322551 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:04.322514    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kpzkn" event={"ID":"73906112-b7ef-4e92-865a-6fbf041dd5bf","Type":"ContainerStarted","Data":"069c7df73368da5ec424111ed324a974b265169c7af43514b3b1b2d001b214ab"}
Jun 02 15:48:04.322917 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:04.322638    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:48:04.349919 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:04.349873    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kpzkn" podStartSLOduration=65.403132064 podStartE2EDuration="1m8.349859462s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:48:01.052343002 +0000 UTC m=+65.676131827" lastFinishedPulling="2026-06-02 15:48:03.999070398 +0000 UTC m=+68.622859225" observedRunningTime="2026-06-02 15:48:04.34744212 +0000 UTC m=+68.971230967" watchObservedRunningTime="2026-06-02 15:48:04.349859462 +0000 UTC m=+68.973648308"
Jun 02 15:48:33.244905 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:33.244871    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:48:33.245361 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:33.244953    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:48:33.245361 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:33.245038    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 15:48:33.245361 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:33.245068    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 15:48:33.245361 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:33.245107    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls podName:14fc5c79-98d8-47a1-8c19-2cb07454b83e nodeName:}" failed. No retries permitted until 2026-06-02 15:49:37.245087216 +0000 UTC m=+161.868876058 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls") pod "dns-default-88dbl" (UID: "14fc5c79-98d8-47a1-8c19-2cb07454b83e") : secret "dns-default-metrics-tls" not found
Jun 02 15:48:33.245361 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:33.245123    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert podName:09c32fc9-879b-4dd1-9b1a-6b16d9972168 nodeName:}" failed. No retries permitted until 2026-06-02 15:49:37.2451164 +0000 UTC m=+161.868905224 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert") pod "ingress-canary-7fbbj" (UID: "09c32fc9-879b-4dd1-9b1a-6b16d9972168") : secret "canary-serving-cert" not found
Jun 02 15:48:35.327054 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:35.327006    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kpzkn"
Jun 02 15:48:49.957972 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.957938    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"]
Jun 02 15:48:49.959808 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.959783    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"]
Jun 02 15:48:49.959938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.959919    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"
Jun 02 15:48:49.961398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.961378    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:49.963489 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.963473    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jun 02 15:48:49.964602 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.964584    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jun 02 15:48:49.964692 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.964589    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4wtx5\""
Jun 02 15:48:49.969752 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.969723    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jun 02 15:48:49.969752 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.969733    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 02 15:48:49.969902 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.969879    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 02 15:48:49.971383 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.971341    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xsp2g\""
Jun 02 15:48:49.971466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.971410    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jun 02 15:48:49.980659 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.980637    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"]
Jun 02 15:48:49.981743 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:49.981714    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"]
Jun 02 15:48:50.053301 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.053270    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c99f78fdd-9bq29"]
Jun 02 15:48:50.055052 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.055029    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.057551 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.057527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmx7\" (UniqueName: \"kubernetes.io/projected/c673ec07-8d69-4136-8f96-eefd17d29c83-kube-api-access-lzmx7\") pod \"volume-data-source-validator-7598fd5bd-gs7nr\" (UID: \"c673ec07-8d69-4136-8f96-eefd17d29c83\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"
Jun 02 15:48:50.057653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.057582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e07e5347-9398-4ad3-9d41-e7948879b7b4-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.057653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.057610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rnpv\" (UniqueName: \"kubernetes.io/projected/e07e5347-9398-4ad3-9d41-e7948879b7b4-kube-api-access-8rnpv\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.057653 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.057644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.059263 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.059228    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jun 02 15:48:50.059445 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.059420    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jun 02 15:48:50.059566 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.059546    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kkg82\""
Jun 02 15:48:50.060124 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.060103    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jun 02 15:48:50.065001 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.064985    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jun 02 15:48:50.073644 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.073620    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c99f78fdd-9bq29"]
Jun 02 15:48:50.158561 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-image-registry-private-configuration\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.158561 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158574    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmx7\" (UniqueName: \"kubernetes.io/projected/c673ec07-8d69-4136-8f96-eefd17d29c83-kube-api-access-lzmx7\") pod \"volume-data-source-validator-7598fd5bd-gs7nr\" (UID: \"c673ec07-8d69-4136-8f96-eefd17d29c83\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"
Jun 02 15:48:50.158804 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158647    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e07e5347-9398-4ad3-9d41-e7948879b7b4-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.158804 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158679    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rnpv\" (UniqueName: \"kubernetes.io/projected/e07e5347-9398-4ad3-9d41-e7948879b7b4-kube-api-access-8rnpv\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.158804 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-certificates\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.158921 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-ca-trust-extracted\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.158921 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkqtn\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-kube-api-access-gkqtn\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.158996 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158942    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.158996 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-trusted-ca\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.158996 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.158990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-installation-pull-secrets\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.159141 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.159031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.159141 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.159041    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:50.159141 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.159064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-bound-sa-token\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.159141 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.159096    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls podName:e07e5347-9398-4ad3-9d41-e7948879b7b4 nodeName:}" failed. No retries permitted until 2026-06-02 15:48:50.659080361 +0000 UTC m=+115.282869186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-t5n2c" (UID: "e07e5347-9398-4ad3-9d41-e7948879b7b4") : secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:50.159362 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.159344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e07e5347-9398-4ad3-9d41-e7948879b7b4-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.171511 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.171482    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmx7\" (UniqueName: \"kubernetes.io/projected/c673ec07-8d69-4136-8f96-eefd17d29c83-kube-api-access-lzmx7\") pod \"volume-data-source-validator-7598fd5bd-gs7nr\" (UID: \"c673ec07-8d69-4136-8f96-eefd17d29c83\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"
Jun 02 15:48:50.175857 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.175833    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rnpv\" (UniqueName: \"kubernetes.io/projected/e07e5347-9398-4ad3-9d41-e7948879b7b4-kube-api-access-8rnpv\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.260036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.259919    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gkqtn\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-kube-api-access-gkqtn\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.259979    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-trusted-ca\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.259998    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-installation-pull-secrets\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260036 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260040    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260059    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-bound-sa-token\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260101    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-image-registry-private-configuration\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260167    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-certificates\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.260185    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.260209    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c99f78fdd-9bq29: secret "image-registry-tls" not found
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260210    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-ca-trust-extracted\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260352 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.260291    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls podName:774ff4f5-76fe-455f-9bb9-5c40c2d217ac nodeName:}" failed. No retries permitted until 2026-06-02 15:48:50.760269865 +0000 UTC m=+115.384058709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls") pod "image-registry-5c99f78fdd-9bq29" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac") : secret "image-registry-tls" not found
Jun 02 15:48:50.260689 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260601    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-ca-trust-extracted\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.260739 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.260701    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-certificates\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.261149 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.261129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-trusted-ca\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.262929 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.262906    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-installation-pull-secrets\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.263052 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.263033    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-image-registry-private-configuration\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.269199 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.269175    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"
Jun 02 15:48:50.270315 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.270175    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-bound-sa-token\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.271155 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.271129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkqtn\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-kube-api-access-gkqtn\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.394176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.394143    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr"]
Jun 02 15:48:50.397237 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:48:50.397207    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc673ec07_8d69_4136_8f96_eefd17d29c83.slice/crio-196c9bad9f851ab71c3f20aab729b32c9ebabeaf5c75b3b2ec83887279fe74e5 WatchSource:0}: Error finding container 196c9bad9f851ab71c3f20aab729b32c9ebabeaf5c75b3b2ec83887279fe74e5: Status 404 returned error can't find the container with id 196c9bad9f851ab71c3f20aab729b32c9ebabeaf5c75b3b2ec83887279fe74e5
Jun 02 15:48:50.410363 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.410336    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr" event={"ID":"c673ec07-8d69-4136-8f96-eefd17d29c83","Type":"ContainerStarted","Data":"196c9bad9f851ab71c3f20aab729b32c9ebabeaf5c75b3b2ec83887279fe74e5"}
Jun 02 15:48:50.663675 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.663637    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:50.663885 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.663815    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:50.663960 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.663899    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls podName:e07e5347-9398-4ad3-9d41-e7948879b7b4 nodeName:}" failed. No retries permitted until 2026-06-02 15:48:51.663879034 +0000 UTC m=+116.287667865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-t5n2c" (UID: "e07e5347-9398-4ad3-9d41-e7948879b7b4") : secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:50.764789 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:50.764743    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:50.764969 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.764923    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:48:50.764969 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.764946    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c99f78fdd-9bq29: secret "image-registry-tls" not found
Jun 02 15:48:50.765085 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:50.765022    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls podName:774ff4f5-76fe-455f-9bb9-5c40c2d217ac nodeName:}" failed. No retries permitted until 2026-06-02 15:48:51.764988799 +0000 UTC m=+116.388777625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls") pod "image-registry-5c99f78fdd-9bq29" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac") : secret "image-registry-tls" not found
Jun 02 15:48:51.672563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:51.672462    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:51.672925 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:51.672611    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:51.672925 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:51.672676    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls podName:e07e5347-9398-4ad3-9d41-e7948879b7b4 nodeName:}" failed. No retries permitted until 2026-06-02 15:48:53.672660687 +0000 UTC m=+118.296449511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-t5n2c" (UID: "e07e5347-9398-4ad3-9d41-e7948879b7b4") : secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:51.773745 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:51.773694    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:51.773898 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:51.773839    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:48:51.773898 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:51.773854    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c99f78fdd-9bq29: secret "image-registry-tls" not found
Jun 02 15:48:51.773962 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:51.773906    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls podName:774ff4f5-76fe-455f-9bb9-5c40c2d217ac nodeName:}" failed. No retries permitted until 2026-06-02 15:48:53.773889298 +0000 UTC m=+118.397678130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls") pod "image-registry-5c99f78fdd-9bq29" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac") : secret "image-registry-tls" not found
Jun 02 15:48:52.416809 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:52.416775    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr" event={"ID":"c673ec07-8d69-4136-8f96-eefd17d29c83","Type":"ContainerStarted","Data":"b38214c43cfed791fe6503d5997e8399b950652431944aa651878896bf99a2a3"}
Jun 02 15:48:52.437449 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:52.437400    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-gs7nr" podStartSLOduration=2.462959144 podStartE2EDuration="3.437385341s" podCreationTimestamp="2026-06-02 15:48:49 +0000 UTC" firstStartedPulling="2026-06-02 15:48:50.39908531 +0000 UTC m=+115.022874134" lastFinishedPulling="2026-06-02 15:48:51.373511504 +0000 UTC m=+115.997300331" observedRunningTime="2026-06-02 15:48:52.436856315 +0000 UTC m=+117.060645166" watchObservedRunningTime="2026-06-02 15:48:52.437385341 +0000 UTC m=+117.061174185"
Jun 02 15:48:53.687951 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:53.687917    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:53.688363 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:53.688104    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:53.688363 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:53.688172    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls podName:e07e5347-9398-4ad3-9d41-e7948879b7b4 nodeName:}" failed. No retries permitted until 2026-06-02 15:48:57.688156924 +0000 UTC m=+122.311945748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-t5n2c" (UID: "e07e5347-9398-4ad3-9d41-e7948879b7b4") : secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:53.788318 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:53.788280    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:53.788456 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:53.788436    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:48:53.788513 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:53.788459    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c99f78fdd-9bq29: secret "image-registry-tls" not found
Jun 02 15:48:53.788552 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:53.788517    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls podName:774ff4f5-76fe-455f-9bb9-5c40c2d217ac nodeName:}" failed. No retries permitted until 2026-06-02 15:48:57.788498664 +0000 UTC m=+122.412287492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls") pod "image-registry-5c99f78fdd-9bq29" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac") : secret "image-registry-tls" not found
Jun 02 15:48:54.805557 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.805523    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"]
Jun 02 15:48:54.807453 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.807434    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"
Jun 02 15:48:54.810284 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.810264    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jun 02 15:48:54.811352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.811334    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b8qkx\""
Jun 02 15:48:54.811441 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.811338    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jun 02 15:48:54.826259 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.826235    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"]
Jun 02 15:48:54.896112 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.896072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtgpk\" (UniqueName: \"kubernetes.io/projected/b74f94ba-6a90-4d66-8c84-b71e9f1f2644-kube-api-access-wtgpk\") pod \"migrator-96c6f749-8jslt\" (UID: \"b74f94ba-6a90-4d66-8c84-b71e9f1f2644\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"
Jun 02 15:48:54.996903 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:54.996856    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wtgpk\" (UniqueName: \"kubernetes.io/projected/b74f94ba-6a90-4d66-8c84-b71e9f1f2644-kube-api-access-wtgpk\") pod \"migrator-96c6f749-8jslt\" (UID: \"b74f94ba-6a90-4d66-8c84-b71e9f1f2644\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"
Jun 02 15:48:55.006890 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:55.006864    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtgpk\" (UniqueName: \"kubernetes.io/projected/b74f94ba-6a90-4d66-8c84-b71e9f1f2644-kube-api-access-wtgpk\") pod \"migrator-96c6f749-8jslt\" (UID: \"b74f94ba-6a90-4d66-8c84-b71e9f1f2644\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"
Jun 02 15:48:55.116309 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:55.116212    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"
Jun 02 15:48:55.238890 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:55.238861    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt"]
Jun 02 15:48:55.241796 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:48:55.241769    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74f94ba_6a90_4d66_8c84_b71e9f1f2644.slice/crio-b850a9d7b9ba78219b57d294c475e66ba895391f3e8814308e18b19a470ad72a WatchSource:0}: Error finding container b850a9d7b9ba78219b57d294c475e66ba895391f3e8814308e18b19a470ad72a: Status 404 returned error can't find the container with id b850a9d7b9ba78219b57d294c475e66ba895391f3e8814308e18b19a470ad72a
Jun 02 15:48:55.422542 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:55.422457    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt" event={"ID":"b74f94ba-6a90-4d66-8c84-b71e9f1f2644","Type":"ContainerStarted","Data":"b850a9d7b9ba78219b57d294c475e66ba895391f3e8814308e18b19a470ad72a"}
Jun 02 15:48:56.425965 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:56.425877    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt" event={"ID":"b74f94ba-6a90-4d66-8c84-b71e9f1f2644","Type":"ContainerStarted","Data":"51f45a60faaf3c7eedcae7422e23beb58bc542507eef6236a016784e8de2186b"}
Jun 02 15:48:56.425965 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:56.425918    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt" event={"ID":"b74f94ba-6a90-4d66-8c84-b71e9f1f2644","Type":"ContainerStarted","Data":"aa76d4540e242220c6d7fedace2ba66c5500c7e70d67e569d7b39a46bbdf588a"}
Jun 02 15:48:56.447382 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:56.447325    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-8jslt" podStartSLOduration=1.627255941 podStartE2EDuration="2.44730836s" podCreationTimestamp="2026-06-02 15:48:54 +0000 UTC" firstStartedPulling="2026-06-02 15:48:55.243565281 +0000 UTC m=+119.867354118" lastFinishedPulling="2026-06-02 15:48:56.063617711 +0000 UTC m=+120.687406537" observedRunningTime="2026-06-02 15:48:56.446141551 +0000 UTC m=+121.069930392" watchObservedRunningTime="2026-06-02 15:48:56.44730836 +0000 UTC m=+121.071097211"
Jun 02 15:48:57.553824 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:57.553793    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2ghc2_1992ad1c-cbee-4529-a8a8-b05b61961292/dns-node-resolver/0.log"
Jun 02 15:48:57.719461 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:57.719431    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:48:57.719641 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:57.719595    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:57.719707 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:57.719678    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls podName:e07e5347-9398-4ad3-9d41-e7948879b7b4 nodeName:}" failed. No retries permitted until 2026-06-02 15:49:05.719656288 +0000 UTC m=+130.343445115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-t5n2c" (UID: "e07e5347-9398-4ad3-9d41-e7948879b7b4") : secret "cluster-monitoring-operator-tls" not found
Jun 02 15:48:57.820105 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:57.819992    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:48:57.820260 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:57.820130    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 15:48:57.820260 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:57.820149    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c99f78fdd-9bq29: secret "image-registry-tls" not found
Jun 02 15:48:57.820260 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:48:57.820202    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls podName:774ff4f5-76fe-455f-9bb9-5c40c2d217ac nodeName:}" failed. No retries permitted until 2026-06-02 15:49:05.820187264 +0000 UTC m=+130.443976088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls") pod "image-registry-5c99f78fdd-9bq29" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac") : secret "image-registry-tls" not found
Jun 02 15:48:58.555291 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:58.555266    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6glkd_d60aee99-f527-46fb-8992-2945e473f6d4/node-ca/0.log"
Jun 02 15:48:59.953359 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:48:59.953329    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-8jslt_b74f94ba-6a90-4d66-8c84-b71e9f1f2644/migrator/0.log"
Jun 02 15:49:00.154026 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:00.153977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-8jslt_b74f94ba-6a90-4d66-8c84-b71e9f1f2644/graceful-termination/0.log"
Jun 02 15:49:04.672502 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:04.672464    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:49:04.673102 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:04.672603    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 15:49:04.673102 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:04.672707    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs podName:aca2f9e4-d1e9-470c-b700-d6cc6f3be66c nodeName:}" failed. No retries permitted until 2026-06-02 15:51:06.672686723 +0000 UTC m=+251.296475549 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs") pod "network-metrics-daemon-5r476" (UID: "aca2f9e4-d1e9-470c-b700-d6cc6f3be66c") : secret "metrics-daemon-secret" not found
Jun 02 15:49:05.778458 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:05.778414    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:49:05.778945 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:05.778593    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 02 15:49:05.778945 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:05.778697    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls podName:e07e5347-9398-4ad3-9d41-e7948879b7b4 nodeName:}" failed. No retries permitted until 2026-06-02 15:49:21.778674922 +0000 UTC m=+146.402463748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-t5n2c" (UID: "e07e5347-9398-4ad3-9d41-e7948879b7b4") : secret "cluster-monitoring-operator-tls" not found
Jun 02 15:49:05.879797 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:05.879762    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:49:05.882469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:05.882434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"image-registry-5c99f78fdd-9bq29\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") " pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:49:05.968542 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:05.968509    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kkg82\""
Jun 02 15:49:05.975572 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:05.975551    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:49:06.108029 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:06.107983    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c99f78fdd-9bq29"]
Jun 02 15:49:06.111400 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:06.111372    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod774ff4f5_76fe_455f_9bb9_5c40c2d217ac.slice/crio-a86c93e2e446edc37e8eadc2ba562521ba2eee1896ae73dc184b7b79e4950ade WatchSource:0}: Error finding container a86c93e2e446edc37e8eadc2ba562521ba2eee1896ae73dc184b7b79e4950ade: Status 404 returned error can't find the container with id a86c93e2e446edc37e8eadc2ba562521ba2eee1896ae73dc184b7b79e4950ade
Jun 02 15:49:06.448441 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:06.448341    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" event={"ID":"774ff4f5-76fe-455f-9bb9-5c40c2d217ac","Type":"ContainerStarted","Data":"26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08"}
Jun 02 15:49:06.448441 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:06.448388    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" event={"ID":"774ff4f5-76fe-455f-9bb9-5c40c2d217ac","Type":"ContainerStarted","Data":"a86c93e2e446edc37e8eadc2ba562521ba2eee1896ae73dc184b7b79e4950ade"}
Jun 02 15:49:06.448666 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:06.448499    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:49:06.472981 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:06.472932    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" podStartSLOduration=16.472918382 podStartE2EDuration="16.472918382s" podCreationTimestamp="2026-06-02 15:48:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:49:06.47207751 +0000 UTC m=+131.095866356" watchObservedRunningTime="2026-06-02 15:49:06.472918382 +0000 UTC m=+131.096707228"
Jun 02 15:49:18.716279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.716233    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d647j"]
Jun 02 15:49:18.721652 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.721631    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.725432 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.725409    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 02 15:49:18.726069 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.726043    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 02 15:49:18.726737 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.726717    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 02 15:49:18.726868 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.726733    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 02 15:49:18.726987 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.726797    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-589bn\""
Jun 02 15:49:18.737641 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.737618    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d647j"]
Jun 02 15:49:18.768715 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.768687    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"]
Jun 02 15:49:18.772052 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.772038    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:18.775578 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.775562    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tpvzz\""
Jun 02 15:49:18.775641 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.775603    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jun 02 15:49:18.775807 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.775793    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jun 02 15:49:18.777629 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.777598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72088158-d1c4-4d2f-be92-bb3f772aa99c-crio-socket\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.777759 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.777646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72088158-d1c4-4d2f-be92-bb3f772aa99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.777759 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.777672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkrp\" (UniqueName: \"kubernetes.io/projected/72088158-d1c4-4d2f-be92-bb3f772aa99c-kube-api-access-7xkrp\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.777883 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.777786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72088158-d1c4-4d2f-be92-bb3f772aa99c-data-volume\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.777883 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.777810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72088158-d1c4-4d2f-be92-bb3f772aa99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.785175 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.785156    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c99f78fdd-9bq29"]
Jun 02 15:49:18.786052 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.786033    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"]
Jun 02 15:49:18.826259 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.826224    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fdd9c8df7-f9twg"]
Jun 02 15:49:18.829758 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.829735    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.878217 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878181    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72088158-d1c4-4d2f-be92-bb3f772aa99c-data-volume\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878217 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878216    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72088158-d1c4-4d2f-be92-bb3f772aa99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb28d427-0780-43ac-a3f5-eef65787cb92-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9q855\" (UID: \"bb28d427-0780-43ac-a3f5-eef65787cb92\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:18.878422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878273    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72088158-d1c4-4d2f-be92-bb3f772aa99c-crio-socket\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878296    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72088158-d1c4-4d2f-be92-bb3f772aa99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878319    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkrp\" (UniqueName: \"kubernetes.io/projected/72088158-d1c4-4d2f-be92-bb3f772aa99c-kube-api-access-7xkrp\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb28d427-0780-43ac-a3f5-eef65787cb92-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-9q855\" (UID: \"bb28d427-0780-43ac-a3f5-eef65787cb92\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:18.878660 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878433    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72088158-d1c4-4d2f-be92-bb3f772aa99c-crio-socket\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878660 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878574    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72088158-d1c4-4d2f-be92-bb3f772aa99c-data-volume\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.878881 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.878861    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72088158-d1c4-4d2f-be92-bb3f772aa99c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.880658 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.880642    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72088158-d1c4-4d2f-be92-bb3f772aa99c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.896899 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.896870    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fdd9c8df7-f9twg"]
Jun 02 15:49:18.897747 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.897724    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-j2jg5"]
Jun 02 15:49:18.901317 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.901299    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:18.913392 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.913375    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 02 15:49:18.913490 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.913471    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 02 15:49:18.913548 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.913531    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-sjsjq\""
Jun 02 15:49:18.915732 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.915711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkrp\" (UniqueName: \"kubernetes.io/projected/72088158-d1c4-4d2f-be92-bb3f772aa99c-kube-api-access-7xkrp\") pod \"insights-runtime-extractor-d647j\" (UID: \"72088158-d1c4-4d2f-be92-bb3f772aa99c\") " pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:18.922591 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.922570    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-j2jg5"]
Jun 02 15:49:18.979543 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6969dfd7-09e7-4fec-baba-a90816d3f83a-ca-trust-extracted\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979543 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkpnd\" (UniqueName: \"kubernetes.io/projected/fdd2a710-f503-42f2-8397-72f3106a2630-kube-api-access-tkpnd\") pod \"downloads-c65b8dc4b-j2jg5\" (UID: \"fdd2a710-f503-42f2-8397-72f3106a2630\") " pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:18.979714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6969dfd7-09e7-4fec-baba-a90816d3f83a-installation-pull-secrets\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979585    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb28d427-0780-43ac-a3f5-eef65787cb92-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9q855\" (UID: \"bb28d427-0780-43ac-a3f5-eef65787cb92\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:18.979714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6969dfd7-09e7-4fec-baba-a90816d3f83a-registry-certificates\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6969dfd7-09e7-4fec-baba-a90816d3f83a-trusted-ca\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-bound-sa-token\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6969dfd7-09e7-4fec-baba-a90816d3f83a-image-registry-private-configuration\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znkdx\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-kube-api-access-znkdx\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-registry-tls\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:18.979892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.979800    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb28d427-0780-43ac-a3f5-eef65787cb92-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-9q855\" (UID: \"bb28d427-0780-43ac-a3f5-eef65787cb92\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:18.980412 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.980341    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb28d427-0780-43ac-a3f5-eef65787cb92-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-9q855\" (UID: \"bb28d427-0780-43ac-a3f5-eef65787cb92\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:18.982270 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:18.982246    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bb28d427-0780-43ac-a3f5-eef65787cb92-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9q855\" (UID: \"bb28d427-0780-43ac-a3f5-eef65787cb92\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:19.030872 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.030843    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d647j"
Jun 02 15:49:19.080649 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080563    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"
Jun 02 15:49:19.080783 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080762    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-registry-tls\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.080852 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080841    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6969dfd7-09e7-4fec-baba-a90816d3f83a-ca-trust-extracted\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.080907 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080870    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tkpnd\" (UniqueName: \"kubernetes.io/projected/fdd2a710-f503-42f2-8397-72f3106a2630-kube-api-access-tkpnd\") pod \"downloads-c65b8dc4b-j2jg5\" (UID: \"fdd2a710-f503-42f2-8397-72f3106a2630\") " pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:19.080956 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080910    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6969dfd7-09e7-4fec-baba-a90816d3f83a-installation-pull-secrets\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.080956 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080946    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6969dfd7-09e7-4fec-baba-a90816d3f83a-registry-certificates\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.081077 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080968    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6969dfd7-09e7-4fec-baba-a90816d3f83a-trusted-ca\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.081077 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.080996    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-bound-sa-token\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.081177 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.081072    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6969dfd7-09e7-4fec-baba-a90816d3f83a-image-registry-private-configuration\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.081177 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.081102    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-znkdx\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-kube-api-access-znkdx\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.081883 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.081293    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6969dfd7-09e7-4fec-baba-a90816d3f83a-ca-trust-extracted\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.083797 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.083714    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6969dfd7-09e7-4fec-baba-a90816d3f83a-trusted-ca\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.089095 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.084292    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6969dfd7-09e7-4fec-baba-a90816d3f83a-registry-certificates\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.089095 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.084309    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6969dfd7-09e7-4fec-baba-a90816d3f83a-installation-pull-secrets\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.089095 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.086840    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-registry-tls\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.093775 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.093744    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6969dfd7-09e7-4fec-baba-a90816d3f83a-image-registry-private-configuration\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.093875 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.093787    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkpnd\" (UniqueName: \"kubernetes.io/projected/fdd2a710-f503-42f2-8397-72f3106a2630-kube-api-access-tkpnd\") pod \"downloads-c65b8dc4b-j2jg5\" (UID: \"fdd2a710-f503-42f2-8397-72f3106a2630\") " pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:19.094613 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.094573    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znkdx\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-kube-api-access-znkdx\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.096474 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.096446    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6969dfd7-09e7-4fec-baba-a90816d3f83a-bound-sa-token\") pod \"image-registry-fdd9c8df7-f9twg\" (UID: \"6969dfd7-09e7-4fec-baba-a90816d3f83a\") " pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.138198 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.138163    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.172168 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.172115    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d647j"]
Jun 02 15:49:19.176881 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:19.176839    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72088158_d1c4_4d2f_be92_bb3f772aa99c.slice/crio-72a660b86b8e1855a11fa211d5414e62d662dfccc1458c7cbade2fde17831ccb WatchSource:0}: Error finding container 72a660b86b8e1855a11fa211d5414e62d662dfccc1458c7cbade2fde17831ccb: Status 404 returned error can't find the container with id 72a660b86b8e1855a11fa211d5414e62d662dfccc1458c7cbade2fde17831ccb
Jun 02 15:49:19.210041 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.209932    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:19.226829 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.226787    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855"]
Jun 02 15:49:19.231292 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:19.231248    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb28d427_0780_43ac_a3f5_eef65787cb92.slice/crio-afa8ab9888fad5fa01eb16c4d55b534417402ea55749c635f8eeba19055657d5 WatchSource:0}: Error finding container afa8ab9888fad5fa01eb16c4d55b534417402ea55749c635f8eeba19055657d5: Status 404 returned error can't find the container with id afa8ab9888fad5fa01eb16c4d55b534417402ea55749c635f8eeba19055657d5
Jun 02 15:49:19.280839 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.280787    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fdd9c8df7-f9twg"]
Jun 02 15:49:19.285466 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:19.285438    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6969dfd7_09e7_4fec_baba_a90816d3f83a.slice/crio-6752d4edd48c2586b34d94db6dc1293700d8cde63b639c47539ecd448c12817a WatchSource:0}: Error finding container 6752d4edd48c2586b34d94db6dc1293700d8cde63b639c47539ecd448c12817a: Status 404 returned error can't find the container with id 6752d4edd48c2586b34d94db6dc1293700d8cde63b639c47539ecd448c12817a
Jun 02 15:49:19.351680 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.351654    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-j2jg5"]
Jun 02 15:49:19.356505 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:19.356477    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd2a710_f503_42f2_8397_72f3106a2630.slice/crio-b99b8ea94f00547ca2e40ec041546305c7d0c3fc47e1c8947bb2cdf965c1dd3c WatchSource:0}: Error finding container b99b8ea94f00547ca2e40ec041546305c7d0c3fc47e1c8947bb2cdf965c1dd3c: Status 404 returned error can't find the container with id b99b8ea94f00547ca2e40ec041546305c7d0c3fc47e1c8947bb2cdf965c1dd3c
Jun 02 15:49:19.482891 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.482798    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-j2jg5" event={"ID":"fdd2a710-f503-42f2-8397-72f3106a2630","Type":"ContainerStarted","Data":"b99b8ea94f00547ca2e40ec041546305c7d0c3fc47e1c8947bb2cdf965c1dd3c"}
Jun 02 15:49:19.484224 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.484198    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d647j" event={"ID":"72088158-d1c4-4d2f-be92-bb3f772aa99c","Type":"ContainerStarted","Data":"b73213b9599a6ea6869ae5a24c2dbf6821c674d5ceec95f0a2afd9f5ba1b7824"}
Jun 02 15:49:19.484224 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.484227    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d647j" event={"ID":"72088158-d1c4-4d2f-be92-bb3f772aa99c","Type":"ContainerStarted","Data":"72a660b86b8e1855a11fa211d5414e62d662dfccc1458c7cbade2fde17831ccb"}
Jun 02 15:49:19.485710 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.485680    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg" event={"ID":"6969dfd7-09e7-4fec-baba-a90816d3f83a","Type":"ContainerStarted","Data":"6e7dc6f9aa9d3d6a4526f1ed6e2883cbed4f56bdd148d7539c13b90891d60a08"}
Jun 02 15:49:19.485829 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.485717    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg" event={"ID":"6969dfd7-09e7-4fec-baba-a90816d3f83a","Type":"ContainerStarted","Data":"6752d4edd48c2586b34d94db6dc1293700d8cde63b639c47539ecd448c12817a"}
Jun 02 15:49:19.485829 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.485758    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:19.486710 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.486686    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855" event={"ID":"bb28d427-0780-43ac-a3f5-eef65787cb92","Type":"ContainerStarted","Data":"afa8ab9888fad5fa01eb16c4d55b534417402ea55749c635f8eeba19055657d5"}
Jun 02 15:49:19.507466 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:19.507404    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg" podStartSLOduration=1.5073829989999998 podStartE2EDuration="1.507382999s" podCreationTimestamp="2026-06-02 15:49:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:49:19.506664632 +0000 UTC m=+144.130453478" watchObservedRunningTime="2026-06-02 15:49:19.507382999 +0000 UTC m=+144.131171849"
Jun 02 15:49:20.491133 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:20.491039    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855" event={"ID":"bb28d427-0780-43ac-a3f5-eef65787cb92","Type":"ContainerStarted","Data":"03a3043e73b144aadc98607d1c819c0561cd1d4c1e1c5e3a78401a827d24cbe2"}
Jun 02 15:49:20.493004 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:20.492971    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d647j" event={"ID":"72088158-d1c4-4d2f-be92-bb3f772aa99c","Type":"ContainerStarted","Data":"efac69d1a3deae8fe46602630b1d0282e4af313b115d1564b9d139ff8d6ef885"}
Jun 02 15:49:20.509677 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:20.509616    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9q855" podStartSLOduration=1.477938605 podStartE2EDuration="2.509599661s" podCreationTimestamp="2026-06-02 15:49:18 +0000 UTC" firstStartedPulling="2026-06-02 15:49:19.233424976 +0000 UTC m=+143.857213801" lastFinishedPulling="2026-06-02 15:49:20.265086019 +0000 UTC m=+144.888874857" observedRunningTime="2026-06-02 15:49:20.508822671 +0000 UTC m=+145.132611519" watchObservedRunningTime="2026-06-02 15:49:20.509599661 +0000 UTC m=+145.133388510"
Jun 02 15:49:21.805608 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:21.805574    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:49:21.808577 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:21.808549    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e07e5347-9398-4ad3-9d41-e7948879b7b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-t5n2c\" (UID: \"e07e5347-9398-4ad3-9d41-e7948879b7b4\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:49:22.077766 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:22.077729    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xsp2g\""
Jun 02 15:49:22.084396 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:22.084366    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"
Jun 02 15:49:22.236037 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:22.234766    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c"]
Jun 02 15:49:22.238635 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:22.238607    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07e5347_9398_4ad3_9d41_e7948879b7b4.slice/crio-7bd91e17939349e2cec1b8c3072d6abe4ad725057c3d4209fc637f1acca8ab2e WatchSource:0}: Error finding container 7bd91e17939349e2cec1b8c3072d6abe4ad725057c3d4209fc637f1acca8ab2e: Status 404 returned error can't find the container with id 7bd91e17939349e2cec1b8c3072d6abe4ad725057c3d4209fc637f1acca8ab2e
Jun 02 15:49:22.500360 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:22.500258    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d647j" event={"ID":"72088158-d1c4-4d2f-be92-bb3f772aa99c","Type":"ContainerStarted","Data":"dba32e814a6a46dc46065f09ad4d7862fc26f8c71890922b6178250ed4a64d6c"}
Jun 02 15:49:22.501333 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:22.501304    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c" event={"ID":"e07e5347-9398-4ad3-9d41-e7948879b7b4","Type":"ContainerStarted","Data":"7bd91e17939349e2cec1b8c3072d6abe4ad725057c3d4209fc637f1acca8ab2e"}
Jun 02 15:49:22.523780 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:22.523726    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d647j" podStartSLOduration=1.9948438290000001 podStartE2EDuration="4.523710867s" podCreationTimestamp="2026-06-02 15:49:18 +0000 UTC" firstStartedPulling="2026-06-02 15:49:19.255530299 +0000 UTC m=+143.879319124" lastFinishedPulling="2026-06-02 15:49:21.784397318 +0000 UTC m=+146.408186162" observedRunningTime="2026-06-02 15:49:22.521878917 +0000 UTC m=+147.145667764" watchObservedRunningTime="2026-06-02 15:49:22.523710867 +0000 UTC m=+147.147499713"
Jun 02 15:49:24.509709 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.509669    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c" event={"ID":"e07e5347-9398-4ad3-9d41-e7948879b7b4","Type":"ContainerStarted","Data":"81cdce3087404dacf15ea4019860a1cc0d84d73a8f29275224e5673e351f61b9"}
Jun 02 15:49:24.578700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.578636    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-t5n2c" podStartSLOduration=33.768222358 podStartE2EDuration="35.578614235s" podCreationTimestamp="2026-06-02 15:48:49 +0000 UTC" firstStartedPulling="2026-06-02 15:49:22.240438057 +0000 UTC m=+146.864226892" lastFinishedPulling="2026-06-02 15:49:24.050829931 +0000 UTC m=+148.674618769" observedRunningTime="2026-06-02 15:49:24.530737003 +0000 UTC m=+149.154525849" watchObservedRunningTime="2026-06-02 15:49:24.578614235 +0000 UTC m=+149.202403085"
Jun 02 15:49:24.579512 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.579489    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"]
Jun 02 15:49:24.583714 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.583695    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:24.586582 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.586557    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jun 02 15:49:24.586691 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.586587    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vjdnr\""
Jun 02 15:49:24.597168 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.597105    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"]
Jun 02 15:49:24.734405 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.734366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a2c4c393-af6c-4c29-8487-ce43eca6725b-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xv5tg\" (UID: \"a2c4c393-af6c-4c29-8487-ce43eca6725b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:24.834998 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.834959    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a2c4c393-af6c-4c29-8487-ce43eca6725b-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xv5tg\" (UID: \"a2c4c393-af6c-4c29-8487-ce43eca6725b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:24.838064 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.838036    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a2c4c393-af6c-4c29-8487-ce43eca6725b-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xv5tg\" (UID: \"a2c4c393-af6c-4c29-8487-ce43eca6725b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:24.894205 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:24.894170    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:25.036226 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:25.036194    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"]
Jun 02 15:49:25.039867 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:25.039838    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c4c393_af6c_4c29_8487_ce43eca6725b.slice/crio-2f1586273d0232df0dad58d6cf3212393158cf58ae1ff5f950b2e1969e125b75 WatchSource:0}: Error finding container 2f1586273d0232df0dad58d6cf3212393158cf58ae1ff5f950b2e1969e125b75: Status 404 returned error can't find the container with id 2f1586273d0232df0dad58d6cf3212393158cf58ae1ff5f950b2e1969e125b75
Jun 02 15:49:25.514301 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:25.514265    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg" event={"ID":"a2c4c393-af6c-4c29-8487-ce43eca6725b","Type":"ContainerStarted","Data":"2f1586273d0232df0dad58d6cf3212393158cf58ae1ff5f950b2e1969e125b75"}
Jun 02 15:49:26.518641 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:26.518552    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg" event={"ID":"a2c4c393-af6c-4c29-8487-ce43eca6725b","Type":"ContainerStarted","Data":"f56ca7c01a3c93bd124d188e093a70ca27560bf873deba6fb189e40bfeda8947"}
Jun 02 15:49:26.519118 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:26.518815    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:26.524652 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:26.524627    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg"
Jun 02 15:49:26.535153 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:26.535097    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xv5tg" podStartSLOduration=1.34947285 podStartE2EDuration="2.535083439s" podCreationTimestamp="2026-06-02 15:49:24 +0000 UTC" firstStartedPulling="2026-06-02 15:49:25.042253494 +0000 UTC m=+149.666042332" lastFinishedPulling="2026-06-02 15:49:26.227864084 +0000 UTC m=+150.851652921" observedRunningTime="2026-06-02 15:49:26.534327663 +0000 UTC m=+151.158116511" watchObservedRunningTime="2026-06-02 15:49:26.535083439 +0000 UTC m=+151.158872286"
Jun 02 15:49:28.791099 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:28.791052    2563 patch_prober.go:28] interesting pod/image-registry-5c99f78fdd-9bq29 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 02 15:49:28.791555 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:28.791124    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:49:32.037580 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.037540    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"]
Jun 02 15:49:32.042379 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.042351    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.045334 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.045308    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jun 02 15:49:32.045473 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.045345    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jun 02 15:49:32.045473 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.045414    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jlcj4\""
Jun 02 15:49:32.046896 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.046879    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 02 15:49:32.054518 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.054486    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"]
Jun 02 15:49:32.070065 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.070035    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zjnws"]
Jun 02 15:49:32.073235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.073213    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.077319 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.077300    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 02 15:49:32.077416 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.077390    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2wjpl\""
Jun 02 15:49:32.077620 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.077602    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 02 15:49:32.079212 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.079193    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 02 15:49:32.199881 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.199847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-tls\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200064 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.199888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f64c06dd-bb9b-426d-b49f-b8f412d3093e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.200064 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.199994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f64c06dd-bb9b-426d-b49f-b8f412d3093e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.200064 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-textfile\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200293 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-wtmp\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200293 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f64c06dd-bb9b-426d-b49f-b8f412d3093e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.200293 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2pwg\" (UniqueName: \"kubernetes.io/projected/6dd45487-bc47-4833-82f9-1802d25e487b-kube-api-access-k2pwg\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200293 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200293 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-root\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200499 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200499 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-sys\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.200499 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpspg\" (UniqueName: \"kubernetes.io/projected/f64c06dd-bb9b-426d-b49f-b8f412d3093e-kube-api-access-zpspg\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.200499 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.200413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dd45487-bc47-4833-82f9-1802d25e487b-metrics-client-ca\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301362 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301326    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zpspg\" (UniqueName: \"kubernetes.io/projected/f64c06dd-bb9b-426d-b49f-b8f412d3093e-kube-api-access-zpspg\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.301556 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301375    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dd45487-bc47-4833-82f9-1802d25e487b-metrics-client-ca\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301556 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301403    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-tls\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301556 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301426    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f64c06dd-bb9b-426d-b49f-b8f412d3093e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.301556 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301474    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f64c06dd-bb9b-426d-b49f-b8f412d3093e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.301556 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301504    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-textfile\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301668    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-wtmp\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301758    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f64c06dd-bb9b-426d-b49f-b8f412d3093e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.301832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301780    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k2pwg\" (UniqueName: \"kubernetes.io/projected/6dd45487-bc47-4833-82f9-1802d25e487b-kube-api-access-k2pwg\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301810    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.301832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301830    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-root\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302112 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301852    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302112 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301887    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-sys\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302112 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.301969    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-sys\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302112 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.302043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-wtmp\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302327 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.302116    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6dd45487-bc47-4833-82f9-1802d25e487b-root\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302327 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.302129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dd45487-bc47-4833-82f9-1802d25e487b-metrics-client-ca\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.302327 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.302227    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f64c06dd-bb9b-426d-b49f-b8f412d3093e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.302803 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.302775    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.303094 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.303067    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-textfile\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.304656 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.304631    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-tls\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.304756 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.304663    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f64c06dd-bb9b-426d-b49f-b8f412d3093e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.304899 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.304880    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dd45487-bc47-4833-82f9-1802d25e487b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.304972 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.304916    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f64c06dd-bb9b-426d-b49f-b8f412d3093e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.311251 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.311222    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpspg\" (UniqueName: \"kubernetes.io/projected/f64c06dd-bb9b-426d-b49f-b8f412d3093e-kube-api-access-zpspg\") pod \"openshift-state-metrics-85658796db-lsmgx\" (UID: \"f64c06dd-bb9b-426d-b49f-b8f412d3093e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.311500 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.311483    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2pwg\" (UniqueName: \"kubernetes.io/projected/6dd45487-bc47-4833-82f9-1802d25e487b-kube-api-access-k2pwg\") pod \"node-exporter-zjnws\" (UID: \"6dd45487-bc47-4833-82f9-1802d25e487b\") " pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.351905 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.351870    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"
Jun 02 15:49:32.355821 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:32.355782    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-7fbbj" podUID="09c32fc9-879b-4dd1-9b1a-6b16d9972168"
Jun 02 15:49:32.371352 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:32.371290    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-88dbl" podUID="14fc5c79-98d8-47a1-8c19-2cb07454b83e"
Jun 02 15:49:32.383409 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.383377    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zjnws"
Jun 02 15:49:32.534196 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.534159    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:49:32.534387 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:32.534362    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-88dbl"
Jun 02 15:49:33.987424 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:33.987384    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5r476" podUID="aca2f9e4-d1e9-470c-b700-d6cc6f3be66c"
Jun 02 15:49:34.157101 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.157053    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8676877756-zk85p"]
Jun 02 15:49:34.162327 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.162301    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.165311 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.165191    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-47h7f0ie2iuc5\""
Jun 02 15:49:34.165454 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.165345    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jun 02 15:49:34.165454 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.165416    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jun 02 15:49:34.165575 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.165473    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jun 02 15:49:34.165623 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.165569    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4kmdz\""
Jun 02 15:49:34.165781 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.165695    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jun 02 15:49:34.166316 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.166291    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jun 02 15:49:34.171690 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.171670    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8676877756-zk85p"]
Jun 02 15:49:34.322024 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.321978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322217 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25e2b5c-21e7-450c-b856-ec1639508495-metrics-client-ca\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322217 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322217 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322391 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322391 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-tls\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322483 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-grpc-tls\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.322483 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.322446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlkxw\" (UniqueName: \"kubernetes.io/projected/a25e2b5c-21e7-450c-b856-ec1639508495-kube-api-access-qlkxw\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.423615 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423578    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-tls\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.423615 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423626    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-grpc-tls\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.423862 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423660    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qlkxw\" (UniqueName: \"kubernetes.io/projected/a25e2b5c-21e7-450c-b856-ec1639508495-kube-api-access-qlkxw\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.423862 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423696    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.423862 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423769    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25e2b5c-21e7-450c-b856-ec1639508495-metrics-client-ca\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.424031 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423884    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.424031 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423929    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.424031 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.423977    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.424738 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.424627    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25e2b5c-21e7-450c-b856-ec1639508495-metrics-client-ca\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.427073 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.426945    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.427194 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.427087    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-grpc-tls\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.427412 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.427375    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.427537 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.427515    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.427585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.427547    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.427761 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.427740    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a25e2b5c-21e7-450c-b856-ec1639508495-secret-thanos-querier-tls\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.433627 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.433605    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlkxw\" (UniqueName: \"kubernetes.io/projected/a25e2b5c-21e7-450c-b856-ec1639508495-kube-api-access-qlkxw\") pod \"thanos-querier-8676877756-zk85p\" (UID: \"a25e2b5c-21e7-450c-b856-ec1639508495\") " pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:34.473605 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:34.473565    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:35.251477 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:35.251444    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd45487_bc47_4833_82f9_1802d25e487b.slice/crio-443f2bcebf4f401c1fda23a213e1b8ab54dd9bfb396e25eb1fd7ea0043c569a9 WatchSource:0}: Error finding container 443f2bcebf4f401c1fda23a213e1b8ab54dd9bfb396e25eb1fd7ea0043c569a9: Status 404 returned error can't find the container with id 443f2bcebf4f401c1fda23a213e1b8ab54dd9bfb396e25eb1fd7ea0043c569a9
Jun 02 15:49:35.399045 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.398998    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8676877756-zk85p"]
Jun 02 15:49:35.402044 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.402005    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-lsmgx"]
Jun 02 15:49:35.402444 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:35.402422    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25e2b5c_21e7_450c_b856_ec1639508495.slice/crio-911b28928669180e77a903f5c7b419d80d0de6ea636bbecfbfdd8955e9a6238f WatchSource:0}: Error finding container 911b28928669180e77a903f5c7b419d80d0de6ea636bbecfbfdd8955e9a6238f: Status 404 returned error can't find the container with id 911b28928669180e77a903f5c7b419d80d0de6ea636bbecfbfdd8955e9a6238f
Jun 02 15:49:35.405258 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:35.405236    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64c06dd_bb9b_426d_b49f_b8f412d3093e.slice/crio-8007971b6db8340239181c120164edb77058667c43186fdf69a0bab7c23d9f61 WatchSource:0}: Error finding container 8007971b6db8340239181c120164edb77058667c43186fdf69a0bab7c23d9f61: Status 404 returned error can't find the container with id 8007971b6db8340239181c120164edb77058667c43186fdf69a0bab7c23d9f61
Jun 02 15:49:35.543986 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.543956    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx" event={"ID":"f64c06dd-bb9b-426d-b49f-b8f412d3093e","Type":"ContainerStarted","Data":"bf590ca55e1b5c27d015f337d32b744020cf6809f2c2cb5775d6464209d10d22"}
Jun 02 15:49:35.544128 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.543995    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx" event={"ID":"f64c06dd-bb9b-426d-b49f-b8f412d3093e","Type":"ContainerStarted","Data":"a7cfc46a43c78409757cfc3c690cfa87509776d2f819955d70e0ad0e1ba10681"}
Jun 02 15:49:35.544128 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.544005    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx" event={"ID":"f64c06dd-bb9b-426d-b49f-b8f412d3093e","Type":"ContainerStarted","Data":"8007971b6db8340239181c120164edb77058667c43186fdf69a0bab7c23d9f61"}
Jun 02 15:49:35.545127 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.545094    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjnws" event={"ID":"6dd45487-bc47-4833-82f9-1802d25e487b","Type":"ContainerStarted","Data":"443f2bcebf4f401c1fda23a213e1b8ab54dd9bfb396e25eb1fd7ea0043c569a9"}
Jun 02 15:49:35.546610 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.546587    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-j2jg5" event={"ID":"fdd2a710-f503-42f2-8397-72f3106a2630","Type":"ContainerStarted","Data":"67e7159874a7f7bb9b51cffbe72fea05dc42cfcb4f4ac2dd4b0cc1107d1a9926"}
Jun 02 15:49:35.546885 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.546824    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:35.548030 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.547976    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"911b28928669180e77a903f5c7b419d80d0de6ea636bbecfbfdd8955e9a6238f"}
Jun 02 15:49:35.548100 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.548029    2563 patch_prober.go:28] interesting pod/downloads-c65b8dc4b-j2jg5 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.15:8080/\": dial tcp 10.134.0.15:8080: connect: connection refused" start-of-body=
Jun 02 15:49:35.548100 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.548081    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-c65b8dc4b-j2jg5" podUID="fdd2a710-f503-42f2-8397-72f3106a2630" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.15:8080/\": dial tcp 10.134.0.15:8080: connect: connection refused"
Jun 02 15:49:35.564147 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:35.564086    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-j2jg5" podStartSLOduration=1.581366535 podStartE2EDuration="17.564067705s" podCreationTimestamp="2026-06-02 15:49:18 +0000 UTC" firstStartedPulling="2026-06-02 15:49:19.35841836 +0000 UTC m=+143.982207184" lastFinishedPulling="2026-06-02 15:49:35.341119513 +0000 UTC m=+159.964908354" observedRunningTime="2026-06-02 15:49:35.562649327 +0000 UTC m=+160.186438175" watchObservedRunningTime="2026-06-02 15:49:35.564067705 +0000 UTC m=+160.187856552"
Jun 02 15:49:36.566887 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.566729    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-j2jg5"
Jun 02 15:49:36.817005 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.816427    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"]
Jun 02 15:49:36.821353 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.820884    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:36.823904 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.823619    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jun 02 15:49:36.823904 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.823678    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-t7ggx\""
Jun 02 15:49:36.830666 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.830355    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"]
Jun 02 15:49:36.952150 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:36.952094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/202eeb56-5528-4d44-823e-363bdc40f087-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-6pj6d\" (UID: \"202eeb56-5528-4d44-823e-363bdc40f087\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:37.053472 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.053390    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/202eeb56-5528-4d44-823e-363bdc40f087-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-6pj6d\" (UID: \"202eeb56-5528-4d44-823e-363bdc40f087\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:37.053599 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:37.053579    2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jun 02 15:49:37.053682 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:37.053650    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202eeb56-5528-4d44-823e-363bdc40f087-monitoring-plugin-cert podName:202eeb56-5528-4d44-823e-363bdc40f087 nodeName:}" failed. No retries permitted until 2026-06-02 15:49:37.553628698 +0000 UTC m=+162.177417541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/202eeb56-5528-4d44-823e-363bdc40f087-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-6pj6d" (UID: "202eeb56-5528-4d44-823e-363bdc40f087") : secret "monitoring-plugin-cert" not found
Jun 02 15:49:37.258051 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.257373    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:49:37.258051 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.257468    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:49:37.273359 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.273324    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09c32fc9-879b-4dd1-9b1a-6b16d9972168-cert\") pod \"ingress-canary-7fbbj\" (UID: \"09c32fc9-879b-4dd1-9b1a-6b16d9972168\") " pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:49:37.274525 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.274502    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14fc5c79-98d8-47a1-8c19-2cb07454b83e-metrics-tls\") pod \"dns-default-88dbl\" (UID: \"14fc5c79-98d8-47a1-8c19-2cb07454b83e\") " pod="openshift-dns/dns-default-88dbl"
Jun 02 15:49:37.338367 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.338328    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gg4df\""
Jun 02 15:49:37.338600 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.338583    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-25zcr\""
Jun 02 15:49:37.346799 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.345962    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-88dbl"
Jun 02 15:49:37.346799 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.346491    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fbbj"
Jun 02 15:49:37.559247 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.559204    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx" event={"ID":"f64c06dd-bb9b-426d-b49f-b8f412d3093e","Type":"ContainerStarted","Data":"cbfceb1424bf7fefef29d8661b2c7ee9960b2e51b9b973a7917787c9274c5996"}
Jun 02 15:49:37.560470 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.560444    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/202eeb56-5528-4d44-823e-363bdc40f087-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-6pj6d\" (UID: \"202eeb56-5528-4d44-823e-363bdc40f087\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:37.562547 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.562504    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjnws" event={"ID":"6dd45487-bc47-4833-82f9-1802d25e487b","Type":"ContainerStarted","Data":"0179826620f42898c1fd5e55c94dbcecb695b59c5096f14c41257bfcc966c927"}
Jun 02 15:49:37.564698 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.564641    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/202eeb56-5528-4d44-823e-363bdc40f087-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-6pj6d\" (UID: \"202eeb56-5528-4d44-823e-363bdc40f087\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:37.582770 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.582712    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-lsmgx" podStartSLOduration=4.159846435 podStartE2EDuration="5.582696276s" podCreationTimestamp="2026-06-02 15:49:32 +0000 UTC" firstStartedPulling="2026-06-02 15:49:35.533575469 +0000 UTC m=+160.157364295" lastFinishedPulling="2026-06-02 15:49:36.956425296 +0000 UTC m=+161.580214136" observedRunningTime="2026-06-02 15:49:37.580767789 +0000 UTC m=+162.204556637" watchObservedRunningTime="2026-06-02 15:49:37.582696276 +0000 UTC m=+162.206485126"
Jun 02 15:49:37.734027 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.733967    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:37.967573 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.967461    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"]
Jun 02 15:49:37.993581 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:37.993551    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7fbbj"]
Jun 02 15:49:38.019200 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.019170    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-88dbl"]
Jun 02 15:49:38.164628 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:49:38.164538    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202eeb56_5528_4d44_823e_363bdc40f087.slice/crio-a3480a2d90d20480f7c55768576da33ebbbacf6a892bf0cb18af87d8bb1ead0f WatchSource:0}: Error finding container a3480a2d90d20480f7c55768576da33ebbbacf6a892bf0cb18af87d8bb1ead0f: Status 404 returned error can't find the container with id a3480a2d90d20480f7c55768576da33ebbbacf6a892bf0cb18af87d8bb1ead0f
Jun 02 15:49:38.271130 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.271089    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8" podUID="a08cf0eb-b597-4156-b44f-a89f6d0b43f5" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.7:8000/readyz\": dial tcp 10.134.0.7:8000: connect: connection refused"
Jun 02 15:49:38.568091 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.567988    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"dc8d353f2efbeb0002fcb4c53f5971e906fbeb237adfe551846c0f72ae5762b4"}
Jun 02 15:49:38.568091 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.568045    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"3c80d07289ac007882c7348513f41d81c5f507c6d6477a9348ec322d0767563d"}
Jun 02 15:49:38.568091 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.568059    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"24a02562aab8d75be129393483f61cddf41ca53f16221b8aa85771a63b611116"}
Jun 02 15:49:38.569481 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.569447    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-88dbl" event={"ID":"14fc5c79-98d8-47a1-8c19-2cb07454b83e","Type":"ContainerStarted","Data":"857dfaae2da4c544542d357c31f570fc246e7d3b7fc5efdd03e45a6886129c3c"}
Jun 02 15:49:38.571096 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.571068    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7fbbj" event={"ID":"09c32fc9-879b-4dd1-9b1a-6b16d9972168","Type":"ContainerStarted","Data":"46c909c255fdc29f71420a780ddf92d6a958f3cc6ef459946f0735e9e8fe1698"}
Jun 02 15:49:38.572969 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.572507    2563 generic.go:358] "Generic (PLEG): container finished" podID="a08cf0eb-b597-4156-b44f-a89f6d0b43f5" containerID="08754197ac0436715cbbfe9bb9370ca07ce2dd5108de99a80d5b1ef6403c435c" exitCode=1
Jun 02 15:49:38.572969 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.572550    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8" event={"ID":"a08cf0eb-b597-4156-b44f-a89f6d0b43f5","Type":"ContainerDied","Data":"08754197ac0436715cbbfe9bb9370ca07ce2dd5108de99a80d5b1ef6403c435c"}
Jun 02 15:49:38.573144 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.572984    2563 scope.go:117] "RemoveContainer" containerID="08754197ac0436715cbbfe9bb9370ca07ce2dd5108de99a80d5b1ef6403c435c"
Jun 02 15:49:38.573942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.573921    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d" event={"ID":"202eeb56-5528-4d44-823e-363bdc40f087","Type":"ContainerStarted","Data":"a3480a2d90d20480f7c55768576da33ebbbacf6a892bf0cb18af87d8bb1ead0f"}
Jun 02 15:49:38.575656 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.575634    2563 generic.go:358] "Generic (PLEG): container finished" podID="6dd45487-bc47-4833-82f9-1802d25e487b" containerID="0179826620f42898c1fd5e55c94dbcecb695b59c5096f14c41257bfcc966c927" exitCode=0
Jun 02 15:49:38.575786 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.575762    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjnws" event={"ID":"6dd45487-bc47-4833-82f9-1802d25e487b","Type":"ContainerDied","Data":"0179826620f42898c1fd5e55c94dbcecb695b59c5096f14c41257bfcc966c927"}
Jun 02 15:49:38.792984 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.792559    2563 patch_prober.go:28] interesting pod/image-registry-5c99f78fdd-9bq29 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 02 15:49:38.792984 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:38.792618    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:49:39.146912 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.146586    2563 patch_prober.go:28] interesting pod/image-registry-fdd9c8df7-f9twg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 02 15:49:39.146912 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.146660    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg" podUID="6969dfd7-09e7-4fec-baba-a90816d3f83a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:49:39.584106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.583997    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjnws" event={"ID":"6dd45487-bc47-4833-82f9-1802d25e487b","Type":"ContainerStarted","Data":"67d17f5b2247cf94ba841ba911e51b451f3ef0afe3550691d00abc169a4b44f4"}
Jun 02 15:49:39.584106 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.584073    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjnws" event={"ID":"6dd45487-bc47-4833-82f9-1802d25e487b","Type":"ContainerStarted","Data":"57079325e285824836dc5556c7300239a303a8b68862573c85ff9eff2c0473c6"}
Jun 02 15:49:39.586814 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.586639    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8" event={"ID":"a08cf0eb-b597-4156-b44f-a89f6d0b43f5","Type":"ContainerStarted","Data":"b32754a3915ad3c5d78c3d5da88ae7b76845313c381293557357a2b7cf5eb1e0"}
Jun 02 15:49:39.587723 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.587227    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:49:39.588309 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.588279    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ffd444b74-fjvn8"
Jun 02 15:49:39.636240 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:39.636061    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zjnws" podStartSLOduration=5.935988623 podStartE2EDuration="7.636043649s" podCreationTimestamp="2026-06-02 15:49:32 +0000 UTC" firstStartedPulling="2026-06-02 15:49:35.253723157 +0000 UTC m=+159.877511987" lastFinishedPulling="2026-06-02 15:49:36.953778188 +0000 UTC m=+161.577567013" observedRunningTime="2026-06-02 15:49:39.613277189 +0000 UTC m=+164.237066064" watchObservedRunningTime="2026-06-02 15:49:39.636043649 +0000 UTC m=+164.259832499"
Jun 02 15:49:40.498360 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:40.497951    2563 patch_prober.go:28] interesting pod/image-registry-fdd9c8df7-f9twg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 02 15:49:40.498360 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:40.498046    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg" podUID="6969dfd7-09e7-4fec-baba-a90816d3f83a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:49:42.604046 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.603375    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d" event={"ID":"202eeb56-5528-4d44-823e-363bdc40f087","Type":"ContainerStarted","Data":"9bd3c5acd3a488426c5cdddf15944794a57103b1c70a3297941c3c4ab1f770c6"}
Jun 02 15:49:42.604472 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.604114    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:42.610147 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.610070    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"a78adb72fb40ccbe9edad7ebaa24f5533e1986d4e500fd9455c063455f94481e"}
Jun 02 15:49:42.610147 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.610112    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"0089e9ba360987fea9a4f9d35b7a6e3c2619fa1288074f2d0a4698437fdd7cfb"}
Jun 02 15:49:42.612277 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.611875    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-88dbl" event={"ID":"14fc5c79-98d8-47a1-8c19-2cb07454b83e","Type":"ContainerStarted","Data":"c739086b2f20285c2b591ace7fe3c1b9501ba582cc31360d61d0a974e7238379"}
Jun 02 15:49:42.612277 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.612143    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d"
Jun 02 15:49:42.614202 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.613568    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7fbbj" event={"ID":"09c32fc9-879b-4dd1-9b1a-6b16d9972168","Type":"ContainerStarted","Data":"1152e3e5fb924602b0c52fc5387a7a6a1a7e019efa64eb5fa06a58185f555dfd"}
Jun 02 15:49:42.625615 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.625198    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-6pj6d" podStartSLOduration=2.539074891 podStartE2EDuration="6.625180578s" podCreationTimestamp="2026-06-02 15:49:36 +0000 UTC" firstStartedPulling="2026-06-02 15:49:38.189264653 +0000 UTC m=+162.813053480" lastFinishedPulling="2026-06-02 15:49:42.275370336 +0000 UTC m=+166.899159167" observedRunningTime="2026-06-02 15:49:42.623493953 +0000 UTC m=+167.247282797" watchObservedRunningTime="2026-06-02 15:49:42.625180578 +0000 UTC m=+167.248969426"
Jun 02 15:49:42.646372 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:42.646315    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7fbbj" podStartSLOduration=129.554571094 podStartE2EDuration="2m13.646299189s" podCreationTimestamp="2026-06-02 15:47:29 +0000 UTC" firstStartedPulling="2026-06-02 15:49:38.188838175 +0000 UTC m=+162.812627007" lastFinishedPulling="2026-06-02 15:49:42.280566274 +0000 UTC m=+166.904355102" observedRunningTime="2026-06-02 15:49:42.645950284 +0000 UTC m=+167.269739145" watchObservedRunningTime="2026-06-02 15:49:42.646299189 +0000 UTC m=+167.270088035"
Jun 02 15:49:43.623077 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.623033    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" event={"ID":"a25e2b5c-21e7-450c-b856-ec1639508495","Type":"ContainerStarted","Data":"d027eabd54ecbf00cdf7222fe74deeded7acd39137011c0bc6339e064c4f7df3"}
Jun 02 15:49:43.623649 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.623485    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:43.625456 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.625427    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-88dbl" event={"ID":"14fc5c79-98d8-47a1-8c19-2cb07454b83e","Type":"ContainerStarted","Data":"51382ba239df87ae1ec243b8924ee03aaa18983aa34d60dc86cd9e0d23fad882"}
Jun 02 15:49:43.630569 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.630532    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8676877756-zk85p"
Jun 02 15:49:43.668399 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.668323    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8676877756-zk85p" podStartSLOduration=2.797050812 podStartE2EDuration="9.66830502s" podCreationTimestamp="2026-06-02 15:49:34 +0000 UTC" firstStartedPulling="2026-06-02 15:49:35.403913075 +0000 UTC m=+160.027701901" lastFinishedPulling="2026-06-02 15:49:42.275167269 +0000 UTC m=+166.898956109" observedRunningTime="2026-06-02 15:49:43.664680857 +0000 UTC m=+168.288469705" watchObservedRunningTime="2026-06-02 15:49:43.66830502 +0000 UTC m=+168.292093868"
Jun 02 15:49:43.743069 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.742986    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-88dbl" podStartSLOduration=130.656642593 podStartE2EDuration="2m14.742969213s" podCreationTimestamp="2026-06-02 15:47:29 +0000 UTC" firstStartedPulling="2026-06-02 15:49:38.188844961 +0000 UTC m=+162.812633797" lastFinishedPulling="2026-06-02 15:49:42.275171588 +0000 UTC m=+166.898960417" observedRunningTime="2026-06-02 15:49:43.742040573 +0000 UTC m=+168.365829417" watchObservedRunningTime="2026-06-02 15:49:43.742969213 +0000 UTC m=+168.366758061"
Jun 02 15:49:43.805118 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:43.805070    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerName="registry" containerID="cri-o://26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08" gracePeriod=30
Jun 02 15:49:44.077146 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.077121    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:49:44.127092 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127059    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-trusted-ca\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127247 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127117    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-installation-pull-secrets\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127247 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127158    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127247 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127192    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-ca-trust-extracted\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127247 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127235    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-certificates\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127450 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127261    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkqtn\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-kube-api-access-gkqtn\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127450 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127296    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-bound-sa-token\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127450 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127330    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-image-registry-private-configuration\") pod \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\" (UID: \"774ff4f5-76fe-455f-9bb9-5c40c2d217ac\") "
Jun 02 15:49:44.127600 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127489    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:49:44.127600 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127595    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-trusted-ca\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.127704 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.127672    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:49:44.129938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.129909    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:49:44.130193 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.130167    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:49:44.130193 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.130177    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:49:44.130193 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.130188    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:49:44.130524 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.130218    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-kube-api-access-gkqtn" (OuterVolumeSpecName: "kube-api-access-gkqtn") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "kube-api-access-gkqtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:49:44.135782 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.135753    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "774ff4f5-76fe-455f-9bb9-5c40c2d217ac" (UID: "774ff4f5-76fe-455f-9bb9-5c40c2d217ac"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228746    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-bound-sa-token\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228777    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-image-registry-private-configuration\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228790    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-installation-pull-secrets\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228800    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-tls\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228810    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-ca-trust-extracted\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228818    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-registry-certificates\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.228823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.228827    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkqtn\" (UniqueName: \"kubernetes.io/projected/774ff4f5-76fe-455f-9bb9-5c40c2d217ac-kube-api-access-gkqtn\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:49:44.629172 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.629135    2563 generic.go:358] "Generic (PLEG): container finished" podID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerID="26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08" exitCode=0
Jun 02 15:49:44.629700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.629200    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29"
Jun 02 15:49:44.629700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.629236    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" event={"ID":"774ff4f5-76fe-455f-9bb9-5c40c2d217ac","Type":"ContainerDied","Data":"26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08"}
Jun 02 15:49:44.629700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.629274    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c99f78fdd-9bq29" event={"ID":"774ff4f5-76fe-455f-9bb9-5c40c2d217ac","Type":"ContainerDied","Data":"a86c93e2e446edc37e8eadc2ba562521ba2eee1896ae73dc184b7b79e4950ade"}
Jun 02 15:49:44.629700 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.629289    2563 scope.go:117] "RemoveContainer" containerID="26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08"
Jun 02 15:49:44.629995 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.629976    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-88dbl"
Jun 02 15:49:44.637303 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.637283    2563 scope.go:117] "RemoveContainer" containerID="26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08"
Jun 02 15:49:44.637667 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:49:44.637641    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08\": container with ID starting with 26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08 not found: ID does not exist" containerID="26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08"
Jun 02 15:49:44.637724 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.637679    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08"} err="failed to get container status \"26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08\": rpc error: code = NotFound desc = could not find container \"26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08\": container with ID starting with 26d6b4c71362c805698b35d6c0eb537db85d537ba3f70c71b04901668a172a08 not found: ID does not exist"
Jun 02 15:49:44.652326 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.652305    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c99f78fdd-9bq29"]
Jun 02 15:49:44.657376 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:44.657355    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c99f78fdd-9bq29"]
Jun 02 15:49:45.981157 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:45.981125    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" path="/var/lib/kubelet/pods/774ff4f5-76fe-455f-9bb9-5c40c2d217ac/volumes"
Jun 02 15:49:46.977660 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:46.977604    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:49:49.143101 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:49.143070    2563 patch_prober.go:28] interesting pod/image-registry-fdd9c8df7-f9twg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 02 15:49:49.143493 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:49.143133    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg" podUID="6969dfd7-09e7-4fec-baba-a90816d3f83a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:49:50.497266 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:50.497237    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fdd9c8df7-f9twg"
Jun 02 15:49:54.636394 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:49:54.636357    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-88dbl"
Jun 02 15:51:06.718304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:06.718259    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:51:06.720818 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:06.720794    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aca2f9e4-d1e9-470c-b700-d6cc6f3be66c-metrics-certs\") pod \"network-metrics-daemon-5r476\" (UID: \"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c\") " pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:51:06.781559 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:06.781525    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j9fj4\""
Jun 02 15:51:06.789229 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:06.789201    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5r476"
Jun 02 15:51:06.921924 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:06.921721    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5r476"]
Jun 02 15:51:06.924648 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:51:06.924612    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca2f9e4_d1e9_470c_b700_d6cc6f3be66c.slice/crio-1f9555112240d0221e1fab870e67717e1da0b5f86a6e13fde85e50dfb81080f0 WatchSource:0}: Error finding container 1f9555112240d0221e1fab870e67717e1da0b5f86a6e13fde85e50dfb81080f0: Status 404 returned error can't find the container with id 1f9555112240d0221e1fab870e67717e1da0b5f86a6e13fde85e50dfb81080f0
Jun 02 15:51:07.860213 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:07.860151    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5r476" event={"ID":"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c","Type":"ContainerStarted","Data":"1f9555112240d0221e1fab870e67717e1da0b5f86a6e13fde85e50dfb81080f0"}
Jun 02 15:51:08.864578 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:08.864536    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5r476" event={"ID":"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c","Type":"ContainerStarted","Data":"d549c3a43933e85f9edd77c35d81240cd270ae9350f680ed8804e97896afd563"}
Jun 02 15:51:08.864578 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:08.864581    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5r476" event={"ID":"aca2f9e4-d1e9-470c-b700-d6cc6f3be66c","Type":"ContainerStarted","Data":"9c5599ee970ce582a91b883ca9884f4f682ac4e182e2df011d1dab425588e3b9"}
Jun 02 15:51:08.886078 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:08.885986    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5r476" podStartSLOduration=251.922796377 podStartE2EDuration="4m12.885968042s" podCreationTimestamp="2026-06-02 15:46:56 +0000 UTC" firstStartedPulling="2026-06-02 15:51:06.926420465 +0000 UTC m=+251.550209291" lastFinishedPulling="2026-06-02 15:51:07.889592116 +0000 UTC m=+252.513380956" observedRunningTime="2026-06-02 15:51:08.884099797 +0000 UTC m=+253.507888644" watchObservedRunningTime="2026-06-02 15:51:08.885968042 +0000 UTC m=+253.509756891"
Jun 02 15:51:43.627766 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.627719    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vrpfz"]
Jun 02 15:51:43.628385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.628201    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerName="registry"
Jun 02 15:51:43.628385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.628222    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerName="registry"
Jun 02 15:51:43.628385 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.628319    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="774ff4f5-76fe-455f-9bb9-5c40c2d217ac" containerName="registry"
Jun 02 15:51:43.630192 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.630171    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.634543 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.634522    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 02 15:51:43.641314 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.641289    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vrpfz"]
Jun 02 15:51:43.695097 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.695061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21e6a21a-a998-48cb-aac3-b9ab616597d2-kubelet-config\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.695097 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.695103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21e6a21a-a998-48cb-aac3-b9ab616597d2-original-pull-secret\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.695322 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.695207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21e6a21a-a998-48cb-aac3-b9ab616597d2-dbus\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.796224 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.796175    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21e6a21a-a998-48cb-aac3-b9ab616597d2-original-pull-secret\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.796405 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.796261    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21e6a21a-a998-48cb-aac3-b9ab616597d2-dbus\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.796405 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.796322    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21e6a21a-a998-48cb-aac3-b9ab616597d2-kubelet-config\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.796405 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.796392    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21e6a21a-a998-48cb-aac3-b9ab616597d2-kubelet-config\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.796516 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.796446    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21e6a21a-a998-48cb-aac3-b9ab616597d2-dbus\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.798611 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.798582    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21e6a21a-a998-48cb-aac3-b9ab616597d2-original-pull-secret\") pod \"global-pull-secret-syncer-vrpfz\" (UID: \"21e6a21a-a998-48cb-aac3-b9ab616597d2\") " pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:43.940270 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:43.940182    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vrpfz"
Jun 02 15:51:44.076193 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:44.076053    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vrpfz"]
Jun 02 15:51:44.078877 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:51:44.078837    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e6a21a_a998_48cb_aac3_b9ab616597d2.slice/crio-992590bd796ce4df00486f313b550bbd88728ffb7536bf57101e6daeefd35355 WatchSource:0}: Error finding container 992590bd796ce4df00486f313b550bbd88728ffb7536bf57101e6daeefd35355: Status 404 returned error can't find the container with id 992590bd796ce4df00486f313b550bbd88728ffb7536bf57101e6daeefd35355
Jun 02 15:51:44.967134 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:44.967090    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vrpfz" event={"ID":"21e6a21a-a998-48cb-aac3-b9ab616597d2","Type":"ContainerStarted","Data":"992590bd796ce4df00486f313b550bbd88728ffb7536bf57101e6daeefd35355"}
Jun 02 15:51:48.985199 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:48.985163    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vrpfz" event={"ID":"21e6a21a-a998-48cb-aac3-b9ab616597d2","Type":"ContainerStarted","Data":"2a6a40a5157fe4620dc571aba6b1e57f1cd3e7dfc9abd7531b4433049e19e6a3"}
Jun 02 15:51:49.004898 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:49.004842    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vrpfz" podStartSLOduration=2.151948787 podStartE2EDuration="6.004816352s" podCreationTimestamp="2026-06-02 15:51:43 +0000 UTC" firstStartedPulling="2026-06-02 15:51:44.080450325 +0000 UTC m=+288.704239149" lastFinishedPulling="2026-06-02 15:51:47.933317886 +0000 UTC m=+292.557106714" observedRunningTime="2026-06-02 15:51:49.004359117 +0000 UTC m=+293.628147963" watchObservedRunningTime="2026-06-02 15:51:49.004816352 +0000 UTC m=+293.628605195"
Jun 02 15:51:55.863367 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:51:55.863336    2563 kubelet.go:1628] "Image garbage collection succeeded"
Jun 02 15:52:36.164813 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.164770    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"]
Jun 02 15:52:36.167290 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.167266    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.171128 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.171105    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:52:36.171229 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.171166    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:52:36.172396 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.172372    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6jj48\""
Jun 02 15:52:36.187913 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.187893    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"]
Jun 02 15:52:36.307205 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.307164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p4hg\" (UniqueName: \"kubernetes.io/projected/04e5d53e-3c9c-4f80-86bf-694e61890df7-kube-api-access-8p4hg\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.307378 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.307303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.307378 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.307354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.408574 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.408517    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.408766 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.408588    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.408766 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.408619    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8p4hg\" (UniqueName: \"kubernetes.io/projected/04e5d53e-3c9c-4f80-86bf-694e61890df7-kube-api-access-8p4hg\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.408932 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.408908    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.408990 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.408935    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.419663 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.419594    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p4hg\" (UniqueName: \"kubernetes.io/projected/04e5d53e-3c9c-4f80-86bf-694e61890df7-kube-api-access-8p4hg\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.475940 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.475904    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:36.614351 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.614285    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"]
Jun 02 15:52:36.617813 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:52:36.617782    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e5d53e_3c9c_4f80_86bf_694e61890df7.slice/crio-62a66c64546bd3144040b43335daf72856967f9c0284d82e977be7b69b8feb0c WatchSource:0}: Error finding container 62a66c64546bd3144040b43335daf72856967f9c0284d82e977be7b69b8feb0c: Status 404 returned error can't find the container with id 62a66c64546bd3144040b43335daf72856967f9c0284d82e977be7b69b8feb0c
Jun 02 15:52:36.619655 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:36.619637    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 15:52:37.123496 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:37.123456    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" event={"ID":"04e5d53e-3c9c-4f80-86bf-694e61890df7","Type":"ContainerStarted","Data":"62a66c64546bd3144040b43335daf72856967f9c0284d82e977be7b69b8feb0c"}
Jun 02 15:52:43.143039 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:43.142989    2563 generic.go:358] "Generic (PLEG): container finished" podID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerID="11502d41081211150926570548cd5604d6837c5152d5875841bbbc2108d97475" exitCode=0
Jun 02 15:52:43.143527 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:43.143041    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" event={"ID":"04e5d53e-3c9c-4f80-86bf-694e61890df7","Type":"ContainerDied","Data":"11502d41081211150926570548cd5604d6837c5152d5875841bbbc2108d97475"}
Jun 02 15:52:45.151521 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:45.151482    2563 generic.go:358] "Generic (PLEG): container finished" podID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerID="b2a3a6218d3b259d07498bc5ae45cab837bbf0cdc5b279644825821316a5e9ae" exitCode=0
Jun 02 15:52:45.151521 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:45.151525    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" event={"ID":"04e5d53e-3c9c-4f80-86bf-694e61890df7","Type":"ContainerDied","Data":"b2a3a6218d3b259d07498bc5ae45cab837bbf0cdc5b279644825821316a5e9ae"}
Jun 02 15:52:52.174196 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:52.174163    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" event={"ID":"04e5d53e-3c9c-4f80-86bf-694e61890df7","Type":"ContainerStarted","Data":"4375060f83cf329a89cc389d75f30f7c9815b17e9efc4e75e61b57ddf8a98cd1"}
Jun 02 15:52:52.199622 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:52.199558    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" podStartSLOduration=0.751822264 podStartE2EDuration="16.19954277s" podCreationTimestamp="2026-06-02 15:52:36 +0000 UTC" firstStartedPulling="2026-06-02 15:52:36.619790662 +0000 UTC m=+341.243579487" lastFinishedPulling="2026-06-02 15:52:52.067511168 +0000 UTC m=+356.691299993" observedRunningTime="2026-06-02 15:52:52.197168443 +0000 UTC m=+356.820957289" watchObservedRunningTime="2026-06-02 15:52:52.19954277 +0000 UTC m=+356.823331617"
Jun 02 15:52:53.178834 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:53.178794    2563 generic.go:358] "Generic (PLEG): container finished" podID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerID="4375060f83cf329a89cc389d75f30f7c9815b17e9efc4e75e61b57ddf8a98cd1" exitCode=0
Jun 02 15:52:53.179203 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:53.178874    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" event={"ID":"04e5d53e-3c9c-4f80-86bf-694e61890df7","Type":"ContainerDied","Data":"4375060f83cf329a89cc389d75f30f7c9815b17e9efc4e75e61b57ddf8a98cd1"}
Jun 02 15:52:54.307421 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.307393    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:54.355793 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.355762    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-util\") pod \"04e5d53e-3c9c-4f80-86bf-694e61890df7\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") "
Jun 02 15:52:54.355960 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.355815    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p4hg\" (UniqueName: \"kubernetes.io/projected/04e5d53e-3c9c-4f80-86bf-694e61890df7-kube-api-access-8p4hg\") pod \"04e5d53e-3c9c-4f80-86bf-694e61890df7\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") "
Jun 02 15:52:54.355960 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.355832    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-bundle\") pod \"04e5d53e-3c9c-4f80-86bf-694e61890df7\" (UID: \"04e5d53e-3c9c-4f80-86bf-694e61890df7\") "
Jun 02 15:52:54.356580 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.356494    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-bundle" (OuterVolumeSpecName: "bundle") pod "04e5d53e-3c9c-4f80-86bf-694e61890df7" (UID: "04e5d53e-3c9c-4f80-86bf-694e61890df7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:52:54.358176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.358147    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e5d53e-3c9c-4f80-86bf-694e61890df7-kube-api-access-8p4hg" (OuterVolumeSpecName: "kube-api-access-8p4hg") pod "04e5d53e-3c9c-4f80-86bf-694e61890df7" (UID: "04e5d53e-3c9c-4f80-86bf-694e61890df7"). InnerVolumeSpecName "kube-api-access-8p4hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:52:54.360334 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.360308    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-util" (OuterVolumeSpecName: "util") pod "04e5d53e-3c9c-4f80-86bf-694e61890df7" (UID: "04e5d53e-3c9c-4f80-86bf-694e61890df7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:52:54.457120 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.457006    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:52:54.457120 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.457062    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p4hg\" (UniqueName: \"kubernetes.io/projected/04e5d53e-3c9c-4f80-86bf-694e61890df7-kube-api-access-8p4hg\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:52:54.457120 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:54.457073    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04e5d53e-3c9c-4f80-86bf-694e61890df7-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:52:55.185924 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:55.185887    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw" event={"ID":"04e5d53e-3c9c-4f80-86bf-694e61890df7","Type":"ContainerDied","Data":"62a66c64546bd3144040b43335daf72856967f9c0284d82e977be7b69b8feb0c"}
Jun 02 15:52:55.185924 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:55.185921    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62a66c64546bd3144040b43335daf72856967f9c0284d82e977be7b69b8feb0c"
Jun 02 15:52:55.186148 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:55.185936    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29c45qlw"
Jun 02 15:52:58.552126 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552091    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"]
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552387    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="pull"
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552397    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="pull"
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552415    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="util"
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552420    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="util"
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552425    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="extract"
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552430    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="extract"
Jun 02 15:52:58.552592 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.552490    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="04e5d53e-3c9c-4f80-86bf-694e61890df7" containerName="extract"
Jun 02 15:52:58.598058 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.597992    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"]
Jun 02 15:52:58.598239 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.598168    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.601760 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.601734    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jun 02 15:52:58.601760 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.601751    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jun 02 15:52:58.601947 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.601793    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-lx728\""
Jun 02 15:52:58.601947 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.601744    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jun 02 15:52:58.689665 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.689621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/bc3f613a-ca64-4d9e-a5d6-64ac22b5a655-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-hcszd\" (UID: \"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.689843 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.689733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mm2x\" (UniqueName: \"kubernetes.io/projected/bc3f613a-ca64-4d9e-a5d6-64ac22b5a655-kube-api-access-2mm2x\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-hcszd\" (UID: \"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.790602 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.790557    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/bc3f613a-ca64-4d9e-a5d6-64ac22b5a655-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-hcszd\" (UID: \"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.790797 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.790643    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2mm2x\" (UniqueName: \"kubernetes.io/projected/bc3f613a-ca64-4d9e-a5d6-64ac22b5a655-kube-api-access-2mm2x\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-hcszd\" (UID: \"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.793068 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.793046    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/bc3f613a-ca64-4d9e-a5d6-64ac22b5a655-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-hcszd\" (UID: \"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.800353 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.800324    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mm2x\" (UniqueName: \"kubernetes.io/projected/bc3f613a-ca64-4d9e-a5d6-64ac22b5a655-kube-api-access-2mm2x\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-hcszd\" (UID: \"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:58.907632 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:58.907546    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:52:59.064394 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:59.064282    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"]
Jun 02 15:52:59.067206 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:52:59.067182    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3f613a_ca64_4d9e_a5d6_64ac22b5a655.slice/crio-e9e8da1778247764d9dab14fb3829f69e92cd1e0cea891f5727c80f4c13637ef WatchSource:0}: Error finding container e9e8da1778247764d9dab14fb3829f69e92cd1e0cea891f5727c80f4c13637ef: Status 404 returned error can't find the container with id e9e8da1778247764d9dab14fb3829f69e92cd1e0cea891f5727c80f4c13637ef
Jun 02 15:52:59.198124 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:52:59.198034    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd" event={"ID":"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655","Type":"ContainerStarted","Data":"e9e8da1778247764d9dab14fb3829f69e92cd1e0cea891f5727c80f4c13637ef"}
Jun 02 15:53:05.225223 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.225129    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd" event={"ID":"bc3f613a-ca64-4d9e-a5d6-64ac22b5a655","Type":"ContainerStarted","Data":"209b707514ee3b1558f2c58f244cebcf67d5c398d8b7eb469a6a60b5dfd5d39d"}
Jun 02 15:53:05.225640 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.225272    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:53:05.249598 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.249543    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd" podStartSLOduration=1.4885489170000001 podStartE2EDuration="7.249527935s" podCreationTimestamp="2026-06-02 15:52:58 +0000 UTC" firstStartedPulling="2026-06-02 15:52:59.06883694 +0000 UTC m=+363.692625765" lastFinishedPulling="2026-06-02 15:53:04.829815959 +0000 UTC m=+369.453604783" observedRunningTime="2026-06-02 15:53:05.246619668 +0000 UTC m=+369.870408528" watchObservedRunningTime="2026-06-02 15:53:05.249527935 +0000 UTC m=+369.873316782"
Jun 02 15:53:05.405680 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.405650    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-995z8"]
Jun 02 15:53:05.407853 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.407834    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.410832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.410811    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jun 02 15:53:05.410832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.410825    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jun 02 15:53:05.410997 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.410827    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-5sct4\""
Jun 02 15:53:05.417145 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.417120    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-995z8"]
Jun 02 15:53:05.546635 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.546541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zlxn\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-kube-api-access-4zlxn\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.546635 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.546585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.546635 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.546607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-cabundle0\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.647322 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.647282    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4zlxn\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-kube-api-access-4zlxn\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.647322 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.647326    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.647532 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.647355    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-cabundle0\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.647532 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.647443    2563 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:53:05.647532 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.647466    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:53:05.647532 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.647475    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-995z8: references non-existent secret key: ca.crt
Jun 02 15:53:05.647532 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.647529    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates podName:3ce0c767-e5d6-4d49-8d32-cd29205a5c17 nodeName:}" failed. No retries permitted until 2026-06-02 15:53:06.147513889 +0000 UTC m=+370.771302715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates") pod "keda-operator-ffbb595cb-995z8" (UID: "3ce0c767-e5d6-4d49-8d32-cd29205a5c17") : references non-existent secret key: ca.crt
Jun 02 15:53:05.648030 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.647988    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-cabundle0\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.657164 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.657141    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zlxn\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-kube-api-access-4zlxn\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:05.749911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.749877    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"]
Jun 02 15:53:05.752266 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.752248    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.757371 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.757348    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jun 02 15:53:05.763143 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.763120    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"]
Jun 02 15:53:05.849555 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.849513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e4a71959-0059-497e-927a-c87b50c2407f-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.849555 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.849555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.849764 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.849575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlxnh\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-kube-api-access-wlxnh\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.950401 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.950362    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e4a71959-0059-497e-927a-c87b50c2407f-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.950401 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.950407    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.950661 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.950427    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wlxnh\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-kube-api-access-wlxnh\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.950661 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.950540    2563 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:53:05.950661 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.950560    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:53:05.950661 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.950576    2563 projected.go:264] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jun 02 15:53:05.950661 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.950599    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jun 02 15:53:05.950661 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:05.950661    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates podName:e4a71959-0059-497e-927a-c87b50c2407f nodeName:}" failed. No retries permitted until 2026-06-02 15:53:06.450642703 +0000 UTC m=+371.074431542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates") pod "keda-metrics-apiserver-7c9f485588-m5vbp" (UID: "e4a71959-0059-497e-927a-c87b50c2407f") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jun 02 15:53:05.950848 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.950770    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e4a71959-0059-497e-927a-c87b50c2407f-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:05.967802 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:05.967776    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlxnh\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-kube-api-access-wlxnh\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:06.151422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:06.151318    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:06.151578 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.151475    2563 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:53:06.151578 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.151496    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:53:06.151578 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.151506    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-995z8: references non-existent secret key: ca.crt
Jun 02 15:53:06.151578 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.151563    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates podName:3ce0c767-e5d6-4d49-8d32-cd29205a5c17 nodeName:}" failed. No retries permitted until 2026-06-02 15:53:07.151547736 +0000 UTC m=+371.775336560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates") pod "keda-operator-ffbb595cb-995z8" (UID: "3ce0c767-e5d6-4d49-8d32-cd29205a5c17") : references non-existent secret key: ca.crt
Jun 02 15:53:06.453750 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:06.453644    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:06.454174 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.453789    2563 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:53:06.454174 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.453815    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:53:06.454174 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.453833    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp: references non-existent secret key: tls.crt
Jun 02 15:53:06.454174 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:06.453886    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates podName:e4a71959-0059-497e-927a-c87b50c2407f nodeName:}" failed. No retries permitted until 2026-06-02 15:53:07.453871719 +0000 UTC m=+372.077660545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates") pod "keda-metrics-apiserver-7c9f485588-m5vbp" (UID: "e4a71959-0059-497e-927a-c87b50c2407f") : references non-existent secret key: tls.crt
Jun 02 15:53:07.159245 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:07.159198    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:07.159431 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.159354    2563 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:53:07.159431 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.159378    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:53:07.159431 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.159387    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-995z8: references non-existent secret key: ca.crt
Jun 02 15:53:07.159531 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.159456    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates podName:3ce0c767-e5d6-4d49-8d32-cd29205a5c17 nodeName:}" failed. No retries permitted until 2026-06-02 15:53:09.159437576 +0000 UTC m=+373.783226403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates") pod "keda-operator-ffbb595cb-995z8" (UID: "3ce0c767-e5d6-4d49-8d32-cd29205a5c17") : references non-existent secret key: ca.crt
Jun 02 15:53:07.462799 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:07.462689    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:07.463202 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.462834    2563 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:53:07.463202 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.462856    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:53:07.463202 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.462874    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp: references non-existent secret key: tls.crt
Jun 02 15:53:07.463202 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:07.462927    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates podName:e4a71959-0059-497e-927a-c87b50c2407f nodeName:}" failed. No retries permitted until 2026-06-02 15:53:09.462914342 +0000 UTC m=+374.086703167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates") pod "keda-metrics-apiserver-7c9f485588-m5vbp" (UID: "e4a71959-0059-497e-927a-c87b50c2407f") : references non-existent secret key: tls.crt
Jun 02 15:53:09.177478 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:09.177420    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:09.177869 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.177567    2563 secret.go:281] references non-existent secret key: ca.crt
Jun 02 15:53:09.177869 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.177590    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jun 02 15:53:09.177869 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.177600    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-995z8: references non-existent secret key: ca.crt
Jun 02 15:53:09.177869 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.177653    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates podName:3ce0c767-e5d6-4d49-8d32-cd29205a5c17 nodeName:}" failed. No retries permitted until 2026-06-02 15:53:13.177637831 +0000 UTC m=+377.801426655 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates") pod "keda-operator-ffbb595cb-995z8" (UID: "3ce0c767-e5d6-4d49-8d32-cd29205a5c17") : references non-existent secret key: ca.crt
Jun 02 15:53:09.480502 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:09.480383    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:09.480679 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.480548    2563 secret.go:281] references non-existent secret key: tls.crt
Jun 02 15:53:09.480679 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.480574    2563 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jun 02 15:53:09.480679 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.480597    2563 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp: references non-existent secret key: tls.crt
Jun 02 15:53:09.480679 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:53:09.480670    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates podName:e4a71959-0059-497e-927a-c87b50c2407f nodeName:}" failed. No retries permitted until 2026-06-02 15:53:13.48065165 +0000 UTC m=+378.104440488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates") pod "keda-metrics-apiserver-7c9f485588-m5vbp" (UID: "e4a71959-0059-497e-927a-c87b50c2407f") : references non-existent secret key: tls.crt
Jun 02 15:53:13.212467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.212424    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:13.215022 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.214985    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3ce0c767-e5d6-4d49-8d32-cd29205a5c17-certificates\") pod \"keda-operator-ffbb595cb-995z8\" (UID: \"3ce0c767-e5d6-4d49-8d32-cd29205a5c17\") " pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:13.217844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.217822    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:13.364165 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.364128    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-995z8"]
Jun 02 15:53:13.367498 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:53:13.367466    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce0c767_e5d6_4d49_8d32_cd29205a5c17.slice/crio-23546e08809b2a613dd221701fd570e5f781cba991a11949736d5101d9e9f097 WatchSource:0}: Error finding container 23546e08809b2a613dd221701fd570e5f781cba991a11949736d5101d9e9f097: Status 404 returned error can't find the container with id 23546e08809b2a613dd221701fd570e5f781cba991a11949736d5101d9e9f097
Jun 02 15:53:13.514716 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.514616    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:13.517515 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.517474    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4a71959-0059-497e-927a-c87b50c2407f-certificates\") pod \"keda-metrics-apiserver-7c9f485588-m5vbp\" (UID: \"e4a71959-0059-497e-927a-c87b50c2407f\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:13.563260 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.563216    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:13.704216 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:13.704185    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"]
Jun 02 15:53:13.708358 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:53:13.708325    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a71959_0059_497e_927a_c87b50c2407f.slice/crio-3c35ffa311a735823f7fe13ae2281da1f2dce1d68715bad3048a6e5163bc646b WatchSource:0}: Error finding container 3c35ffa311a735823f7fe13ae2281da1f2dce1d68715bad3048a6e5163bc646b: Status 404 returned error can't find the container with id 3c35ffa311a735823f7fe13ae2281da1f2dce1d68715bad3048a6e5163bc646b
Jun 02 15:53:14.254811 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:14.254777    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-995z8" event={"ID":"3ce0c767-e5d6-4d49-8d32-cd29205a5c17","Type":"ContainerStarted","Data":"23546e08809b2a613dd221701fd570e5f781cba991a11949736d5101d9e9f097"}
Jun 02 15:53:14.255712 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:14.255689    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp" event={"ID":"e4a71959-0059-497e-927a-c87b50c2407f","Type":"ContainerStarted","Data":"3c35ffa311a735823f7fe13ae2281da1f2dce1d68715bad3048a6e5163bc646b"}
Jun 02 15:53:17.272908 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:17.272873    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp" event={"ID":"e4a71959-0059-497e-927a-c87b50c2407f","Type":"ContainerStarted","Data":"8162b05c0903c9572af349c50b303c1c8f957d1342b0336d2f02a52b520274ca"}
Jun 02 15:53:17.273295 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:17.272938    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:17.299023 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:17.298953    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp" podStartSLOduration=8.898886928 podStartE2EDuration="12.298938567s" podCreationTimestamp="2026-06-02 15:53:05 +0000 UTC" firstStartedPulling="2026-06-02 15:53:13.709620315 +0000 UTC m=+378.333409140" lastFinishedPulling="2026-06-02 15:53:17.109671943 +0000 UTC m=+381.733460779" observedRunningTime="2026-06-02 15:53:17.296847597 +0000 UTC m=+381.920636445" watchObservedRunningTime="2026-06-02 15:53:17.298938567 +0000 UTC m=+381.922727414"
Jun 02 15:53:19.280778 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:19.280738    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-995z8" event={"ID":"3ce0c767-e5d6-4d49-8d32-cd29205a5c17","Type":"ContainerStarted","Data":"86db841d5fe996c50b2c778321ace50157bc4325cd67a82ea46e80dc5edb2f5a"}
Jun 02 15:53:19.281184 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:19.280867    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:19.305608 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:19.305551    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-995z8" podStartSLOduration=8.537392005 podStartE2EDuration="14.305535938s" podCreationTimestamp="2026-06-02 15:53:05 +0000 UTC" firstStartedPulling="2026-06-02 15:53:13.368686014 +0000 UTC m=+377.992474839" lastFinishedPulling="2026-06-02 15:53:19.136829944 +0000 UTC m=+383.760618772" observedRunningTime="2026-06-02 15:53:19.304487319 +0000 UTC m=+383.928276167" watchObservedRunningTime="2026-06-02 15:53:19.305535938 +0000 UTC m=+383.929324784"
Jun 02 15:53:26.231301 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:26.231271    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-hcszd"
Jun 02 15:53:28.280322 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:28.280292    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-m5vbp"
Jun 02 15:53:40.286915 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:40.286881    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-995z8"
Jun 02 15:53:56.417671 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.417630    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"]
Jun 02 15:53:56.423633 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.423606    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.426471 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.426449    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:53:56.426471 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.426461    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6jj48\""
Jun 02 15:53:56.427926 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.427910    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:53:56.430718 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.430653    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"]
Jun 02 15:53:56.455585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.455555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.455585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.455593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlxdl\" (UniqueName: \"kubernetes.io/projected/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-kube-api-access-tlxdl\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.455794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.455626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.556784 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.556748    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.556784 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.556786    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tlxdl\" (UniqueName: \"kubernetes.io/projected/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-kube-api-access-tlxdl\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.556982 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.556819    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.557184 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.557169    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.557239 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.557221    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.566733 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.566696    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlxdl\" (UniqueName: \"kubernetes.io/projected/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-kube-api-access-tlxdl\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.733747 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.733662    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:53:56.866215 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:56.866186    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"]
Jun 02 15:53:56.868745 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:53:56.868714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f09f875_aca9_459f_9dfa_0ab4633cb1b5.slice/crio-4022b35497d3e8d7d8fa811d342f144ce891e17df31fda2adb5c7197d5a9f969 WatchSource:0}: Error finding container 4022b35497d3e8d7d8fa811d342f144ce891e17df31fda2adb5c7197d5a9f969: Status 404 returned error can't find the container with id 4022b35497d3e8d7d8fa811d342f144ce891e17df31fda2adb5c7197d5a9f969
Jun 02 15:53:57.398671 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:57.398637    2563 generic.go:358] "Generic (PLEG): container finished" podID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerID="e59e635e69f5e37a514e3b18ad88290919a21fc6b981f1385bc311558a49216d" exitCode=0
Jun 02 15:53:57.398850 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:57.398732    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5" event={"ID":"8f09f875-aca9-459f-9dfa-0ab4633cb1b5","Type":"ContainerDied","Data":"e59e635e69f5e37a514e3b18ad88290919a21fc6b981f1385bc311558a49216d"}
Jun 02 15:53:57.398850 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:53:57.398780    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5" event={"ID":"8f09f875-aca9-459f-9dfa-0ab4633cb1b5","Type":"ContainerStarted","Data":"4022b35497d3e8d7d8fa811d342f144ce891e17df31fda2adb5c7197d5a9f969"}
Jun 02 15:54:00.410946 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:00.410909    2563 generic.go:358] "Generic (PLEG): container finished" podID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerID="fe49a3a497cc31345a1a19dd46b01bbd6e63a2b4b21eb877fc1dd730880f2f95" exitCode=0
Jun 02 15:54:00.411443 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:00.410986    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5" event={"ID":"8f09f875-aca9-459f-9dfa-0ab4633cb1b5","Type":"ContainerDied","Data":"fe49a3a497cc31345a1a19dd46b01bbd6e63a2b4b21eb877fc1dd730880f2f95"}
Jun 02 15:54:01.416715 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:01.416668    2563 generic.go:358] "Generic (PLEG): container finished" podID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerID="8effdc87e2dbaccb87f794dbfbe70845dc1d094c18ed4961b2fffe1451f4cfc8" exitCode=0
Jun 02 15:54:01.416715 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:01.416704    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5" event={"ID":"8f09f875-aca9-459f-9dfa-0ab4633cb1b5","Type":"ContainerDied","Data":"8effdc87e2dbaccb87f794dbfbe70845dc1d094c18ed4961b2fffe1451f4cfc8"}
Jun 02 15:54:02.548238 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.548210    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:54:02.607844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.607808    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlxdl\" (UniqueName: \"kubernetes.io/projected/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-kube-api-access-tlxdl\") pod \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") "
Jun 02 15:54:02.608041 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.607864    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-util\") pod \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") "
Jun 02 15:54:02.608041 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.607918    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-bundle\") pod \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\" (UID: \"8f09f875-aca9-459f-9dfa-0ab4633cb1b5\") "
Jun 02 15:54:02.608596 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.608567    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-bundle" (OuterVolumeSpecName: "bundle") pod "8f09f875-aca9-459f-9dfa-0ab4633cb1b5" (UID: "8f09f875-aca9-459f-9dfa-0ab4633cb1b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:54:02.610186 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.610161    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-kube-api-access-tlxdl" (OuterVolumeSpecName: "kube-api-access-tlxdl") pod "8f09f875-aca9-459f-9dfa-0ab4633cb1b5" (UID: "8f09f875-aca9-459f-9dfa-0ab4633cb1b5"). InnerVolumeSpecName "kube-api-access-tlxdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:54:02.612563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.612525    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-util" (OuterVolumeSpecName: "util") pod "8f09f875-aca9-459f-9dfa-0ab4633cb1b5" (UID: "8f09f875-aca9-459f-9dfa-0ab4633cb1b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:54:02.709483 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.709393    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:02.709483 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.709429    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlxdl\" (UniqueName: \"kubernetes.io/projected/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-kube-api-access-tlxdl\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:02.709483 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:02.709440    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f09f875-aca9-459f-9dfa-0ab4633cb1b5-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:03.425317 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:03.425281    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5"
Jun 02 15:54:03.425511 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:03.425290    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dpf7p5" event={"ID":"8f09f875-aca9-459f-9dfa-0ab4633cb1b5","Type":"ContainerDied","Data":"4022b35497d3e8d7d8fa811d342f144ce891e17df31fda2adb5c7197d5a9f969"}
Jun 02 15:54:03.425511 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:03.425397    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4022b35497d3e8d7d8fa811d342f144ce891e17df31fda2adb5c7197d5a9f969"
Jun 02 15:54:09.629422 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629381    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"]
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629685    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="util"
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629696    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="util"
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629709    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="extract"
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629715    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="extract"
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629723    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="pull"
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629728    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="pull"
Jun 02 15:54:09.629910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.629780    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="8f09f875-aca9-459f-9dfa-0ab4633cb1b5" containerName="extract"
Jun 02 15:54:09.633912 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.633882    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.637354 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.637324    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-j2qbt\""
Jun 02 15:54:09.637469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.637410    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jun 02 15:54:09.637630 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.637616    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jun 02 15:54:09.651917 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.651754    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"]
Jun 02 15:54:09.771644 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.771607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a998d4a0-040e-42ed-86e2-40d1211c5c5f-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-dvj48\" (UID: \"a998d4a0-040e-42ed-86e2-40d1211c5c5f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.771814 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.771663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvjrm\" (UniqueName: \"kubernetes.io/projected/a998d4a0-040e-42ed-86e2-40d1211c5c5f-kube-api-access-zvjrm\") pod \"cert-manager-operator-controller-manager-54b9655956-dvj48\" (UID: \"a998d4a0-040e-42ed-86e2-40d1211c5c5f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.872226 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.872188    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a998d4a0-040e-42ed-86e2-40d1211c5c5f-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-dvj48\" (UID: \"a998d4a0-040e-42ed-86e2-40d1211c5c5f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.872421 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.872242    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zvjrm\" (UniqueName: \"kubernetes.io/projected/a998d4a0-040e-42ed-86e2-40d1211c5c5f-kube-api-access-zvjrm\") pod \"cert-manager-operator-controller-manager-54b9655956-dvj48\" (UID: \"a998d4a0-040e-42ed-86e2-40d1211c5c5f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.872595 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.872574    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a998d4a0-040e-42ed-86e2-40d1211c5c5f-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-dvj48\" (UID: \"a998d4a0-040e-42ed-86e2-40d1211c5c5f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.893456 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.893387    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvjrm\" (UniqueName: \"kubernetes.io/projected/a998d4a0-040e-42ed-86e2-40d1211c5c5f-kube-api-access-zvjrm\") pod \"cert-manager-operator-controller-manager-54b9655956-dvj48\" (UID: \"a998d4a0-040e-42ed-86e2-40d1211c5c5f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:09.943398 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:09.943359    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"
Jun 02 15:54:10.100872 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:10.100833    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48"]
Jun 02 15:54:10.102040 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:54:10.101986    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda998d4a0_040e_42ed_86e2_40d1211c5c5f.slice/crio-049c94041f3ff1d2ae19d34fe0c1f9444933d5a2e0db118f3a7581d724db0239 WatchSource:0}: Error finding container 049c94041f3ff1d2ae19d34fe0c1f9444933d5a2e0db118f3a7581d724db0239: Status 404 returned error can't find the container with id 049c94041f3ff1d2ae19d34fe0c1f9444933d5a2e0db118f3a7581d724db0239
Jun 02 15:54:10.447503 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:10.447468    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48" event={"ID":"a998d4a0-040e-42ed-86e2-40d1211c5c5f","Type":"ContainerStarted","Data":"049c94041f3ff1d2ae19d34fe0c1f9444933d5a2e0db118f3a7581d724db0239"}
Jun 02 15:54:12.455842 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:12.455800    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48" event={"ID":"a998d4a0-040e-42ed-86e2-40d1211c5c5f","Type":"ContainerStarted","Data":"00d6d8dafd820e0c5e01b06ab9abdea79421925b5ede46239e6954b07455924d"}
Jun 02 15:54:12.486889 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:12.486841    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-dvj48" podStartSLOduration=1.7454688090000001 podStartE2EDuration="3.486825862s" podCreationTimestamp="2026-06-02 15:54:09 +0000 UTC" firstStartedPulling="2026-06-02 15:54:10.104497194 +0000 UTC m=+434.728286020" lastFinishedPulling="2026-06-02 15:54:11.845854244 +0000 UTC m=+436.469643073" observedRunningTime="2026-06-02 15:54:12.485080781 +0000 UTC m=+437.108869628" watchObservedRunningTime="2026-06-02 15:54:12.486825862 +0000 UTC m=+437.110614710"
Jun 02 15:54:18.660655 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.660619    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"]
Jun 02 15:54:18.664265 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.664240    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.667742 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.667689    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:54:18.667859 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.667814    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6jj48\""
Jun 02 15:54:18.668267 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.668248    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:54:18.678901 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.678874    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"]
Jun 02 15:54:18.747043 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.746980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz64g\" (UniqueName: \"kubernetes.io/projected/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-kube-api-access-dz64g\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.747043 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.747044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.747294 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.747177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.847738 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.847697    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.847905 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.847750    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.847905 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.847813    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dz64g\" (UniqueName: \"kubernetes.io/projected/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-kube-api-access-dz64g\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.848122 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.848101    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.848203 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.848182    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.870201 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.870166    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz64g\" (UniqueName: \"kubernetes.io/projected/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-kube-api-access-dz64g\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:18.973320 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:18.973228    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:19.110829 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:19.110803    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"]
Jun 02 15:54:19.112964 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:54:19.112937    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb48b192f_0e8d_4dfe_8897_85b728f0e5a4.slice/crio-b7d272f356d2d2e7beda7372018c2a932d5806353f11740699c092d3bc617946 WatchSource:0}: Error finding container b7d272f356d2d2e7beda7372018c2a932d5806353f11740699c092d3bc617946: Status 404 returned error can't find the container with id b7d272f356d2d2e7beda7372018c2a932d5806353f11740699c092d3bc617946
Jun 02 15:54:19.479755 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:19.479717    2563 generic.go:358] "Generic (PLEG): container finished" podID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerID="787b6cb6579ac515cbdc533c77614d2a547dba89aa71c626e99a4421e727a6c0" exitCode=0
Jun 02 15:54:19.479942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:19.479772    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l" event={"ID":"b48b192f-0e8d-4dfe-8897-85b728f0e5a4","Type":"ContainerDied","Data":"787b6cb6579ac515cbdc533c77614d2a547dba89aa71c626e99a4421e727a6c0"}
Jun 02 15:54:19.479942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:19.479794    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l" event={"ID":"b48b192f-0e8d-4dfe-8897-85b728f0e5a4","Type":"ContainerStarted","Data":"b7d272f356d2d2e7beda7372018c2a932d5806353f11740699c092d3bc617946"}
Jun 02 15:54:22.490836 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:22.490801    2563 generic.go:358] "Generic (PLEG): container finished" podID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerID="8b60f04ff0dddf784b413b53d9231157c6aa1f39fd0e4dc4e51decacd2c44b94" exitCode=0
Jun 02 15:54:22.491256 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:22.490852    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l" event={"ID":"b48b192f-0e8d-4dfe-8897-85b728f0e5a4","Type":"ContainerDied","Data":"8b60f04ff0dddf784b413b53d9231157c6aa1f39fd0e4dc4e51decacd2c44b94"}
Jun 02 15:54:23.495962 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:23.495930    2563 generic.go:358] "Generic (PLEG): container finished" podID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerID="e980c37a7107677f32bcb363933c2d4706e2acf4af7bfd5d11ad3b9710f07a92" exitCode=0
Jun 02 15:54:23.496352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:23.495968    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l" event={"ID":"b48b192f-0e8d-4dfe-8897-85b728f0e5a4","Type":"ContainerDied","Data":"e980c37a7107677f32bcb363933c2d4706e2acf4af7bfd5d11ad3b9710f07a92"}
Jun 02 15:54:24.628618 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.628595    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:24.697956 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.697915    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz64g\" (UniqueName: \"kubernetes.io/projected/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-kube-api-access-dz64g\") pod \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") "
Jun 02 15:54:24.698177 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.697999    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-bundle\") pod \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") "
Jun 02 15:54:24.698177 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.698063    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-util\") pod \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\" (UID: \"b48b192f-0e8d-4dfe-8897-85b728f0e5a4\") "
Jun 02 15:54:24.698456 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.698425    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-bundle" (OuterVolumeSpecName: "bundle") pod "b48b192f-0e8d-4dfe-8897-85b728f0e5a4" (UID: "b48b192f-0e8d-4dfe-8897-85b728f0e5a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:54:24.700373 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.700340    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-kube-api-access-dz64g" (OuterVolumeSpecName: "kube-api-access-dz64g") pod "b48b192f-0e8d-4dfe-8897-85b728f0e5a4" (UID: "b48b192f-0e8d-4dfe-8897-85b728f0e5a4"). InnerVolumeSpecName "kube-api-access-dz64g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:54:24.701925 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.701902    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-util" (OuterVolumeSpecName: "util") pod "b48b192f-0e8d-4dfe-8897-85b728f0e5a4" (UID: "b48b192f-0e8d-4dfe-8897-85b728f0e5a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:54:24.798677 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.798585    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz64g\" (UniqueName: \"kubernetes.io/projected/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-kube-api-access-dz64g\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:24.798677 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.798616    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:24.798677 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:24.798626    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b48b192f-0e8d-4dfe-8897-85b728f0e5a4-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:25.503320 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:25.503283    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l" event={"ID":"b48b192f-0e8d-4dfe-8897-85b728f0e5a4","Type":"ContainerDied","Data":"b7d272f356d2d2e7beda7372018c2a932d5806353f11740699c092d3bc617946"}
Jun 02 15:54:25.503320 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:25.503316    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7d272f356d2d2e7beda7372018c2a932d5806353f11740699c092d3bc617946"
Jun 02 15:54:25.503579 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:25.503360    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgdx9l"
Jun 02 15:54:46.601368 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601319    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"]
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601646    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="util"
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601657    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="util"
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601675    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="pull"
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="pull"
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601688    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="extract"
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601694    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="extract"
Jun 02 15:54:46.601854 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.601747    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="b48b192f-0e8d-4dfe-8897-85b728f0e5a4" containerName="extract"
Jun 02 15:54:46.604739 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.604721    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.607928 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.607906    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:54:46.609277 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.609256    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6jj48\""
Jun 02 15:54:46.609372 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.609318    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:54:46.618722 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.618693    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"]
Jun 02 15:54:46.666573 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.666534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.666776 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.666585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r7zx\" (UniqueName: \"kubernetes.io/projected/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-kube-api-access-2r7zx\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.666776 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.666667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.767285 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.767238    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.767285 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.767297    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2r7zx\" (UniqueName: \"kubernetes.io/projected/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-kube-api-access-2r7zx\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.767545 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.767320    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.767711 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.767689    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.767776 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.767698    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.778028 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.777984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r7zx\" (UniqueName: \"kubernetes.io/projected/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-kube-api-access-2r7zx\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:46.913752 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:46.913649    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:47.074735 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:54:47.074704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb039aa7_dabe_4be9_a425_dcf0d55c6da0.slice/crio-ae4d81e2676b1cfdc573b0d82d7c85bed77c58056dfbecd9c4ecd5184b193dce WatchSource:0}: Error finding container ae4d81e2676b1cfdc573b0d82d7c85bed77c58056dfbecd9c4ecd5184b193dce: Status 404 returned error can't find the container with id ae4d81e2676b1cfdc573b0d82d7c85bed77c58056dfbecd9c4ecd5184b193dce
Jun 02 15:54:47.083094 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:47.083058    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"]
Jun 02 15:54:47.573377 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:47.573331    2563 generic.go:358] "Generic (PLEG): container finished" podID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerID="7426378a20de691fe5922209e00d18c908b72eaa5b54429939e785f5416d2cf0" exitCode=0
Jun 02 15:54:47.573559 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:47.573395    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk" event={"ID":"cb039aa7-dabe-4be9-a425-dcf0d55c6da0","Type":"ContainerDied","Data":"7426378a20de691fe5922209e00d18c908b72eaa5b54429939e785f5416d2cf0"}
Jun 02 15:54:47.573559 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:47.573418    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk" event={"ID":"cb039aa7-dabe-4be9-a425-dcf0d55c6da0","Type":"ContainerStarted","Data":"ae4d81e2676b1cfdc573b0d82d7c85bed77c58056dfbecd9c4ecd5184b193dce"}
Jun 02 15:54:49.581552 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:49.581517    2563 generic.go:358] "Generic (PLEG): container finished" podID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerID="fcbefe406a4a3c2dc3df46e01f2b15b32d671332c3a4c264a3f0f44b4a3d1735" exitCode=0
Jun 02 15:54:49.581935 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:49.581605    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk" event={"ID":"cb039aa7-dabe-4be9-a425-dcf0d55c6da0","Type":"ContainerDied","Data":"fcbefe406a4a3c2dc3df46e01f2b15b32d671332c3a4c264a3f0f44b4a3d1735"}
Jun 02 15:54:50.586469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:50.586438    2563 generic.go:358] "Generic (PLEG): container finished" podID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerID="d181030a6d26a74c0bc224473fa9306de96456ea3d3843df15f925570a52a87d" exitCode=0
Jun 02 15:54:50.586871 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:50.586523    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk" event={"ID":"cb039aa7-dabe-4be9-a425-dcf0d55c6da0","Type":"ContainerDied","Data":"d181030a6d26a74c0bc224473fa9306de96456ea3d3843df15f925570a52a87d"}
Jun 02 15:54:51.716901 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.716873    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:51.803135 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.803082    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r7zx\" (UniqueName: \"kubernetes.io/projected/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-kube-api-access-2r7zx\") pod \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") "
Jun 02 15:54:51.803322 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.803166    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-bundle\") pod \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") "
Jun 02 15:54:51.803322 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.803216    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-util\") pod \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\" (UID: \"cb039aa7-dabe-4be9-a425-dcf0d55c6da0\") "
Jun 02 15:54:51.804047 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.803993    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-bundle" (OuterVolumeSpecName: "bundle") pod "cb039aa7-dabe-4be9-a425-dcf0d55c6da0" (UID: "cb039aa7-dabe-4be9-a425-dcf0d55c6da0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:54:51.805347 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.805325    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-kube-api-access-2r7zx" (OuterVolumeSpecName: "kube-api-access-2r7zx") pod "cb039aa7-dabe-4be9-a425-dcf0d55c6da0" (UID: "cb039aa7-dabe-4be9-a425-dcf0d55c6da0"). InnerVolumeSpecName "kube-api-access-2r7zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:54:51.808745 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.808721    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-util" (OuterVolumeSpecName: "util") pod "cb039aa7-dabe-4be9-a425-dcf0d55c6da0" (UID: "cb039aa7-dabe-4be9-a425-dcf0d55c6da0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:54:51.903806 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.903715    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r7zx\" (UniqueName: \"kubernetes.io/projected/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-kube-api-access-2r7zx\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:51.903806 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.903747    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:51.903806 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:51.903757    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb039aa7-dabe-4be9-a425-dcf0d55c6da0-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:54:52.594993 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:52.594961    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk"
Jun 02 15:54:52.595186 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:52.594960    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835dhnhk" event={"ID":"cb039aa7-dabe-4be9-a425-dcf0d55c6da0","Type":"ContainerDied","Data":"ae4d81e2676b1cfdc573b0d82d7c85bed77c58056dfbecd9c4ecd5184b193dce"}
Jun 02 15:54:52.595186 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:54:52.595089    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae4d81e2676b1cfdc573b0d82d7c85bed77c58056dfbecd9c4ecd5184b193dce"
Jun 02 15:55:01.351879 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.351837    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"]
Jun 02 15:55:01.352415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352336    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="extract"
Jun 02 15:55:01.352415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352357    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="extract"
Jun 02 15:55:01.352415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352373    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="util"
Jun 02 15:55:01.352415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352381    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="util"
Jun 02 15:55:01.352415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352396    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="pull"
Jun 02 15:55:01.352415 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352403    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="pull"
Jun 02 15:55:01.352715 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.352497    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="cb039aa7-dabe-4be9-a425-dcf0d55c6da0" containerName="extract"
Jun 02 15:55:01.356920 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.356897    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.361087 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.361060    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:55:01.361659 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.361643    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6jj48\""
Jun 02 15:55:01.362634 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.362614    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:55:01.371293 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.371265    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"]
Jun 02 15:55:01.379587 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.379559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.379711 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.379666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.379771 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.379731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkq9c\" (UniqueName: \"kubernetes.io/projected/ffa674c0-637e-4cb6-a58c-90721837920c-kube-api-access-nkq9c\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.480260 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.480205    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.480260 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.480264    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nkq9c\" (UniqueName: \"kubernetes.io/projected/ffa674c0-637e-4cb6-a58c-90721837920c-kube-api-access-nkq9c\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.480480 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.480299    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.480649 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.480630    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.480733 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.480652    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.512839 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.512801    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkq9c\" (UniqueName: \"kubernetes.io/projected/ffa674c0-637e-4cb6-a58c-90721837920c-kube-api-access-nkq9c\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.666777 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.666688    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:01.800870 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:01.800846    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"]
Jun 02 15:55:01.802933 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:01.802898    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa674c0_637e_4cb6_a58c_90721837920c.slice/crio-79d1562ad9501d49ccb2001fbdeb3ca44c15d1bf5d0485562957a7146cc99ca7 WatchSource:0}: Error finding container 79d1562ad9501d49ccb2001fbdeb3ca44c15d1bf5d0485562957a7146cc99ca7: Status 404 returned error can't find the container with id 79d1562ad9501d49ccb2001fbdeb3ca44c15d1bf5d0485562957a7146cc99ca7
Jun 02 15:55:02.628903 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:02.628870    2563 generic.go:358] "Generic (PLEG): container finished" podID="ffa674c0-637e-4cb6-a58c-90721837920c" containerID="76f97c2f2b4b17363d044961d578ac842c5d75cab0195cae1a70bdc247d360db" exitCode=0
Jun 02 15:55:02.629395 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:02.628914    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv" event={"ID":"ffa674c0-637e-4cb6-a58c-90721837920c","Type":"ContainerDied","Data":"76f97c2f2b4b17363d044961d578ac842c5d75cab0195cae1a70bdc247d360db"}
Jun 02 15:55:02.629395 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:02.628936    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv" event={"ID":"ffa674c0-637e-4cb6-a58c-90721837920c","Type":"ContainerStarted","Data":"79d1562ad9501d49ccb2001fbdeb3ca44c15d1bf5d0485562957a7146cc99ca7"}
Jun 02 15:55:03.627567 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.627526    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"]
Jun 02 15:55:03.630940 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.630920    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.633900 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.633877    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-jhvbj\""
Jun 02 15:55:03.634062 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.633924    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
Jun 02 15:55:03.634465 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.634450    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
Jun 02 15:55:03.645274 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.645247    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"]
Jun 02 15:55:03.699085 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.699043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb25t\" (UniqueName: \"kubernetes.io/projected/a646b5cf-9871-45da-bc6d-b99223346167-kube-api-access-sb25t\") pod \"servicemesh-operator3-55f49c5f94-xl2p7\" (UID: \"a646b5cf-9871-45da-bc6d-b99223346167\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.699261 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.699144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/a646b5cf-9871-45da-bc6d-b99223346167-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xl2p7\" (UID: \"a646b5cf-9871-45da-bc6d-b99223346167\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.799625 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.799589    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sb25t\" (UniqueName: \"kubernetes.io/projected/a646b5cf-9871-45da-bc6d-b99223346167-kube-api-access-sb25t\") pod \"servicemesh-operator3-55f49c5f94-xl2p7\" (UID: \"a646b5cf-9871-45da-bc6d-b99223346167\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.799819 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.799686    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/a646b5cf-9871-45da-bc6d-b99223346167-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xl2p7\" (UID: \"a646b5cf-9871-45da-bc6d-b99223346167\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.802565 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.802539    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/a646b5cf-9871-45da-bc6d-b99223346167-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xl2p7\" (UID: \"a646b5cf-9871-45da-bc6d-b99223346167\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.812256 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.812227    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb25t\" (UniqueName: \"kubernetes.io/projected/a646b5cf-9871-45da-bc6d-b99223346167-kube-api-access-sb25t\") pod \"servicemesh-operator3-55f49c5f94-xl2p7\" (UID: \"a646b5cf-9871-45da-bc6d-b99223346167\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:03.940555 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:03.940450    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:04.125832 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:04.125804    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"]
Jun 02 15:55:04.127759 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:04.127728    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda646b5cf_9871_45da_bc6d_b99223346167.slice/crio-42bc9f7a635c5db60e5058856d75356bf5ddea75efc4919d8222a204d966174f WatchSource:0}: Error finding container 42bc9f7a635c5db60e5058856d75356bf5ddea75efc4919d8222a204d966174f: Status 404 returned error can't find the container with id 42bc9f7a635c5db60e5058856d75356bf5ddea75efc4919d8222a204d966174f
Jun 02 15:55:04.638665 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:04.638576    2563 generic.go:358] "Generic (PLEG): container finished" podID="ffa674c0-637e-4cb6-a58c-90721837920c" containerID="2c1003b116a14f8270d33fff0fbfa95535dddff8d1fca161214eafb86f3b0c05" exitCode=0
Jun 02 15:55:04.639063 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:04.638664    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv" event={"ID":"ffa674c0-637e-4cb6-a58c-90721837920c","Type":"ContainerDied","Data":"2c1003b116a14f8270d33fff0fbfa95535dddff8d1fca161214eafb86f3b0c05"}
Jun 02 15:55:04.640043 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:04.640022    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7" event={"ID":"a646b5cf-9871-45da-bc6d-b99223346167","Type":"ContainerStarted","Data":"42bc9f7a635c5db60e5058856d75356bf5ddea75efc4919d8222a204d966174f"}
Jun 02 15:55:05.645865 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:05.645827    2563 generic.go:358] "Generic (PLEG): container finished" podID="ffa674c0-637e-4cb6-a58c-90721837920c" containerID="f871c522561f64add2f636503bb289953370e379692ca44f312608e58dd9a765" exitCode=0
Jun 02 15:55:05.646304 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:05.645894    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv" event={"ID":"ffa674c0-637e-4cb6-a58c-90721837920c","Type":"ContainerDied","Data":"f871c522561f64add2f636503bb289953370e379692ca44f312608e58dd9a765"}
Jun 02 15:55:07.033615 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.033586    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:07.128285 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.128252    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-bundle\") pod \"ffa674c0-637e-4cb6-a58c-90721837920c\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") "
Jun 02 15:55:07.128421 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.128300    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkq9c\" (UniqueName: \"kubernetes.io/projected/ffa674c0-637e-4cb6-a58c-90721837920c-kube-api-access-nkq9c\") pod \"ffa674c0-637e-4cb6-a58c-90721837920c\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") "
Jun 02 15:55:07.128421 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.128344    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-util\") pod \"ffa674c0-637e-4cb6-a58c-90721837920c\" (UID: \"ffa674c0-637e-4cb6-a58c-90721837920c\") "
Jun 02 15:55:07.129235 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.129204    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-bundle" (OuterVolumeSpecName: "bundle") pod "ffa674c0-637e-4cb6-a58c-90721837920c" (UID: "ffa674c0-637e-4cb6-a58c-90721837920c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:07.130501 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.130482    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa674c0-637e-4cb6-a58c-90721837920c-kube-api-access-nkq9c" (OuterVolumeSpecName: "kube-api-access-nkq9c") pod "ffa674c0-637e-4cb6-a58c-90721837920c" (UID: "ffa674c0-637e-4cb6-a58c-90721837920c"). InnerVolumeSpecName "kube-api-access-nkq9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:55:07.132777 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.132750    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-util" (OuterVolumeSpecName: "util") pod "ffa674c0-637e-4cb6-a58c-90721837920c" (UID: "ffa674c0-637e-4cb6-a58c-90721837920c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:07.229769 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.229735    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:07.229769 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.229771    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkq9c\" (UniqueName: \"kubernetes.io/projected/ffa674c0-637e-4cb6-a58c-90721837920c-kube-api-access-nkq9c\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:07.229956 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.229786    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa674c0-637e-4cb6-a58c-90721837920c-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:07.655101 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.655057    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv" event={"ID":"ffa674c0-637e-4cb6-a58c-90721837920c","Type":"ContainerDied","Data":"79d1562ad9501d49ccb2001fbdeb3ca44c15d1bf5d0485562957a7146cc99ca7"}
Jun 02 15:55:07.655101 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.655102    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79d1562ad9501d49ccb2001fbdeb3ca44c15d1bf5d0485562957a7146cc99ca7"
Jun 02 15:55:07.655328 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.655077    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762ah8qxv"
Jun 02 15:55:07.656590 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.656558    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7" event={"ID":"a646b5cf-9871-45da-bc6d-b99223346167","Type":"ContainerStarted","Data":"2115da9db0c93a6670b17614b0bf8993aec5f26b9d6ff3cf456186cbd2b99da3"}
Jun 02 15:55:07.656741 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.656660    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:07.688660 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:07.688600    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7" podStartSLOduration=1.725911262 podStartE2EDuration="4.688580754s" podCreationTimestamp="2026-06-02 15:55:03 +0000 UTC" firstStartedPulling="2026-06-02 15:55:04.130232318 +0000 UTC m=+488.754021142" lastFinishedPulling="2026-06-02 15:55:07.092901806 +0000 UTC m=+491.716690634" observedRunningTime="2026-06-02 15:55:07.686963155 +0000 UTC m=+492.310752002" watchObservedRunningTime="2026-06-02 15:55:07.688580754 +0000 UTC m=+492.312369604"
Jun 02 15:55:12.484710 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.484674    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"]
Jun 02 15:55:12.485103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485025    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="extract"
Jun 02 15:55:12.485103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485037    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="extract"
Jun 02 15:55:12.485103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485053    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="util"
Jun 02 15:55:12.485103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485059    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="util"
Jun 02 15:55:12.485103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485068    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="pull"
Jun 02 15:55:12.485103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485073    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="pull"
Jun 02 15:55:12.485337 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.485127    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="ffa674c0-637e-4cb6-a58c-90721837920c" containerName="extract"
Jun 02 15:55:12.488105 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.488085    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.491048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.491021    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jun 02 15:55:12.491048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.491034    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jun 02 15:55:12.491048 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.491025    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jun 02 15:55:12.491585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.491565    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jun 02 15:55:12.492430 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.492412    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jun 02 15:55:12.492718 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.492700    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jun 02 15:55:12.492838 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.492749    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gpwnh\""
Jun 02 15:55:12.500494 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.500468    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"]
Jun 02 15:55:12.576679 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.576860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a359ff-6eda-4d68-b01c-378527114f8b-local-certs\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.576860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-istio-token\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.576860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-kubeconfig\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.576860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h25vg\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-kube-api-access-h25vg\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.576860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-cacerts\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.576860 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.576850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677450    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677521    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a359ff-6eda-4d68-b01c-378527114f8b-local-certs\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677548    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-istio-token\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677573    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-kubeconfig\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677611    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h25vg\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-kube-api-access-h25vg\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677639    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-cacerts\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.677662    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.678176 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.678134    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.681140 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.681109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-kubeconfig\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.681338 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.681319    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.681409 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.681378    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a359ff-6eda-4d68-b01c-378527114f8b-local-certs\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.681846 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.681826    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-cacerts\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.694654 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.694627    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-istio-token\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.694967 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.694942    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h25vg\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-kube-api-access-h25vg\") pod \"istiod-openshift-gateway-767cf5cf55-9zjzf\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.798289 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.798197    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:12.960528 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:12.960499    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"]
Jun 02 15:55:12.962305 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:12.962281    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a359ff_6eda_4d68_b01c_378527114f8b.slice/crio-d418ac8a4b9bf93f02e3d128b8b2c587f525ef427f5162a34bc9ae8cbfc0a011 WatchSource:0}: Error finding container d418ac8a4b9bf93f02e3d128b8b2c587f525ef427f5162a34bc9ae8cbfc0a011: Status 404 returned error can't find the container with id d418ac8a4b9bf93f02e3d128b8b2c587f525ef427f5162a34bc9ae8cbfc0a011
Jun 02 15:55:13.682105 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:13.682058    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" event={"ID":"72a359ff-6eda-4d68-b01c-378527114f8b","Type":"ContainerStarted","Data":"d418ac8a4b9bf93f02e3d128b8b2c587f525ef427f5162a34bc9ae8cbfc0a011"}
Jun 02 15:55:15.539174 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:15.539133    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:55:15.539469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:15.539203    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:55:15.692247 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:15.692207    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" event={"ID":"72a359ff-6eda-4d68-b01c-378527114f8b","Type":"ContainerStarted","Data":"b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da"}
Jun 02 15:55:15.692407 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:15.692268    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:15.715677 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:15.715616    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" podStartSLOduration=1.140815533 podStartE2EDuration="3.715602256s" podCreationTimestamp="2026-06-02 15:55:12 +0000 UTC" firstStartedPulling="2026-06-02 15:55:12.964090736 +0000 UTC m=+497.587879562" lastFinishedPulling="2026-06-02 15:55:15.538877447 +0000 UTC m=+500.162666285" observedRunningTime="2026-06-02 15:55:15.714071494 +0000 UTC m=+500.337860341" watchObservedRunningTime="2026-06-02 15:55:15.715602256 +0000 UTC m=+500.339391102"
Jun 02 15:55:16.699485 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:16.699446    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-767cf5cf55-9zjzf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jun 02 15:55:16.699859 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:16.699508    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:55:18.662899 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:18.662865    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xl2p7"
Jun 02 15:55:19.699401 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:19.699366    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:55:20.284003 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.283965    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"]
Jun 02 15:55:20.287511 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.287485    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.290851 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.290822    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-l6mn4\""
Jun 02 15:55:20.302223 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.302190    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"]
Jun 02 15:55:20.356178 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-workload-certs\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrcx7\" (UniqueName: \"kubernetes.io/projected/33a10624-6046-465c-9827-acc53f72274a-kube-api-access-zrcx7\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a10624-6046-465c-9827-acc53f72274a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356472 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a10624-6046-465c-9827-acc53f72274a-istio-token\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356472 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-credential-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356472 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-istio-data\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356472 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a10624-6046-465c-9827-acc53f72274a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.356649 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.356514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-workload-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457343 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457300    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-workload-certs\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457359    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457397    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zrcx7\" (UniqueName: \"kubernetes.io/projected/33a10624-6046-465c-9827-acc53f72274a-kube-api-access-zrcx7\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457421    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a10624-6046-465c-9827-acc53f72274a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457440    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a10624-6046-465c-9827-acc53f72274a-istio-token\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457466    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-credential-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457504    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-istio-data\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457529 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457530    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a10624-6046-465c-9827-acc53f72274a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457891 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457557    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-workload-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457891 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457795    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-workload-certs\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457995 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457892    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-credential-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.457995 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457977    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-istio-data\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.458087 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.457997    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-workload-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.458525 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.458497    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a10624-6046-465c-9827-acc53f72274a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.460054 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.459998    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a10624-6046-465c-9827-acc53f72274a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.460625 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.460596    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a10624-6046-465c-9827-acc53f72274a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.477711 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.477673    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrcx7\" (UniqueName: \"kubernetes.io/projected/33a10624-6046-465c-9827-acc53f72274a-kube-api-access-zrcx7\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.488519 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.488484    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a10624-6046-465c-9827-acc53f72274a-istio-token\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t\" (UID: \"33a10624-6046-465c-9827-acc53f72274a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.599595 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.599548    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:20.760476 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:20.760451    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"]
Jun 02 15:55:20.762306 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:20.762282    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a10624_6046_465c_9827_acc53f72274a.slice/crio-5851dfd2b8989d1d4cbffaadd8549180a2f838b38909b3461a48074b2e1a2a53 WatchSource:0}: Error finding container 5851dfd2b8989d1d4cbffaadd8549180a2f838b38909b3461a48074b2e1a2a53: Status 404 returned error can't find the container with id 5851dfd2b8989d1d4cbffaadd8549180a2f838b38909b3461a48074b2e1a2a53
Jun 02 15:55:21.723221 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:21.723185    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t" event={"ID":"33a10624-6046-465c-9827-acc53f72274a","Type":"ContainerStarted","Data":"5851dfd2b8989d1d4cbffaadd8549180a2f838b38909b3461a48074b2e1a2a53"}
Jun 02 15:55:22.905370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:22.905332    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:55:22.905754 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:22.905420    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:55:22.905754 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:22.905464    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:55:23.731675 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:23.731629    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t" event={"ID":"33a10624-6046-465c-9827-acc53f72274a","Type":"ContainerStarted","Data":"4d106b911b2e4c5af8787ab8b752557ca9a5be14f3770a727b50dff458242b77"}
Jun 02 15:55:23.758796 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:23.756684    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t" podStartSLOduration=1.615778872 podStartE2EDuration="3.756665444s" podCreationTimestamp="2026-06-02 15:55:20 +0000 UTC" firstStartedPulling="2026-06-02 15:55:20.764201022 +0000 UTC m=+505.387989847" lastFinishedPulling="2026-06-02 15:55:22.905087592 +0000 UTC m=+507.528876419" observedRunningTime="2026-06-02 15:55:23.754993821 +0000 UTC m=+508.378782669" watchObservedRunningTime="2026-06-02 15:55:23.756665444 +0000 UTC m=+508.380454292"
Jun 02 15:55:24.600324 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:24.600286    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:24.604852 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:24.604828    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:24.735719 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:24.735695    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:24.736593 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:24.736577    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t"
Jun 02 15:55:30.704591 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.704550    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"]
Jun 02 15:55:30.708352 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.708333    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.711407 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.711376    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jun 02 15:55:30.711407 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.711401    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jun 02 15:55:30.712573 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.712555    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6jj48\""
Jun 02 15:55:30.718944 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.718921    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"]
Jun 02 15:55:30.815885 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.815846    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"]
Jun 02 15:55:30.819469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.819444    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:30.844820 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.844789    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"]
Jun 02 15:55:30.852042 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.852000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.852206 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.852062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd4b5\" (UniqueName: \"kubernetes.io/projected/461be2b9-9e2a-494b-bd63-1e0747148a5c-kube-api-access-pd4b5\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.852206 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.852141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.930005 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.929963    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"]
Jun 02 15:55:30.933578 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.933560    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:30.943867 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.943839    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"]
Jun 02 15:55:30.953405 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953379    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pd4b5\" (UniqueName: \"kubernetes.io/projected/461be2b9-9e2a-494b-bd63-1e0747148a5c-kube-api-access-pd4b5\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953425    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jsdv\" (UniqueName: \"kubernetes.io/projected/d5e14de9-48ed-4664-88d4-a82f72791225-kube-api-access-9jsdv\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953689    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.953872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.954279 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.954188    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:30.963161 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:30.963103    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd4b5\" (UniqueName: \"kubernetes.io/projected/461be2b9-9e2a-494b-bd63-1e0747148a5c-kube-api-access-pd4b5\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:31.005910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.005881    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"]
Jun 02 15:55:31.009528 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.009512    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.018480 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.018433    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:31.020076 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.020054    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"]
Jun 02 15:55:31.054462 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054418    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdbqr\" (UniqueName: \"kubernetes.io/projected/13c2c206-6e6b-48bd-8bef-ba87cc303f23-kube-api-access-rdbqr\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.054657 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.054657 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054530    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.054657 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054557    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9jsdv\" (UniqueName: \"kubernetes.io/projected/d5e14de9-48ed-4664-88d4-a82f72791225-kube-api-access-9jsdv\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.054657 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054611    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.054657 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.054995 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054967    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.055054 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.054985    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.065276 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.065233    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jsdv\" (UniqueName: \"kubernetes.io/projected/d5e14de9-48ed-4664-88d4-a82f72791225-kube-api-access-9jsdv\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.128421 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.128385    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:31.155471 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155439    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.155649 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-util\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.155649 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-bundle\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.155764 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155655    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rdbqr\" (UniqueName: \"kubernetes.io/projected/13c2c206-6e6b-48bd-8bef-ba87cc303f23-kube-api-access-rdbqr\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.155764 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlcxp\" (UniqueName: \"kubernetes.io/projected/f3712232-c80c-4291-9b38-82b4f0f7ecf8-kube-api-access-tlcxp\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.155855 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155774    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.155904 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155848    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.156027 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.155984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.159723 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.159696    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"]
Jun 02 15:55:31.164051 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:31.164021    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod461be2b9_9e2a_494b_bd63_1e0747148a5c.slice/crio-614be5c3a308ace2d5716de6f1848594a2079831091e33623c57eadd38aa007a WatchSource:0}: Error finding container 614be5c3a308ace2d5716de6f1848594a2079831091e33623c57eadd38aa007a: Status 404 returned error can't find the container with id 614be5c3a308ace2d5716de6f1848594a2079831091e33623c57eadd38aa007a
Jun 02 15:55:31.170969 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.170943    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdbqr\" (UniqueName: \"kubernetes.io/projected/13c2c206-6e6b-48bd-8bef-ba87cc303f23-kube-api-access-rdbqr\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.243996 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.243961    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:31.257282 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.257248    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-util\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.257410 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.257303    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-bundle\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.257410 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.257375    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tlcxp\" (UniqueName: \"kubernetes.io/projected/f3712232-c80c-4291-9b38-82b4f0f7ecf8-kube-api-access-tlcxp\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.257760 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.257735    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-util\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.257881 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.257867    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-bundle\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.266772 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.266623    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlcxp\" (UniqueName: \"kubernetes.io/projected/f3712232-c80c-4291-9b38-82b4f0f7ecf8-kube-api-access-tlcxp\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.276505 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.276481    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"]
Jun 02 15:55:31.314387 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:31.314346    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e14de9_48ed_4664_88d4_a82f72791225.slice/crio-a278f556e376c4036be23797c6178121b896eddaf7d2ef7094f888d7c34a82cc WatchSource:0}: Error finding container a278f556e376c4036be23797c6178121b896eddaf7d2ef7094f888d7c34a82cc: Status 404 returned error can't find the container with id a278f556e376c4036be23797c6178121b896eddaf7d2ef7094f888d7c34a82cc
Jun 02 15:55:31.319616 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.319592    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:31.386612 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.386579    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"]
Jun 02 15:55:31.391191 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:31.390222    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c2c206_6e6b_48bd_8bef_ba87cc303f23.slice/crio-407a8feb6e7a3514f786a4e051f641eedf23660bd12cc86191383ca6aceb4b5f WatchSource:0}: Error finding container 407a8feb6e7a3514f786a4e051f641eedf23660bd12cc86191383ca6aceb4b5f: Status 404 returned error can't find the container with id 407a8feb6e7a3514f786a4e051f641eedf23660bd12cc86191383ca6aceb4b5f
Jun 02 15:55:31.479183 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.479158    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"]
Jun 02 15:55:31.482063 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:31.481993    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3712232_c80c_4291_9b38_82b4f0f7ecf8.slice/crio-71e07cf2d9e527ae32c0cc2676ec1a6f2f1f87a1f6f9679136211dcee19836d9 WatchSource:0}: Error finding container 71e07cf2d9e527ae32c0cc2676ec1a6f2f1f87a1f6f9679136211dcee19836d9: Status 404 returned error can't find the container with id 71e07cf2d9e527ae32c0cc2676ec1a6f2f1f87a1f6f9679136211dcee19836d9
Jun 02 15:55:31.763132 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.763035    2563 generic.go:358] "Generic (PLEG): container finished" podID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerID="bf6869f9d36503af2bc13174927b85b126e27573a757ffa078a22c5944481a45" exitCode=0
Jun 02 15:55:31.763132 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.763098    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk" event={"ID":"f3712232-c80c-4291-9b38-82b4f0f7ecf8","Type":"ContainerDied","Data":"bf6869f9d36503af2bc13174927b85b126e27573a757ffa078a22c5944481a45"}
Jun 02 15:55:31.763578 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.763140    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk" event={"ID":"f3712232-c80c-4291-9b38-82b4f0f7ecf8","Type":"ContainerStarted","Data":"71e07cf2d9e527ae32c0cc2676ec1a6f2f1f87a1f6f9679136211dcee19836d9"}
Jun 02 15:55:31.764615 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.764553    2563 generic.go:358] "Generic (PLEG): container finished" podID="d5e14de9-48ed-4664-88d4-a82f72791225" containerID="408d803d6a9169136042756737a2f92cb5ff8eb62be89269f849908158f1c2b3" exitCode=0
Jun 02 15:55:31.764688 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.764617    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w" event={"ID":"d5e14de9-48ed-4664-88d4-a82f72791225","Type":"ContainerDied","Data":"408d803d6a9169136042756737a2f92cb5ff8eb62be89269f849908158f1c2b3"}
Jun 02 15:55:31.764688 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.764640    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w" event={"ID":"d5e14de9-48ed-4664-88d4-a82f72791225","Type":"ContainerStarted","Data":"a278f556e376c4036be23797c6178121b896eddaf7d2ef7094f888d7c34a82cc"}
Jun 02 15:55:31.766103 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.766077    2563 generic.go:358] "Generic (PLEG): container finished" podID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerID="f0300d4c425400064c56226daaa05c3f30214be7d584dc5657f189e58a640513" exitCode=0
Jun 02 15:55:31.766162 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.766150    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq" event={"ID":"461be2b9-9e2a-494b-bd63-1e0747148a5c","Type":"ContainerDied","Data":"f0300d4c425400064c56226daaa05c3f30214be7d584dc5657f189e58a640513"}
Jun 02 15:55:31.766214 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.766169    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq" event={"ID":"461be2b9-9e2a-494b-bd63-1e0747148a5c","Type":"ContainerStarted","Data":"614be5c3a308ace2d5716de6f1848594a2079831091e33623c57eadd38aa007a"}
Jun 02 15:55:31.767694 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.767676    2563 generic.go:358] "Generic (PLEG): container finished" podID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerID="dcc861cfd4e49c27067e7cefbf307ba85726f319e216d916eec15f4662bdbfb1" exitCode=0
Jun 02 15:55:31.767771 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.767726    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9" event={"ID":"13c2c206-6e6b-48bd-8bef-ba87cc303f23","Type":"ContainerDied","Data":"dcc861cfd4e49c27067e7cefbf307ba85726f319e216d916eec15f4662bdbfb1"}
Jun 02 15:55:31.767771 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:31.767748    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9" event={"ID":"13c2c206-6e6b-48bd-8bef-ba87cc303f23","Type":"ContainerStarted","Data":"407a8feb6e7a3514f786a4e051f641eedf23660bd12cc86191383ca6aceb4b5f"}
Jun 02 15:55:32.772907 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:32.772817    2563 generic.go:358] "Generic (PLEG): container finished" podID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerID="35dd6b2a700e3f4a4b417ecd075886c798bb80f87fc29088ecfbae4e5dc49701" exitCode=0
Jun 02 15:55:32.772907 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:32.772892    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9" event={"ID":"13c2c206-6e6b-48bd-8bef-ba87cc303f23","Type":"ContainerDied","Data":"35dd6b2a700e3f4a4b417ecd075886c798bb80f87fc29088ecfbae4e5dc49701"}
Jun 02 15:55:33.780707 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:33.780666    2563 generic.go:358] "Generic (PLEG): container finished" podID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerID="37ecba98c4346553f6b3a3ca35fcb3ef67308f37b6bf41caa5de480363e00757" exitCode=0
Jun 02 15:55:33.781185 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:33.780747    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq" event={"ID":"461be2b9-9e2a-494b-bd63-1e0747148a5c","Type":"ContainerDied","Data":"37ecba98c4346553f6b3a3ca35fcb3ef67308f37b6bf41caa5de480363e00757"}
Jun 02 15:55:33.782841 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:33.782823    2563 generic.go:358] "Generic (PLEG): container finished" podID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerID="77d15bf55077fa8e56c11e72ff7f35268c55dd82e420b5739728f43278c336ac" exitCode=0
Jun 02 15:55:33.782924 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:33.782908    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9" event={"ID":"13c2c206-6e6b-48bd-8bef-ba87cc303f23","Type":"ContainerDied","Data":"77d15bf55077fa8e56c11e72ff7f35268c55dd82e420b5739728f43278c336ac"}
Jun 02 15:55:34.788313 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.788216    2563 generic.go:358] "Generic (PLEG): container finished" podID="d5e14de9-48ed-4664-88d4-a82f72791225" containerID="97b3c04669103be223f93330278f36db63ca7c26cfb5b1f5ff55ea6cf999db8a" exitCode=0
Jun 02 15:55:34.788313 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.788300    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w" event={"ID":"d5e14de9-48ed-4664-88d4-a82f72791225","Type":"ContainerDied","Data":"97b3c04669103be223f93330278f36db63ca7c26cfb5b1f5ff55ea6cf999db8a"}
Jun 02 15:55:34.790341 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.790314    2563 generic.go:358] "Generic (PLEG): container finished" podID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerID="0e9e49d6e1c26967d6945fb1a3b050e5eca55596803401c4cd35b649a9bd193f" exitCode=0
Jun 02 15:55:34.790473 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.790368    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq" event={"ID":"461be2b9-9e2a-494b-bd63-1e0747148a5c","Type":"ContainerDied","Data":"0e9e49d6e1c26967d6945fb1a3b050e5eca55596803401c4cd35b649a9bd193f"}
Jun 02 15:55:34.791997 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.791977    2563 generic.go:358] "Generic (PLEG): container finished" podID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerID="e1bc2abec0864b9f4e2b57a1821d8283bea4404dfab6486ed05ed6dfa81b0593" exitCode=0
Jun 02 15:55:34.792101 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.792046    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk" event={"ID":"f3712232-c80c-4291-9b38-82b4f0f7ecf8","Type":"ContainerDied","Data":"e1bc2abec0864b9f4e2b57a1821d8283bea4404dfab6486ed05ed6dfa81b0593"}
Jun 02 15:55:34.926006 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:34.925969    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:35.092435 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.092404    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-bundle\") pod \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") "
Jun 02 15:55:35.092604 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.092464    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-util\") pod \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") "
Jun 02 15:55:35.092604 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.092495    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdbqr\" (UniqueName: \"kubernetes.io/projected/13c2c206-6e6b-48bd-8bef-ba87cc303f23-kube-api-access-rdbqr\") pod \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\" (UID: \"13c2c206-6e6b-48bd-8bef-ba87cc303f23\") "
Jun 02 15:55:35.093006 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.092983    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-bundle" (OuterVolumeSpecName: "bundle") pod "13c2c206-6e6b-48bd-8bef-ba87cc303f23" (UID: "13c2c206-6e6b-48bd-8bef-ba87cc303f23"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:35.094789 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.094768    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c2c206-6e6b-48bd-8bef-ba87cc303f23-kube-api-access-rdbqr" (OuterVolumeSpecName: "kube-api-access-rdbqr") pod "13c2c206-6e6b-48bd-8bef-ba87cc303f23" (UID: "13c2c206-6e6b-48bd-8bef-ba87cc303f23"). InnerVolumeSpecName "kube-api-access-rdbqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:55:35.097721 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.097699    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-util" (OuterVolumeSpecName: "util") pod "13c2c206-6e6b-48bd-8bef-ba87cc303f23" (UID: "13c2c206-6e6b-48bd-8bef-ba87cc303f23"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:35.193633 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.193599    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:35.193633 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.193628    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13c2c206-6e6b-48bd-8bef-ba87cc303f23-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:35.193633 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.193638    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdbqr\" (UniqueName: \"kubernetes.io/projected/13c2c206-6e6b-48bd-8bef-ba87cc303f23-kube-api-access-rdbqr\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:35.801118 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.801075    2563 generic.go:358] "Generic (PLEG): container finished" podID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerID="397574c4a0728e92a08346e5440ceaa79033a6740739a1954ba7db098f8fc5bf" exitCode=0
Jun 02 15:55:35.801565 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.801156    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk" event={"ID":"f3712232-c80c-4291-9b38-82b4f0f7ecf8","Type":"ContainerDied","Data":"397574c4a0728e92a08346e5440ceaa79033a6740739a1954ba7db098f8fc5bf"}
Jun 02 15:55:35.802957 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.802935    2563 generic.go:358] "Generic (PLEG): container finished" podID="d5e14de9-48ed-4664-88d4-a82f72791225" containerID="7b637c499fe58c110f3a1bac50b7ae2a47c44cc47f320623a6e1b3f51babccb3" exitCode=0
Jun 02 15:55:35.803089 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.803027    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w" event={"ID":"d5e14de9-48ed-4664-88d4-a82f72791225","Type":"ContainerDied","Data":"7b637c499fe58c110f3a1bac50b7ae2a47c44cc47f320623a6e1b3f51babccb3"}
Jun 02 15:55:35.804591 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.804567    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9" event={"ID":"13c2c206-6e6b-48bd-8bef-ba87cc303f23","Type":"ContainerDied","Data":"407a8feb6e7a3514f786a4e051f641eedf23660bd12cc86191383ca6aceb4b5f"}
Jun 02 15:55:35.804668 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.804592    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88fmfd9"
Jun 02 15:55:35.804668 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.804596    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="407a8feb6e7a3514f786a4e051f641eedf23660bd12cc86191383ca6aceb4b5f"
Jun 02 15:55:35.939532 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:35.939506    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:36.100702 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.100605    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-bundle\") pod \"461be2b9-9e2a-494b-bd63-1e0747148a5c\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") "
Jun 02 15:55:36.100702 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.100663    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-util\") pod \"461be2b9-9e2a-494b-bd63-1e0747148a5c\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") "
Jun 02 15:55:36.100889 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.100721    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd4b5\" (UniqueName: \"kubernetes.io/projected/461be2b9-9e2a-494b-bd63-1e0747148a5c-kube-api-access-pd4b5\") pod \"461be2b9-9e2a-494b-bd63-1e0747148a5c\" (UID: \"461be2b9-9e2a-494b-bd63-1e0747148a5c\") "
Jun 02 15:55:36.101201 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.101170    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-bundle" (OuterVolumeSpecName: "bundle") pod "461be2b9-9e2a-494b-bd63-1e0747148a5c" (UID: "461be2b9-9e2a-494b-bd63-1e0747148a5c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:36.102998 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.102951    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/461be2b9-9e2a-494b-bd63-1e0747148a5c-kube-api-access-pd4b5" (OuterVolumeSpecName: "kube-api-access-pd4b5") pod "461be2b9-9e2a-494b-bd63-1e0747148a5c" (UID: "461be2b9-9e2a-494b-bd63-1e0747148a5c"). InnerVolumeSpecName "kube-api-access-pd4b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:55:36.106072 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.106043    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-util" (OuterVolumeSpecName: "util") pod "461be2b9-9e2a-494b-bd63-1e0747148a5c" (UID: "461be2b9-9e2a-494b-bd63-1e0747148a5c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:36.202232 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.202190    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd4b5\" (UniqueName: \"kubernetes.io/projected/461be2b9-9e2a-494b-bd63-1e0747148a5c-kube-api-access-pd4b5\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:36.202232 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.202227    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:36.202232 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.202238    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461be2b9-9e2a-494b-bd63-1e0747148a5c-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:36.810420 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.810390    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq"
Jun 02 15:55:36.810794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.810384    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30ktkfq" event={"ID":"461be2b9-9e2a-494b-bd63-1e0747148a5c","Type":"ContainerDied","Data":"614be5c3a308ace2d5716de6f1848594a2079831091e33623c57eadd38aa007a"}
Jun 02 15:55:36.810794 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.810533    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="614be5c3a308ace2d5716de6f1848594a2079831091e33623c57eadd38aa007a"
Jun 02 15:55:36.947551 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.947508    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:36.968893 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:36.968863    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:37.110925 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.110826    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-util\") pod \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") "
Jun 02 15:55:37.110925 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.110872    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlcxp\" (UniqueName: \"kubernetes.io/projected/f3712232-c80c-4291-9b38-82b4f0f7ecf8-kube-api-access-tlcxp\") pod \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") "
Jun 02 15:55:37.110925 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.110898    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-bundle\") pod \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\" (UID: \"f3712232-c80c-4291-9b38-82b4f0f7ecf8\") "
Jun 02 15:55:37.111240 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.110938    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-bundle\") pod \"d5e14de9-48ed-4664-88d4-a82f72791225\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") "
Jun 02 15:55:37.111240 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.110965    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-util\") pod \"d5e14de9-48ed-4664-88d4-a82f72791225\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") "
Jun 02 15:55:37.111240 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.111067    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jsdv\" (UniqueName: \"kubernetes.io/projected/d5e14de9-48ed-4664-88d4-a82f72791225-kube-api-access-9jsdv\") pod \"d5e14de9-48ed-4664-88d4-a82f72791225\" (UID: \"d5e14de9-48ed-4664-88d4-a82f72791225\") "
Jun 02 15:55:37.111531 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.111484    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-bundle" (OuterVolumeSpecName: "bundle") pod "d5e14de9-48ed-4664-88d4-a82f72791225" (UID: "d5e14de9-48ed-4664-88d4-a82f72791225"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:37.111677 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.111645    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-bundle" (OuterVolumeSpecName: "bundle") pod "f3712232-c80c-4291-9b38-82b4f0f7ecf8" (UID: "f3712232-c80c-4291-9b38-82b4f0f7ecf8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:37.113275 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.113250    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5e14de9-48ed-4664-88d4-a82f72791225-kube-api-access-9jsdv" (OuterVolumeSpecName: "kube-api-access-9jsdv") pod "d5e14de9-48ed-4664-88d4-a82f72791225" (UID: "d5e14de9-48ed-4664-88d4-a82f72791225"). InnerVolumeSpecName "kube-api-access-9jsdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:55:37.113530 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.113511    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3712232-c80c-4291-9b38-82b4f0f7ecf8-kube-api-access-tlcxp" (OuterVolumeSpecName: "kube-api-access-tlcxp") pod "f3712232-c80c-4291-9b38-82b4f0f7ecf8" (UID: "f3712232-c80c-4291-9b38-82b4f0f7ecf8"). InnerVolumeSpecName "kube-api-access-tlcxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:55:37.115796 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.115776    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-util" (OuterVolumeSpecName: "util") pod "f3712232-c80c-4291-9b38-82b4f0f7ecf8" (UID: "f3712232-c80c-4291-9b38-82b4f0f7ecf8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:37.117542 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.117513    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-util" (OuterVolumeSpecName: "util") pod "d5e14de9-48ed-4664-88d4-a82f72791225" (UID: "d5e14de9-48ed-4664-88d4-a82f72791225"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:55:37.212671 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.212629    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jsdv\" (UniqueName: \"kubernetes.io/projected/d5e14de9-48ed-4664-88d4-a82f72791225-kube-api-access-9jsdv\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:37.212671 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.212666    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:37.212671 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.212676    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlcxp\" (UniqueName: \"kubernetes.io/projected/f3712232-c80c-4291-9b38-82b4f0f7ecf8-kube-api-access-tlcxp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:37.212914 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.212686    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3712232-c80c-4291-9b38-82b4f0f7ecf8-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:37.212914 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.212694    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-bundle\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:37.212914 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.212702    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5e14de9-48ed-4664-88d4-a82f72791225-util\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:55:37.816088 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.816048    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w" event={"ID":"d5e14de9-48ed-4664-88d4-a82f72791225","Type":"ContainerDied","Data":"a278f556e376c4036be23797c6178121b896eddaf7d2ef7094f888d7c34a82cc"}
Jun 02 15:55:37.816088 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.816088    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a278f556e376c4036be23797c6178121b896eddaf7d2ef7094f888d7c34a82cc"
Jun 02 15:55:37.816577 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.816108    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bmv22w"
Jun 02 15:55:37.817808 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.817784    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk" event={"ID":"f3712232-c80c-4291-9b38-82b4f0f7ecf8","Type":"ContainerDied","Data":"71e07cf2d9e527ae32c0cc2676ec1a6f2f1f87a1f6f9679136211dcee19836d9"}
Jun 02 15:55:37.817808 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.817810    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71e07cf2d9e527ae32c0cc2676ec1a6f2f1f87a1f6f9679136211dcee19836d9"
Jun 02 15:55:37.817943 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:37.817818    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e78l2qlk"
Jun 02 15:55:43.023887 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.023795    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"]
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024173    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024188    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024196    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024202    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024211    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024216    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024222    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024227    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024234    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024239    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024244    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024248    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024255    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024261    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024269    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024275    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024287    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024291    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="pull"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024297    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024302    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="util"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024308    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024312    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024318    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024322    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024371    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="13c2c206-6e6b-48bd-8bef-ba87cc303f23" containerName="extract"
Jun 02 15:55:43.024370 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024381    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="f3712232-c80c-4291-9b38-82b4f0f7ecf8" containerName="extract"
Jun 02 15:55:43.025125 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024387    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="d5e14de9-48ed-4664-88d4-a82f72791225" containerName="extract"
Jun 02 15:55:43.025125 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.024395    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="461be2b9-9e2a-494b-bd63-1e0747148a5c" containerName="extract"
Jun 02 15:55:43.027406 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.027389    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:43.030469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.030437    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vjwd8\""
Jun 02 15:55:43.030600 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.030442    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jun 02 15:55:43.031406 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.031386    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jun 02 15:55:43.038300 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.038276    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"]
Jun 02 15:55:43.062844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.062814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrjr8\" (UniqueName: \"kubernetes.io/projected/bf180035-fcec-4771-8f3b-e8c884266cd4-kube-api-access-wrjr8\") pod \"limitador-operator-controller-manager-c7fb4c8d5-ng5cv\" (UID: \"bf180035-fcec-4771-8f3b-e8c884266cd4\") " pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:43.164128 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.164097    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wrjr8\" (UniqueName: \"kubernetes.io/projected/bf180035-fcec-4771-8f3b-e8c884266cd4-kube-api-access-wrjr8\") pod \"limitador-operator-controller-manager-c7fb4c8d5-ng5cv\" (UID: \"bf180035-fcec-4771-8f3b-e8c884266cd4\") " pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:43.173458 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.173432    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrjr8\" (UniqueName: \"kubernetes.io/projected/bf180035-fcec-4771-8f3b-e8c884266cd4-kube-api-access-wrjr8\") pod \"limitador-operator-controller-manager-c7fb4c8d5-ng5cv\" (UID: \"bf180035-fcec-4771-8f3b-e8c884266cd4\") " pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:43.338612 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.338567    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:43.497166 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.497124    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"]
Jun 02 15:55:43.500284 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:43.500252    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf180035_fcec_4771_8f3b_e8c884266cd4.slice/crio-97aa3d013ecc4cde5a4e7adb044d039484129acd0dd4ae95804a804c9c58e2e5 WatchSource:0}: Error finding container 97aa3d013ecc4cde5a4e7adb044d039484129acd0dd4ae95804a804c9c58e2e5: Status 404 returned error can't find the container with id 97aa3d013ecc4cde5a4e7adb044d039484129acd0dd4ae95804a804c9c58e2e5
Jun 02 15:55:43.841095 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:43.841056    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv" event={"ID":"bf180035-fcec-4771-8f3b-e8c884266cd4","Type":"ContainerStarted","Data":"97aa3d013ecc4cde5a4e7adb044d039484129acd0dd4ae95804a804c9c58e2e5"}
Jun 02 15:55:45.914070 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:45.914032    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-xccdr"]
Jun 02 15:55:45.916469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:45.916453    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:55:45.932297 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:45.932233    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cb8nj\""
Jun 02 15:55:45.935335 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:45.935312    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-xccdr"]
Jun 02 15:55:45.988798 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:45.988733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zvwm\" (UniqueName: \"kubernetes.io/projected/c5d7ff2e-3462-4732-84c8-a5b54878b1eb-kube-api-access-5zvwm\") pod \"authorino-operator-7587b89b76-xccdr\" (UID: \"c5d7ff2e-3462-4732-84c8-a5b54878b1eb\") " pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:55:46.089906 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.089866    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5zvwm\" (UniqueName: \"kubernetes.io/projected/c5d7ff2e-3462-4732-84c8-a5b54878b1eb-kube-api-access-5zvwm\") pod \"authorino-operator-7587b89b76-xccdr\" (UID: \"c5d7ff2e-3462-4732-84c8-a5b54878b1eb\") " pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:55:46.102639 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.102604    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zvwm\" (UniqueName: \"kubernetes.io/projected/c5d7ff2e-3462-4732-84c8-a5b54878b1eb-kube-api-access-5zvwm\") pod \"authorino-operator-7587b89b76-xccdr\" (UID: \"c5d7ff2e-3462-4732-84c8-a5b54878b1eb\") " pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:55:46.227447 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.227347    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:55:46.679585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.679551    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-xccdr"]
Jun 02 15:55:46.680700 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:55:46.680668    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d7ff2e_3462_4732_84c8_a5b54878b1eb.slice/crio-6420d622aa0a647b1b85f9f6c54addbd13a2886180b0317908fbc65c85d20a92 WatchSource:0}: Error finding container 6420d622aa0a647b1b85f9f6c54addbd13a2886180b0317908fbc65c85d20a92: Status 404 returned error can't find the container with id 6420d622aa0a647b1b85f9f6c54addbd13a2886180b0317908fbc65c85d20a92
Jun 02 15:55:46.854699 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.854654    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr" event={"ID":"c5d7ff2e-3462-4732-84c8-a5b54878b1eb","Type":"ContainerStarted","Data":"6420d622aa0a647b1b85f9f6c54addbd13a2886180b0317908fbc65c85d20a92"}
Jun 02 15:55:46.856116 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.856080    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv" event={"ID":"bf180035-fcec-4771-8f3b-e8c884266cd4","Type":"ContainerStarted","Data":"a07c91b4bde116d948896cc134f85b19bafc691457ac75b60ac260c578d03e95"}
Jun 02 15:55:46.856369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.856200    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:46.892468 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:46.892393    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv" podStartSLOduration=0.858313071 podStartE2EDuration="3.892372553s" podCreationTimestamp="2026-06-02 15:55:43 +0000 UTC" firstStartedPulling="2026-06-02 15:55:43.502272934 +0000 UTC m=+528.126061759" lastFinishedPulling="2026-06-02 15:55:46.536332406 +0000 UTC m=+531.160121241" observedRunningTime="2026-06-02 15:55:46.891576678 +0000 UTC m=+531.515365525" watchObservedRunningTime="2026-06-02 15:55:46.892372553 +0000 UTC m=+531.516161402"
Jun 02 15:55:48.865908 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:48.865873    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr" event={"ID":"c5d7ff2e-3462-4732-84c8-a5b54878b1eb","Type":"ContainerStarted","Data":"d447a647207fe8f7b0ec5249be614c40219815a5e90486e3815ad2dd57d77b4c"}
Jun 02 15:55:48.866308 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:48.865993    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:55:48.895171 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:48.895109    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr" podStartSLOduration=1.897546025 podStartE2EDuration="3.895094213s" podCreationTimestamp="2026-06-02 15:55:45 +0000 UTC" firstStartedPulling="2026-06-02 15:55:46.683162497 +0000 UTC m=+531.306951328" lastFinishedPulling="2026-06-02 15:55:48.680710692 +0000 UTC m=+533.304499516" observedRunningTime="2026-06-02 15:55:48.892507517 +0000 UTC m=+533.516296363" watchObservedRunningTime="2026-06-02 15:55:48.895094213 +0000 UTC m=+533.518883059"
Jun 02 15:55:57.862783 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:57.862748    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-ng5cv"
Jun 02 15:55:59.872680 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:55:59.872648    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7587b89b76-xccdr"
Jun 02 15:56:41.732149 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.732115    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674b59b84c-h7jdl"]
Jun 02 15:56:41.734478 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.734462    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:41.737679 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.737657    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4lqzs\""
Jun 02 15:56:41.745209 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.745187    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-h7jdl"]
Jun 02 15:56:41.759702 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.759670    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2xm5\" (UniqueName: \"kubernetes.io/projected/572dcb30-f43b-4e3b-bc73-cef31d09b792-kube-api-access-p2xm5\") pod \"authorino-674b59b84c-h7jdl\" (UID: \"572dcb30-f43b-4e3b-bc73-cef31d09b792\") " pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:41.860161 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.860120    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2xm5\" (UniqueName: \"kubernetes.io/projected/572dcb30-f43b-4e3b-bc73-cef31d09b792-kube-api-access-p2xm5\") pod \"authorino-674b59b84c-h7jdl\" (UID: \"572dcb30-f43b-4e3b-bc73-cef31d09b792\") " pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:41.868042 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.867998    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mxh4n"]
Jun 02 15:56:41.870388 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.870366    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:56:41.871740 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.871718    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2xm5\" (UniqueName: \"kubernetes.io/projected/572dcb30-f43b-4e3b-bc73-cef31d09b792-kube-api-access-p2xm5\") pod \"authorino-674b59b84c-h7jdl\" (UID: \"572dcb30-f43b-4e3b-bc73-cef31d09b792\") " pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:41.880309 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.880283    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mxh4n"]
Jun 02 15:56:41.960678 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:41.960640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mww2j\" (UniqueName: \"kubernetes.io/projected/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81-kube-api-access-mww2j\") pod \"authorino-79cbc94b89-mxh4n\" (UID: \"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81\") " pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:56:42.052066 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:42.052033    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:42.061891 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:42.061814    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mww2j\" (UniqueName: \"kubernetes.io/projected/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81-kube-api-access-mww2j\") pod \"authorino-79cbc94b89-mxh4n\" (UID: \"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81\") " pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:56:42.072349 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:42.072316    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mww2j\" (UniqueName: \"kubernetes.io/projected/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81-kube-api-access-mww2j\") pod \"authorino-79cbc94b89-mxh4n\" (UID: \"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81\") " pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:56:42.188059 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:42.188025    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:56:42.188811 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:42.188707    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-h7jdl"]
Jun 02 15:56:42.337589 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:42.337560    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mxh4n"]
Jun 02 15:56:42.338879 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:56:42.338853    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ac84dbe_1d3c_4686_ab0d_dcf483d2bc81.slice/crio-bd00ace1f1cea20db39bace13870d84e398ae25023ddff4bfe88bd1ce58e35ff WatchSource:0}: Error finding container bd00ace1f1cea20db39bace13870d84e398ae25023ddff4bfe88bd1ce58e35ff: Status 404 returned error can't find the container with id bd00ace1f1cea20db39bace13870d84e398ae25023ddff4bfe88bd1ce58e35ff
Jun 02 15:56:43.076567 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:43.076525    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mxh4n" event={"ID":"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81","Type":"ContainerStarted","Data":"bd00ace1f1cea20db39bace13870d84e398ae25023ddff4bfe88bd1ce58e35ff"}
Jun 02 15:56:43.077578 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:43.077550    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-h7jdl" event={"ID":"572dcb30-f43b-4e3b-bc73-cef31d09b792","Type":"ContainerStarted","Data":"130f9d0a0b0b483501d313099841d4146c7cdfec482fa9df65d611ba49907548"}
Jun 02 15:56:46.092845 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:46.092805    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mxh4n" event={"ID":"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81","Type":"ContainerStarted","Data":"6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9"}
Jun 02 15:56:46.094171 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:46.094131    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-h7jdl" event={"ID":"572dcb30-f43b-4e3b-bc73-cef31d09b792","Type":"ContainerStarted","Data":"54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272"}
Jun 02 15:56:46.118982 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:46.118926    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-79cbc94b89-mxh4n" podStartSLOduration=2.277261725 podStartE2EDuration="5.118912793s" podCreationTimestamp="2026-06-02 15:56:41 +0000 UTC" firstStartedPulling="2026-06-02 15:56:42.34021927 +0000 UTC m=+586.964008095" lastFinishedPulling="2026-06-02 15:56:45.181870324 +0000 UTC m=+589.805659163" observedRunningTime="2026-06-02 15:56:46.116136589 +0000 UTC m=+590.739925435" watchObservedRunningTime="2026-06-02 15:56:46.118912793 +0000 UTC m=+590.742701641"
Jun 02 15:56:46.140734 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:46.140677    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-674b59b84c-h7jdl" podStartSLOduration=2.163763934 podStartE2EDuration="5.140662848s" podCreationTimestamp="2026-06-02 15:56:41 +0000 UTC" firstStartedPulling="2026-06-02 15:56:42.194291463 +0000 UTC m=+586.818080288" lastFinishedPulling="2026-06-02 15:56:45.171190373 +0000 UTC m=+589.794979202" observedRunningTime="2026-06-02 15:56:46.137972966 +0000 UTC m=+590.761761815" watchObservedRunningTime="2026-06-02 15:56:46.140662848 +0000 UTC m=+590.764451695"
Jun 02 15:56:46.183181 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:46.183139    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-h7jdl"]
Jun 02 15:56:48.101572 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:48.101534    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-674b59b84c-h7jdl" podUID="572dcb30-f43b-4e3b-bc73-cef31d09b792" containerName="authorino" containerID="cri-o://54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272" gracePeriod=30
Jun 02 15:56:48.347172 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:48.347145    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:48.416493 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:48.416403    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2xm5\" (UniqueName: \"kubernetes.io/projected/572dcb30-f43b-4e3b-bc73-cef31d09b792-kube-api-access-p2xm5\") pod \"572dcb30-f43b-4e3b-bc73-cef31d09b792\" (UID: \"572dcb30-f43b-4e3b-bc73-cef31d09b792\") "
Jun 02 15:56:48.418709 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:48.418678    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572dcb30-f43b-4e3b-bc73-cef31d09b792-kube-api-access-p2xm5" (OuterVolumeSpecName: "kube-api-access-p2xm5") pod "572dcb30-f43b-4e3b-bc73-cef31d09b792" (UID: "572dcb30-f43b-4e3b-bc73-cef31d09b792"). InnerVolumeSpecName "kube-api-access-p2xm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:56:48.517718 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:48.517677    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2xm5\" (UniqueName: \"kubernetes.io/projected/572dcb30-f43b-4e3b-bc73-cef31d09b792-kube-api-access-p2xm5\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:56:49.106942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.106711    2563 generic.go:358] "Generic (PLEG): container finished" podID="572dcb30-f43b-4e3b-bc73-cef31d09b792" containerID="54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272" exitCode=0
Jun 02 15:56:49.106942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.106764    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-h7jdl"
Jun 02 15:56:49.106942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.106771    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-h7jdl" event={"ID":"572dcb30-f43b-4e3b-bc73-cef31d09b792","Type":"ContainerDied","Data":"54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272"}
Jun 02 15:56:49.106942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.106805    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-h7jdl" event={"ID":"572dcb30-f43b-4e3b-bc73-cef31d09b792","Type":"ContainerDied","Data":"130f9d0a0b0b483501d313099841d4146c7cdfec482fa9df65d611ba49907548"}
Jun 02 15:56:49.106942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.106826    2563 scope.go:117] "RemoveContainer" containerID="54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272"
Jun 02 15:56:49.122000 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.121975    2563 scope.go:117] "RemoveContainer" containerID="54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272"
Jun 02 15:56:49.122283 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:56:49.122266    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272\": container with ID starting with 54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272 not found: ID does not exist" containerID="54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272"
Jun 02 15:56:49.122325 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.122291    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272"} err="failed to get container status \"54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272\": rpc error: code = NotFound desc = could not find container \"54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272\": container with ID starting with 54a4a3e25eeaf46c1fd8230dfd94306834c29e8bafab842852154bf7849dc272 not found: ID does not exist"
Jun 02 15:56:49.145666 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.145630    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-h7jdl"]
Jun 02 15:56:49.151632 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.151598    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-674b59b84c-h7jdl"]
Jun 02 15:56:49.981532 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:56:49.981496    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572dcb30-f43b-4e3b-bc73-cef31d09b792" path="/var/lib/kubelet/pods/572dcb30-f43b-4e3b-bc73-cef31d09b792/volumes"
Jun 02 15:57:08.806697 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.806656    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68bd676465-mcvg7"]
Jun 02 15:57:08.807108 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.807067    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572dcb30-f43b-4e3b-bc73-cef31d09b792" containerName="authorino"
Jun 02 15:57:08.807108 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.807080    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="572dcb30-f43b-4e3b-bc73-cef31d09b792" containerName="authorino"
Jun 02 15:57:08.807185 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.807133    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="572dcb30-f43b-4e3b-bc73-cef31d09b792" containerName="authorino"
Jun 02 15:57:08.815931 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.815904    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:08.820554 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.820485    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jun 02 15:57:08.823685 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.823662    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-mcvg7"]
Jun 02 15:57:08.888393 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.888350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9fwc\" (UniqueName: \"kubernetes.io/projected/3f836081-ea4c-49e0-afdc-579993710b48-kube-api-access-h9fwc\") pod \"authorino-68bd676465-mcvg7\" (UID: \"3f836081-ea4c-49e0-afdc-579993710b48\") " pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:08.888393 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.888396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f836081-ea4c-49e0-afdc-579993710b48-tls-cert\") pod \"authorino-68bd676465-mcvg7\" (UID: \"3f836081-ea4c-49e0-afdc-579993710b48\") " pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:08.988942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.988895    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h9fwc\" (UniqueName: \"kubernetes.io/projected/3f836081-ea4c-49e0-afdc-579993710b48-kube-api-access-h9fwc\") pod \"authorino-68bd676465-mcvg7\" (UID: \"3f836081-ea4c-49e0-afdc-579993710b48\") " pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:08.988942 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.988946    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f836081-ea4c-49e0-afdc-579993710b48-tls-cert\") pod \"authorino-68bd676465-mcvg7\" (UID: \"3f836081-ea4c-49e0-afdc-579993710b48\") " pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:08.991620 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:08.991596    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f836081-ea4c-49e0-afdc-579993710b48-tls-cert\") pod \"authorino-68bd676465-mcvg7\" (UID: \"3f836081-ea4c-49e0-afdc-579993710b48\") " pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:09.001703 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:09.001672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9fwc\" (UniqueName: \"kubernetes.io/projected/3f836081-ea4c-49e0-afdc-579993710b48-kube-api-access-h9fwc\") pod \"authorino-68bd676465-mcvg7\" (UID: \"3f836081-ea4c-49e0-afdc-579993710b48\") " pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:09.125614 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:09.125520    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-mcvg7"
Jun 02 15:57:09.266540 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:09.266515    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-mcvg7"]
Jun 02 15:57:09.268747 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:57:09.268719    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f836081_ea4c_49e0_afdc_579993710b48.slice/crio-12dec8636ae6935624658e10e726adeba9c6ffdd9087a9d9a343accad779cd84 WatchSource:0}: Error finding container 12dec8636ae6935624658e10e726adeba9c6ffdd9087a9d9a343accad779cd84: Status 404 returned error can't find the container with id 12dec8636ae6935624658e10e726adeba9c6ffdd9087a9d9a343accad779cd84
Jun 02 15:57:10.186497 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.186400    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-mcvg7" event={"ID":"3f836081-ea4c-49e0-afdc-579993710b48","Type":"ContainerStarted","Data":"f402a6ec456bbb34bf1bb28ff24b70cd5091363e360c50a0cc3f2dd270e39883"}
Jun 02 15:57:10.186497 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.186447    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-mcvg7" event={"ID":"3f836081-ea4c-49e0-afdc-579993710b48","Type":"ContainerStarted","Data":"12dec8636ae6935624658e10e726adeba9c6ffdd9087a9d9a343accad779cd84"}
Jun 02 15:57:10.273951 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.273893    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-68bd676465-mcvg7" podStartSLOduration=1.697062135 podStartE2EDuration="2.273876948s" podCreationTimestamp="2026-06-02 15:57:08 +0000 UTC" firstStartedPulling="2026-06-02 15:57:09.270043315 +0000 UTC m=+613.893832139" lastFinishedPulling="2026-06-02 15:57:09.846858125 +0000 UTC m=+614.470646952" observedRunningTime="2026-06-02 15:57:10.219633604 +0000 UTC m=+614.843422461" watchObservedRunningTime="2026-06-02 15:57:10.273876948 +0000 UTC m=+614.897665795"
Jun 02 15:57:10.274148 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.274006    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mxh4n"]
Jun 02 15:57:10.274248 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.274205    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-79cbc94b89-mxh4n" podUID="6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" containerName="authorino" containerID="cri-o://6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9" gracePeriod=30
Jun 02 15:57:10.532837 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.532810    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:57:10.604546 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.604502    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mww2j\" (UniqueName: \"kubernetes.io/projected/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81-kube-api-access-mww2j\") pod \"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81\" (UID: \"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81\") "
Jun 02 15:57:10.606917 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.606880    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81-kube-api-access-mww2j" (OuterVolumeSpecName: "kube-api-access-mww2j") pod "6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" (UID: "6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81"). InnerVolumeSpecName "kube-api-access-mww2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:57:10.706076 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:10.705943    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mww2j\" (UniqueName: \"kubernetes.io/projected/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81-kube-api-access-mww2j\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:11.190856 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.190817    2563 generic.go:358] "Generic (PLEG): container finished" podID="6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" containerID="6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9" exitCode=0
Jun 02 15:57:11.191331 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.190865    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-mxh4n"
Jun 02 15:57:11.191331 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.190906    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mxh4n" event={"ID":"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81","Type":"ContainerDied","Data":"6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9"}
Jun 02 15:57:11.191331 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.190949    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-mxh4n" event={"ID":"6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81","Type":"ContainerDied","Data":"bd00ace1f1cea20db39bace13870d84e398ae25023ddff4bfe88bd1ce58e35ff"}
Jun 02 15:57:11.191331 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.190971    2563 scope.go:117] "RemoveContainer" containerID="6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9"
Jun 02 15:57:11.200250 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.200234    2563 scope.go:117] "RemoveContainer" containerID="6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9"
Jun 02 15:57:11.200509 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:57:11.200492    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9\": container with ID starting with 6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9 not found: ID does not exist" containerID="6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9"
Jun 02 15:57:11.200556 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.200517    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9"} err="failed to get container status \"6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9\": rpc error: code = NotFound desc = could not find container \"6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9\": container with ID starting with 6fbafce8c1f0febcc5781792eb79bd8899910fb8ed4618aaca434466809930f9 not found: ID does not exist"
Jun 02 15:57:11.217796 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.217747    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mxh4n"]
Jun 02 15:57:11.225405 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.225370    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-mxh4n"]
Jun 02 15:57:11.981223 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:11.981187    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" path="/var/lib/kubelet/pods/6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81/volumes"
Jun 02 15:57:20.365918 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.365869    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"]
Jun 02 15:57:20.366424 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.366278    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" containerName="authorino"
Jun 02 15:57:20.366424 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.366292    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" containerName="authorino"
Jun 02 15:57:20.366424 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.366370    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6ac84dbe-1d3c-4686-ab0d-dcf483d2bc81" containerName="authorino"
Jun 02 15:57:20.369563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.369543    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.421534 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.421499    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"]
Jun 02 15:57:20.496078 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.496278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg5zx\" (UniqueName: \"kubernetes.io/projected/88e65e53-7f53-4667-baf1-c9966e69e948-kube-api-access-vg5zx\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.496278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-cacerts\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.496278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88e65e53-7f53-4667-baf1-c9966e69e948-local-certs\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.496278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88e65e53-7f53-4667-baf1-c9966e69e948-istio-token\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.496278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88e65e53-7f53-4667-baf1-c9966e69e948-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.496278 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.496277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597384 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597345    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597574 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597397    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vg5zx\" (UniqueName: \"kubernetes.io/projected/88e65e53-7f53-4667-baf1-c9966e69e948-kube-api-access-vg5zx\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597574 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597429    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-cacerts\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597574 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597467    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88e65e53-7f53-4667-baf1-c9966e69e948-local-certs\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597574 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597506    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88e65e53-7f53-4667-baf1-c9966e69e948-istio-token\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597777 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597676    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88e65e53-7f53-4667-baf1-c9966e69e948-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.597777 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.597728    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.598718 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.598686    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88e65e53-7f53-4667-baf1-c9966e69e948-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.600042 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.599995    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88e65e53-7f53-4667-baf1-c9966e69e948-local-certs\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.600134 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.600027    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.600203 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.600182    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.600240 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.600205    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88e65e53-7f53-4667-baf1-c9966e69e948-cacerts\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.620034 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.619956    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg5zx\" (UniqueName: \"kubernetes.io/projected/88e65e53-7f53-4667-baf1-c9966e69e948-kube-api-access-vg5zx\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.620667 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.620644    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88e65e53-7f53-4667-baf1-c9966e69e948-istio-token\") pod \"istiod-openshift-gateway-798958bb55-2xdkj\" (UID: \"88e65e53-7f53-4667-baf1-c9966e69e948\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.678791 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.678755    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:20.860791 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.860766    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"]
Jun 02 15:57:20.863049 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:57:20.863003    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e65e53_7f53_4667_baf1_c9966e69e948.slice/crio-8763b84d874c69bd5892332cc582863ab1a74e429a1e60384d61bafaf5d92bcb WatchSource:0}: Error finding container 8763b84d874c69bd5892332cc582863ab1a74e429a1e60384d61bafaf5d92bcb: Status 404 returned error can't find the container with id 8763b84d874c69bd5892332cc582863ab1a74e429a1e60384d61bafaf5d92bcb
Jun 02 15:57:20.865177 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.865149    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:57:20.865241 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:20.865207    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:57:21.234669 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:21.234579    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj" event={"ID":"88e65e53-7f53-4667-baf1-c9966e69e948","Type":"ContainerStarted","Data":"b54f6fee88f9243ac1b46c1f32cd7bb33bc44f8d61c5e1f09f2552e54315161f"}
Jun 02 15:57:21.234669 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:21.234620    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj" event={"ID":"88e65e53-7f53-4667-baf1-c9966e69e948","Type":"ContainerStarted","Data":"8763b84d874c69bd5892332cc582863ab1a74e429a1e60384d61bafaf5d92bcb"}
Jun 02 15:57:21.234892 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:21.234779    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:21.236787 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:21.236760    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-2xdkj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jun 02 15:57:21.236911 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:21.236818    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj" podUID="88e65e53-7f53-4667-baf1-c9966e69e948" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:21.490345 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:21.490220    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj" podStartSLOduration=1.490205176 podStartE2EDuration="1.490205176s" podCreationTimestamp="2026-06-02 15:57:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:57:21.464505354 +0000 UTC m=+626.088294201" watchObservedRunningTime="2026-06-02 15:57:21.490205176 +0000 UTC m=+626.113994022"
Jun 02 15:57:22.239577 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:22.239547    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-2xdkj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jun 02 15:57:22.239745 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:22.239610    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj" podUID="88e65e53-7f53-4667-baf1-c9966e69e948" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 15:57:25.239539 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.239510    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2xdkj"
Jun 02 15:57:25.524736 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.524631    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"]
Jun 02 15:57:25.524952 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.524925    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" containerName="discovery" containerID="cri-o://b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da" gracePeriod=30
Jun 02 15:57:25.784744 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.784670    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:57:25.945930 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.945894    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a359ff-6eda-4d68-b01c-378527114f8b-local-certs\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946135 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.945948    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h25vg\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-kube-api-access-h25vg\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946135 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.945990    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-istio-token\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946135 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.946035    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-ca-configmap\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946308 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.946165    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-kubeconfig\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946308 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.946208    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-dns-cert\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946308 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.946266    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-cacerts\") pod \"72a359ff-6eda-4d68-b01c-378527114f8b\" (UID: \"72a359ff-6eda-4d68-b01c-378527114f8b\") "
Jun 02 15:57:25.946478 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.946401    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 15:57:25.946625 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.946591    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-ca-configmap\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:25.948615 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.948587    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-istio-token" (OuterVolumeSpecName: "istio-token") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:57:25.948823 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.948797    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:57:25.949002 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.948982    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a359ff-6eda-4d68-b01c-378527114f8b-local-certs" (OuterVolumeSpecName: "local-certs") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 15:57:25.949138 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.949089    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:57:25.949138 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.949091    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-kube-api-access-h25vg" (OuterVolumeSpecName: "kube-api-access-h25vg") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "kube-api-access-h25vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:57:25.949426 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:25.949408    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-cacerts" (OuterVolumeSpecName: "cacerts") pod "72a359ff-6eda-4d68-b01c-378527114f8b" (UID: "72a359ff-6eda-4d68-b01c-378527114f8b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:57:26.047357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.047259    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72a359ff-6eda-4d68-b01c-378527114f8b-local-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:26.047357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.047292    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h25vg\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-kube-api-access-h25vg\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:26.047357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.047306    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72a359ff-6eda-4d68-b01c-378527114f8b-istio-token\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:26.047357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.047318    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-kubeconfig\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:26.047357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.047330    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-istio-csr-dns-cert\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:26.047357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.047343    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72a359ff-6eda-4d68-b01c-378527114f8b-cacerts\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:57:26.254850 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.254813    2563 generic.go:358] "Generic (PLEG): container finished" podID="72a359ff-6eda-4d68-b01c-378527114f8b" containerID="b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da" exitCode=0
Jun 02 15:57:26.255266 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.254885    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"
Jun 02 15:57:26.255266 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.254885    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" event={"ID":"72a359ff-6eda-4d68-b01c-378527114f8b","Type":"ContainerDied","Data":"b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da"}
Jun 02 15:57:26.255266 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.254999    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" event={"ID":"72a359ff-6eda-4d68-b01c-378527114f8b","Type":"ContainerDied","Data":"d418ac8a4b9bf93f02e3d128b8b2c587f525ef427f5162a34bc9ae8cbfc0a011"}
Jun 02 15:57:26.255266 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.255038    2563 scope.go:117] "RemoveContainer" containerID="b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da"
Jun 02 15:57:26.264091 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.264066    2563 scope.go:117] "RemoveContainer" containerID="b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da"
Jun 02 15:57:26.264397 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:57:26.264370    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da\": container with ID starting with b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da not found: ID does not exist" containerID="b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da"
Jun 02 15:57:26.264473 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.264407    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da"} err="failed to get container status \"b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da\": rpc error: code = NotFound desc = could not find container \"b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da\": container with ID starting with b9c73da15a31e302d2d2f46b41d604e33f22a28f12b2519260b1386ce793e7da not found: ID does not exist"
Jun 02 15:57:26.301938 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.301905    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"]
Jun 02 15:57:26.316711 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:26.316671    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf"]
Jun 02 15:57:27.982728 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:27.982686    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" path="/var/lib/kubelet/pods/72a359ff-6eda-4d68-b01c-378527114f8b/volumes"
Jun 02 15:57:30.699584 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:30.699548    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-767cf5cf55-9zjzf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jun 02 15:57:30.699998 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:30.699613    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-9zjzf" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.31:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jun 02 15:57:37.917331 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.917284    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-5pqr8"]
Jun 02 15:57:37.919780 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.917806    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" containerName="discovery"
Jun 02 15:57:37.919780 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.917821    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" containerName="discovery"
Jun 02 15:57:37.919780 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.917876    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72a359ff-6eda-4d68-b01c-378527114f8b" containerName="discovery"
Jun 02 15:57:37.920830 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.920810    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:37.925065 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.925042    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jun 02 15:57:37.925263 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.925042    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-8v9rp\""
Jun 02 15:57:37.925263 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.925043    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jun 02 15:57:37.925263 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.925043    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jun 02 15:57:37.932122 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:37.932094    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-5pqr8"]
Jun 02 15:57:38.024688 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.024647    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-79f457656b-kcmhm"]
Jun 02 15:57:38.028269 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.028245    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.036038 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.032658    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jun 02 15:57:38.036038 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.032677    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-pgf2g\""
Jun 02 15:57:38.047229 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.047199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-cert\") pod \"kserve-controller-manager-7b649f477f-5pqr8\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") " pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.047366 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.047237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzs24\" (UniqueName: \"kubernetes.io/projected/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-kube-api-access-mzs24\") pod \"kserve-controller-manager-7b649f477f-5pqr8\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") " pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.053038 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.052976    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-kcmhm"]
Jun 02 15:57:38.148120 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.148087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bf4x\" (UniqueName: \"kubernetes.io/projected/ab5e627c-b245-423e-9d5c-ed89d7bbf8f5-kube-api-access-7bf4x\") pod \"seaweedfs-79f457656b-kcmhm\" (UID: \"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5\") " pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.148333 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.148155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e627c-b245-423e-9d5c-ed89d7bbf8f5-data\") pod \"seaweedfs-79f457656b-kcmhm\" (UID: \"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5\") " pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.148333 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.148224    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-cert\") pod \"kserve-controller-manager-7b649f477f-5pqr8\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") " pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.148333 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.148251    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mzs24\" (UniqueName: \"kubernetes.io/projected/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-kube-api-access-mzs24\") pod \"kserve-controller-manager-7b649f477f-5pqr8\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") " pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.150795 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.150764    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-cert\") pod \"kserve-controller-manager-7b649f477f-5pqr8\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") " pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.163634 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.163598    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzs24\" (UniqueName: \"kubernetes.io/projected/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-kube-api-access-mzs24\") pod \"kserve-controller-manager-7b649f477f-5pqr8\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") " pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.231941 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.231841    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:38.249040 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.248979    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e627c-b245-423e-9d5c-ed89d7bbf8f5-data\") pod \"seaweedfs-79f457656b-kcmhm\" (UID: \"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5\") " pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.249200 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.249097    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7bf4x\" (UniqueName: \"kubernetes.io/projected/ab5e627c-b245-423e-9d5c-ed89d7bbf8f5-kube-api-access-7bf4x\") pod \"seaweedfs-79f457656b-kcmhm\" (UID: \"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5\") " pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.249381 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.249360    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e627c-b245-423e-9d5c-ed89d7bbf8f5-data\") pod \"seaweedfs-79f457656b-kcmhm\" (UID: \"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5\") " pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.260893 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.260864    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bf4x\" (UniqueName: \"kubernetes.io/projected/ab5e627c-b245-423e-9d5c-ed89d7bbf8f5-kube-api-access-7bf4x\") pod \"seaweedfs-79f457656b-kcmhm\" (UID: \"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5\") " pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.343369 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.343334    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:38.377962 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.377935    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-5pqr8"]
Jun 02 15:57:38.379649 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:57:38.379616    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d847a18_af14_4c1c_bc1f_49fbee5f2eac.slice/crio-59f62086218f7eb782815a6e6e0e08b22ec0d3b4210c82689dabc8f47e0ad0c8 WatchSource:0}: Error finding container 59f62086218f7eb782815a6e6e0e08b22ec0d3b4210c82689dabc8f47e0ad0c8: Status 404 returned error can't find the container with id 59f62086218f7eb782815a6e6e0e08b22ec0d3b4210c82689dabc8f47e0ad0c8
Jun 02 15:57:38.381175 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.381153    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 15:57:38.493202 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:38.493174    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-kcmhm"]
Jun 02 15:57:38.494950 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:57:38.494920    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5e627c_b245_423e_9d5c_ed89d7bbf8f5.slice/crio-26f889842a9c6f89ad9a440639cfa6e800d8aac9f80b68b955708fd13cbfdb74 WatchSource:0}: Error finding container 26f889842a9c6f89ad9a440639cfa6e800d8aac9f80b68b955708fd13cbfdb74: Status 404 returned error can't find the container with id 26f889842a9c6f89ad9a440639cfa6e800d8aac9f80b68b955708fd13cbfdb74
Jun 02 15:57:39.306791 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:39.306748    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8" event={"ID":"0d847a18-af14-4c1c-bc1f-49fbee5f2eac","Type":"ContainerStarted","Data":"59f62086218f7eb782815a6e6e0e08b22ec0d3b4210c82689dabc8f47e0ad0c8"}
Jun 02 15:57:39.308155 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:39.308123    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-kcmhm" event={"ID":"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5","Type":"ContainerStarted","Data":"26f889842a9c6f89ad9a440639cfa6e800d8aac9f80b68b955708fd13cbfdb74"}
Jun 02 15:57:42.328703 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:42.328651    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8" event={"ID":"0d847a18-af14-4c1c-bc1f-49fbee5f2eac","Type":"ContainerStarted","Data":"c1010b91c55af088d85b16cb9aa14bd633f90d463bce673f09132e4f5fb4892b"}
Jun 02 15:57:42.329198 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:42.328839    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:57:42.381949 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:42.381760    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8" podStartSLOduration=1.579101105 podStartE2EDuration="5.381741264s" podCreationTimestamp="2026-06-02 15:57:37 +0000 UTC" firstStartedPulling="2026-06-02 15:57:38.381339333 +0000 UTC m=+643.005128160" lastFinishedPulling="2026-06-02 15:57:42.18397949 +0000 UTC m=+646.807768319" observedRunningTime="2026-06-02 15:57:42.372159498 +0000 UTC m=+646.995948345" watchObservedRunningTime="2026-06-02 15:57:42.381741264 +0000 UTC m=+647.005530113"
Jun 02 15:57:43.334114 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:43.334074    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-kcmhm" event={"ID":"ab5e627c-b245-423e-9d5c-ed89d7bbf8f5","Type":"ContainerStarted","Data":"6f690f3f17c51681192bd0531c5b29cc12e66d677b00974a2a7a1a8961cfc00c"}
Jun 02 15:57:43.334585 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:43.334142    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:57:43.365152 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:43.365095    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-79f457656b-kcmhm" podStartSLOduration=2.5796259580000003 podStartE2EDuration="6.365077627s" podCreationTimestamp="2026-06-02 15:57:37 +0000 UTC" firstStartedPulling="2026-06-02 15:57:38.496168474 +0000 UTC m=+643.119957299" lastFinishedPulling="2026-06-02 15:57:42.281620143 +0000 UTC m=+646.905408968" observedRunningTime="2026-06-02 15:57:43.363635149 +0000 UTC m=+647.987423997" watchObservedRunningTime="2026-06-02 15:57:43.365077627 +0000 UTC m=+647.988866526"
Jun 02 15:57:49.340899 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:57:49.340866    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-79f457656b-kcmhm"
Jun 02 15:58:13.339890 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:13.339858    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:58:14.219046 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.218995    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-5pqr8"]
Jun 02 15:58:14.219256 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.219214    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8" podUID="0d847a18-af14-4c1c-bc1f-49fbee5f2eac" containerName="manager" containerID="cri-o://c1010b91c55af088d85b16cb9aa14bd633f90d463bce673f09132e4f5fb4892b" gracePeriod=10
Jun 02 15:58:14.255043 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.254996    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-7dx4d"]
Jun 02 15:58:14.257550 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.257533    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.270458 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.270425    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-7dx4d"]
Jun 02 15:58:14.369469 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.369434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73a342c0-d213-40ce-a603-77ebce7c4354-cert\") pod \"kserve-controller-manager-7b649f477f-7dx4d\" (UID: \"73a342c0-d213-40ce-a603-77ebce7c4354\") " pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.369879 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.369580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9qx2\" (UniqueName: \"kubernetes.io/projected/73a342c0-d213-40ce-a603-77ebce7c4354-kube-api-access-p9qx2\") pod \"kserve-controller-manager-7b649f477f-7dx4d\" (UID: \"73a342c0-d213-40ce-a603-77ebce7c4354\") " pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.455645 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.455500    2563 generic.go:358] "Generic (PLEG): container finished" podID="0d847a18-af14-4c1c-bc1f-49fbee5f2eac" containerID="c1010b91c55af088d85b16cb9aa14bd633f90d463bce673f09132e4f5fb4892b" exitCode=0
Jun 02 15:58:14.455645 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.455587    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8" event={"ID":"0d847a18-af14-4c1c-bc1f-49fbee5f2eac","Type":"ContainerDied","Data":"c1010b91c55af088d85b16cb9aa14bd633f90d463bce673f09132e4f5fb4892b"}
Jun 02 15:58:14.470496 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.470469    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9qx2\" (UniqueName: \"kubernetes.io/projected/73a342c0-d213-40ce-a603-77ebce7c4354-kube-api-access-p9qx2\") pod \"kserve-controller-manager-7b649f477f-7dx4d\" (UID: \"73a342c0-d213-40ce-a603-77ebce7c4354\") " pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.470638 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.470513    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73a342c0-d213-40ce-a603-77ebce7c4354-cert\") pod \"kserve-controller-manager-7b649f477f-7dx4d\" (UID: \"73a342c0-d213-40ce-a603-77ebce7c4354\") " pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.473040 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.472996    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73a342c0-d213-40ce-a603-77ebce7c4354-cert\") pod \"kserve-controller-manager-7b649f477f-7dx4d\" (UID: \"73a342c0-d213-40ce-a603-77ebce7c4354\") " pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.480973 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.480946    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9qx2\" (UniqueName: \"kubernetes.io/projected/73a342c0-d213-40ce-a603-77ebce7c4354-kube-api-access-p9qx2\") pod \"kserve-controller-manager-7b649f477f-7dx4d\" (UID: \"73a342c0-d213-40ce-a603-77ebce7c4354\") " pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.495621 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.495599    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:58:14.571639 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.571605    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-cert\") pod \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") "
Jun 02 15:58:14.571799 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.571709    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzs24\" (UniqueName: \"kubernetes.io/projected/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-kube-api-access-mzs24\") pod \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\" (UID: \"0d847a18-af14-4c1c-bc1f-49fbee5f2eac\") "
Jun 02 15:58:14.573915 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.573883    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-kube-api-access-mzs24" (OuterVolumeSpecName: "kube-api-access-mzs24") pod "0d847a18-af14-4c1c-bc1f-49fbee5f2eac" (UID: "0d847a18-af14-4c1c-bc1f-49fbee5f2eac"). InnerVolumeSpecName "kube-api-access-mzs24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 15:58:14.573915 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.573908    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-cert" (OuterVolumeSpecName: "cert") pod "0d847a18-af14-4c1c-bc1f-49fbee5f2eac" (UID: "0d847a18-af14-4c1c-bc1f-49fbee5f2eac"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 15:58:14.611419 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.611385    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:14.672688 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.672652    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzs24\" (UniqueName: \"kubernetes.io/projected/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-kube-api-access-mzs24\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:58:14.672688 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.672692    2563 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d847a18-af14-4c1c-bc1f-49fbee5f2eac-cert\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 15:58:14.759230 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:14.759199    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-7dx4d"]
Jun 02 15:58:14.760615 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:58:14.760589    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a342c0_d213_40ce_a603_77ebce7c4354.slice/crio-c215a2d8ee1ce097255e09e9a93ea2e5f9f0afc957e3dd7cb0f5110d4b1ed073 WatchSource:0}: Error finding container c215a2d8ee1ce097255e09e9a93ea2e5f9f0afc957e3dd7cb0f5110d4b1ed073: Status 404 returned error can't find the container with id c215a2d8ee1ce097255e09e9a93ea2e5f9f0afc957e3dd7cb0f5110d4b1ed073
Jun 02 15:58:15.460094 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.460053    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d" event={"ID":"73a342c0-d213-40ce-a603-77ebce7c4354","Type":"ContainerStarted","Data":"5ff27d61edd404eee1de43353e148050a9cbf9d307d2c6bff6e9cbd2ae845194"}
Jun 02 15:58:15.460512 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.460101    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d" event={"ID":"73a342c0-d213-40ce-a603-77ebce7c4354","Type":"ContainerStarted","Data":"c215a2d8ee1ce097255e09e9a93ea2e5f9f0afc957e3dd7cb0f5110d4b1ed073"}
Jun 02 15:58:15.460512 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.460125    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:15.461273 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.461252    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8"
Jun 02 15:58:15.461375 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.461250    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-5pqr8" event={"ID":"0d847a18-af14-4c1c-bc1f-49fbee5f2eac","Type":"ContainerDied","Data":"59f62086218f7eb782815a6e6e0e08b22ec0d3b4210c82689dabc8f47e0ad0c8"}
Jun 02 15:58:15.461443 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.461389    2563 scope.go:117] "RemoveContainer" containerID="c1010b91c55af088d85b16cb9aa14bd633f90d463bce673f09132e4f5fb4892b"
Jun 02 15:58:15.489964 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.489910    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d" podStartSLOduration=1.2179811520000001 podStartE2EDuration="1.48989358s" podCreationTimestamp="2026-06-02 15:58:14 +0000 UTC" firstStartedPulling="2026-06-02 15:58:14.761816093 +0000 UTC m=+679.385604919" lastFinishedPulling="2026-06-02 15:58:15.033728523 +0000 UTC m=+679.657517347" observedRunningTime="2026-06-02 15:58:15.487807422 +0000 UTC m=+680.111596270" watchObservedRunningTime="2026-06-02 15:58:15.48989358 +0000 UTC m=+680.113682426"
Jun 02 15:58:15.517570 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.517527    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-5pqr8"]
Jun 02 15:58:15.527949 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.527920    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-5pqr8"]
Jun 02 15:58:15.982401 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:15.982367    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d847a18-af14-4c1c-bc1f-49fbee5f2eac" path="/var/lib/kubelet/pods/0d847a18-af14-4c1c-bc1f-49fbee5f2eac/volumes"
Jun 02 15:58:46.472671 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:46.472593    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7b649f477f-7dx4d"
Jun 02 15:58:47.609861 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.609819    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-4j6g6"]
Jun 02 15:58:47.610399 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.610250    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d847a18-af14-4c1c-bc1f-49fbee5f2eac" containerName="manager"
Jun 02 15:58:47.610399 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.610263    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d847a18-af14-4c1c-bc1f-49fbee5f2eac" containerName="manager"
Jun 02 15:58:47.610399 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.610320    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0d847a18-af14-4c1c-bc1f-49fbee5f2eac" containerName="manager"
Jun 02 15:58:47.613438 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.613412    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:47.616524 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.616502    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jun 02 15:58:47.616639 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.616550    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-kr444\""
Jun 02 15:58:47.626695 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.626667    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-4j6g6"]
Jun 02 15:58:47.665384 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.665350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6xmr\" (UniqueName: \"kubernetes.io/projected/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-kube-api-access-w6xmr\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:47.665547 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.665401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-cert\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:47.766591 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.766553    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6xmr\" (UniqueName: \"kubernetes.io/projected/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-kube-api-access-w6xmr\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:47.766760 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.766617    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-cert\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:47.766760 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:58:47.766706    2563 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jun 02 15:58:47.766833 ip-10-0-140-193 kubenswrapper[2563]: E0602 15:58:47.766766    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-cert podName:cb1fd274-e48c-4c1b-b5ce-65fe8e74753d nodeName:}" failed. No retries permitted until 2026-06-02 15:58:48.266750325 +0000 UTC m=+712.890539149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-cert") pod "odh-model-controller-696fc77849-4j6g6" (UID: "cb1fd274-e48c-4c1b-b5ce-65fe8e74753d") : secret "odh-model-controller-webhook-cert" not found
Jun 02 15:58:47.785196 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:47.785163    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6xmr\" (UniqueName: \"kubernetes.io/projected/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-kube-api-access-w6xmr\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:48.271346 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:48.271308    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-cert\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:48.273880 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:48.273860    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb1fd274-e48c-4c1b-b5ce-65fe8e74753d-cert\") pod \"odh-model-controller-696fc77849-4j6g6\" (UID: \"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d\") " pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:48.524852 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:48.524721    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:48.664907 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:48.664876    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-4j6g6"]
Jun 02 15:58:48.666549 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:58:48.666523    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1fd274_e48c_4c1b_b5ce_65fe8e74753d.slice/crio-a8312a1eb55c691549c1da5bb3baf0926431ba79f55cb2d16efbcfa46d6dff13 WatchSource:0}: Error finding container a8312a1eb55c691549c1da5bb3baf0926431ba79f55cb2d16efbcfa46d6dff13: Status 404 returned error can't find the container with id a8312a1eb55c691549c1da5bb3baf0926431ba79f55cb2d16efbcfa46d6dff13
Jun 02 15:58:49.604381 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:49.604332    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-4j6g6" event={"ID":"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d","Type":"ContainerStarted","Data":"a8312a1eb55c691549c1da5bb3baf0926431ba79f55cb2d16efbcfa46d6dff13"}
Jun 02 15:58:51.617461 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:51.617374    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-4j6g6" event={"ID":"cb1fd274-e48c-4c1b-b5ce-65fe8e74753d","Type":"ContainerStarted","Data":"0d6bd285033fa565a812ccfabc057b3d92538ebd72152af3d5420dc3a54cd5e2"}
Jun 02 15:58:51.617844 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:51.617491    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:58:51.676931 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:58:51.676854    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-4j6g6" podStartSLOduration=2.033969814 podStartE2EDuration="4.676839181s" podCreationTimestamp="2026-06-02 15:58:47 +0000 UTC" firstStartedPulling="2026-06-02 15:58:48.667870722 +0000 UTC m=+713.291659550" lastFinishedPulling="2026-06-02 15:58:51.310740093 +0000 UTC m=+715.934528917" observedRunningTime="2026-06-02 15:58:51.675191911 +0000 UTC m=+716.298980768" watchObservedRunningTime="2026-06-02 15:58:51.676839181 +0000 UTC m=+716.300628028"
Jun 02 15:59:02.624680 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:02.624649    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-4j6g6"
Jun 02 15:59:25.725648 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.725580    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"]
Jun 02 15:59:25.777538 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.777492    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"]
Jun 02 15:59:25.777707 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.777643    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.780910 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.780880    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jun 02 15:59:25.781060 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.781006    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jun 02 15:59:25.781357 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.781340    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-7ntn2\""
Jun 02 15:59:25.782486 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.782467    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
Jun 02 15:59:25.805114 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-envoy\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805114 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-token\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-credential-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-workload-certs\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b77c\" (UniqueName: \"kubernetes.io/projected/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-kube-api-access-7b77c\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-data\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-workload-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.805459 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.805452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-podinfo\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906283 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906239    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-credential-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906283 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906281    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-workload-certs\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906315    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906367    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7b77c\" (UniqueName: \"kubernetes.io/projected/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-kube-api-access-7b77c\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906405    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-data\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906435    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-workload-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906563 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906486    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-podinfo\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906815 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906573    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-envoy\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906815 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906602    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-token\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906815 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906758    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-credential-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906957 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906831    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-data\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.906957 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.906850    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-workload-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.907491 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.907427    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.907745 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.907723    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-workload-certs\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.909548 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.909474    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-envoy\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.909866 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.909849    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-podinfo\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.924208 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.924175    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b77c\" (UniqueName: \"kubernetes.io/projected/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-kube-api-access-7b77c\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:25.924208 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:25.924203    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1104eedb-d3d9-41ec-b3b9-b13de6b78f1e-istio-token\") pod \"router-gateway-1-openshift-default-7b9cf87876-jdt24\" (UID: \"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:26.088852 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.088810    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:26.275028 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.274986    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"]
Jun 02 15:59:26.278203 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:59:26.278171    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1104eedb_d3d9_41ec_b3b9_b13de6b78f1e.slice/crio-ba06d3f4fc2cd79539e645bd842562957b15c38d1adc61cb7123b1ca8322a657 WatchSource:0}: Error finding container ba06d3f4fc2cd79539e645bd842562957b15c38d1adc61cb7123b1ca8322a657: Status 404 returned error can't find the container with id ba06d3f4fc2cd79539e645bd842562957b15c38d1adc61cb7123b1ca8322a657
Jun 02 15:59:26.280520 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.280476    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:59:26.280589 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.280552    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:59:26.280589 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.280586    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jun 02 15:59:26.755458 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.755353    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24" event={"ID":"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e","Type":"ContainerStarted","Data":"13562a5f549d76ec1b37570528debd561128bc78daea2a8abe079ee0708ee4d9"}
Jun 02 15:59:26.755458 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.755412    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24" event={"ID":"1104eedb-d3d9-41ec-b3b9-b13de6b78f1e","Type":"ContainerStarted","Data":"ba06d3f4fc2cd79539e645bd842562957b15c38d1adc61cb7123b1ca8322a657"}
Jun 02 15:59:26.792636 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:26.792564    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24" podStartSLOduration=1.792548885 podStartE2EDuration="1.792548885s" podCreationTimestamp="2026-06-02 15:59:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:59:26.789942852 +0000 UTC m=+751.413731700" watchObservedRunningTime="2026-06-02 15:59:26.792548885 +0000 UTC m=+751.416337732"
Jun 02 15:59:27.089436 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:27.089399    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:27.094834 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:27.094803    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:27.760374 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:27.760336    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:27.761277 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:27.761258    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-jdt24"
Jun 02 15:59:50.103990 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.103941    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"]
Jun 02 15:59:50.108442 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.108420    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.114930 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.114902    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-mzhfw\""
Jun 02 15:59:50.115990 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.115969    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8729g\""
Jun 02 15:59:50.116156 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.116137    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jun 02 15:59:50.125494 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.125455    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"]
Jun 02 15:59:50.225275 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.225238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.225467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.225281    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2vjc\" (UniqueName: \"kubernetes.io/projected/bf103929-09ce-46d2-9b1b-6592180c7079-kube-api-access-q2vjc\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.225467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.225314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf103929-09ce-46d2-9b1b-6592180c7079-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.225467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.225376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.225467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.225399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.225467 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.225414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326237 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326197    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326237 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326245    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326527 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326268    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326527 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326345    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326527 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326366    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2vjc\" (UniqueName: \"kubernetes.io/projected/bf103929-09ce-46d2-9b1b-6592180c7079-kube-api-access-q2vjc\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326527 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326407    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf103929-09ce-46d2-9b1b-6592180c7079-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326741 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326696    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-tmp\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326801 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326755    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326801 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326782    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.326801 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.326788    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-cache\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.329029 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.328990    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf103929-09ce-46d2-9b1b-6592180c7079-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.341434 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.341403    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2vjc\" (UniqueName: \"kubernetes.io/projected/bf103929-09ce-46d2-9b1b-6592180c7079-kube-api-access-q2vjc\") pod \"scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.418375 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.418269    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 15:59:50.587751 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.587719    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"]
Jun 02 15:59:50.588738 ip-10-0-140-193 kubenswrapper[2563]: W0602 15:59:50.588709    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf103929_09ce_46d2_9b1b_6592180c7079.slice/crio-0dc04f1a7df56e7d92f4b600613f662978d367381dc1e9703f3da5732d80a91a WatchSource:0}: Error finding container 0dc04f1a7df56e7d92f4b600613f662978d367381dc1e9703f3da5732d80a91a: Status 404 returned error can't find the container with id 0dc04f1a7df56e7d92f4b600613f662978d367381dc1e9703f3da5732d80a91a
Jun 02 15:59:50.858396 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:50.858354    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerStarted","Data":"0dc04f1a7df56e7d92f4b600613f662978d367381dc1e9703f3da5732d80a91a"}
Jun 02 15:59:54.877523 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:54.877425    2563 generic.go:358] "Generic (PLEG): container finished" podID="bf103929-09ce-46d2-9b1b-6592180c7079" containerID="231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9" exitCode=0
Jun 02 15:59:54.877523 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:54.877499    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerDied","Data":"231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9"}
Jun 02 15:59:56.889157 ip-10-0-140-193 kubenswrapper[2563]: I0602 15:59:56.889108    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerStarted","Data":"8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0"}
Jun 02 16:00:29.034319 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:29.034284    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerStarted","Data":"c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7"}
Jun 02 16:00:29.034783 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:29.034587    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:00:29.037217 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:29.037191    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:00:29.063396 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:29.063332    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" podStartSLOduration=1.191222026 podStartE2EDuration="39.063312652s" podCreationTimestamp="2026-06-02 15:59:50 +0000 UTC" firstStartedPulling="2026-06-02 15:59:50.591110807 +0000 UTC m=+775.214899634" lastFinishedPulling="2026-06-02 16:00:28.463201435 +0000 UTC m=+813.086990260" observedRunningTime="2026-06-02 16:00:29.060871744 +0000 UTC m=+813.684660588" watchObservedRunningTime="2026-06-02 16:00:29.063312652 +0000 UTC m=+813.687101518"
Jun 02 16:00:30.418578 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:30.418528    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:00:30.418578 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:30.418567    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:00:40.420487 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:40.420452    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:00:40.421590 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:00:40.421568    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:01:11.253946 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:11.253907    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"]
Jun 02 16:01:11.254500 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:11.254243    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="main" containerID="cri-o://8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0" gracePeriod=30
Jun 02 16:01:11.254500 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:11.254296    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="tokenizer" containerID="cri-o://c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7" gracePeriod=30
Jun 02 16:01:12.203275 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.203240    2563 generic.go:358] "Generic (PLEG): container finished" podID="bf103929-09ce-46d2-9b1b-6592180c7079" containerID="8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0" exitCode=0
Jun 02 16:01:12.203453 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.203319    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerDied","Data":"8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0"}
Jun 02 16:01:12.600467 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.600441    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:01:12.696674 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696642    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-uds\") pod \"bf103929-09ce-46d2-9b1b-6592180c7079\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") "
Jun 02 16:01:12.696674 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696680    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-kserve-provision-location\") pod \"bf103929-09ce-46d2-9b1b-6592180c7079\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") "
Jun 02 16:01:12.696912 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696744    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-cache\") pod \"bf103929-09ce-46d2-9b1b-6592180c7079\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") "
Jun 02 16:01:12.696912 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696772    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf103929-09ce-46d2-9b1b-6592180c7079-tls-certs\") pod \"bf103929-09ce-46d2-9b1b-6592180c7079\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") "
Jun 02 16:01:12.696912 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696793    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2vjc\" (UniqueName: \"kubernetes.io/projected/bf103929-09ce-46d2-9b1b-6592180c7079-kube-api-access-q2vjc\") pod \"bf103929-09ce-46d2-9b1b-6592180c7079\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") "
Jun 02 16:01:12.696912 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696824    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-tmp\") pod \"bf103929-09ce-46d2-9b1b-6592180c7079\" (UID: \"bf103929-09ce-46d2-9b1b-6592180c7079\") "
Jun 02 16:01:12.697165 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.696936    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "bf103929-09ce-46d2-9b1b-6592180c7079" (UID: "bf103929-09ce-46d2-9b1b-6592180c7079"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:12.697165 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.697047    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "bf103929-09ce-46d2-9b1b-6592180c7079" (UID: "bf103929-09ce-46d2-9b1b-6592180c7079"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:12.697165 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.697084    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-uds\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:12.697285 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.697212    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "bf103929-09ce-46d2-9b1b-6592180c7079" (UID: "bf103929-09ce-46d2-9b1b-6592180c7079"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:12.697463 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.697440    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "bf103929-09ce-46d2-9b1b-6592180c7079" (UID: "bf103929-09ce-46d2-9b1b-6592180c7079"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:12.699167 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.699145    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf103929-09ce-46d2-9b1b-6592180c7079-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf103929-09ce-46d2-9b1b-6592180c7079" (UID: "bf103929-09ce-46d2-9b1b-6592180c7079"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:01:12.699246 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.699157    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf103929-09ce-46d2-9b1b-6592180c7079-kube-api-access-q2vjc" (OuterVolumeSpecName: "kube-api-access-q2vjc") pod "bf103929-09ce-46d2-9b1b-6592180c7079" (UID: "bf103929-09ce-46d2-9b1b-6592180c7079"). InnerVolumeSpecName "kube-api-access-q2vjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:01:12.798221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.798131    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:12.798221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.798166    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:12.798221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.798177    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf103929-09ce-46d2-9b1b-6592180c7079-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:12.798221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.798186    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2vjc\" (UniqueName: \"kubernetes.io/projected/bf103929-09ce-46d2-9b1b-6592180c7079-kube-api-access-q2vjc\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:12.798221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:12.798195    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf103929-09ce-46d2-9b1b-6592180c7079-tokenizer-tmp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:13.209198 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.209154    2563 generic.go:358] "Generic (PLEG): container finished" podID="bf103929-09ce-46d2-9b1b-6592180c7079" containerID="c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7" exitCode=0
Jun 02 16:01:13.209390 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.209208    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerDied","Data":"c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7"}
Jun 02 16:01:13.209390 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.209231    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"
Jun 02 16:01:13.209390 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.209243    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg" event={"ID":"bf103929-09ce-46d2-9b1b-6592180c7079","Type":"ContainerDied","Data":"0dc04f1a7df56e7d92f4b600613f662978d367381dc1e9703f3da5732d80a91a"}
Jun 02 16:01:13.209390 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.209261    2563 scope.go:117] "RemoveContainer" containerID="c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7"
Jun 02 16:01:13.220833 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.220783    2563 scope.go:117] "RemoveContainer" containerID="8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0"
Jun 02 16:01:13.229665 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.229641    2563 scope.go:117] "RemoveContainer" containerID="231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9"
Jun 02 16:01:13.237509 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.237485    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"]
Jun 02 16:01:13.239174 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.238894    2563 scope.go:117] "RemoveContainer" containerID="c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7"
Jun 02 16:01:13.239537 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:01:13.239511    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7\": container with ID starting with c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7 not found: ID does not exist" containerID="c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7"
Jun 02 16:01:13.239717 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.239544    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7"} err="failed to get container status \"c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7\": rpc error: code = NotFound desc = could not find container \"c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7\": container with ID starting with c86f2ad491fa8c9c80894a00dcab60afa5772bdc2845f4a3d5d98fe26cfa17a7 not found: ID does not exist"
Jun 02 16:01:13.239717 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.239570    2563 scope.go:117] "RemoveContainer" containerID="8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0"
Jun 02 16:01:13.239912 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:01:13.239893    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0\": container with ID starting with 8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0 not found: ID does not exist" containerID="8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0"
Jun 02 16:01:13.240002 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.239922    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0"} err="failed to get container status \"8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0\": rpc error: code = NotFound desc = could not find container \"8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0\": container with ID starting with 8d90bd10d2ec1229287ed2e17da1dc228f8478a8a464f5f17786a27d02e1e4d0 not found: ID does not exist"
Jun 02 16:01:13.240002 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.239945    2563 scope.go:117] "RemoveContainer" containerID="231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9"
Jun 02 16:01:13.240254 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:01:13.240236    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9\": container with ID starting with 231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9 not found: ID does not exist" containerID="231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9"
Jun 02 16:01:13.240310 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.240260    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9"} err="failed to get container status \"231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9\": rpc error: code = NotFound desc = could not find container \"231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9\": container with ID starting with 231cb2fee4cb37c5f91591fd5946401d35aebe7f88c08285b3a48870c957aac9 not found: ID does not exist"
Jun 02 16:01:13.241962 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.241922    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-5cf449g5rg"]
Jun 02 16:01:13.982156 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:13.982114    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" path="/var/lib/kubelet/pods/bf103929-09ce-46d2-9b1b-6592180c7079/volumes"
Jun 02 16:01:17.619329 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619298    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"]
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619732    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="storage-initializer"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619744    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="storage-initializer"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619756    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="tokenizer"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619766    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="tokenizer"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619791    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="main"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619799    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="main"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619861    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="tokenizer"
Jun 02 16:01:17.619875 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.619874    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bf103929-09ce-46d2-9b1b-6592180c7079" containerName="main"
Jun 02 16:01:17.623043 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.622998    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.625976 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.625953    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8729g\""
Jun 02 16:01:17.626917 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.626897    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jun 02 16:01:17.626917 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.626907    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-gqrnd\""
Jun 02 16:01:17.631999 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.631978    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"]
Jun 02 16:01:17.641646 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.641617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-tmp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.641780 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.641660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfqmc\" (UniqueName: \"kubernetes.io/projected/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kube-api-access-tfqmc\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.641780 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.641754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-cache\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.641885 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.641790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.641885 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.641809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.641885 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.641834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743140 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743100    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-cache\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743140 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743145    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743356 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743164    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743356 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743283    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743433 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743412    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-tmp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743474 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743442    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tfqmc\" (UniqueName: \"kubernetes.io/projected/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kube-api-access-tfqmc\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743562 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743534    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-cache\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743731 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743581    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-uds\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743829 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743628    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.743829 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.743774    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-tmp\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.745838 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.745820    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.753357 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.753323    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfqmc\" (UniqueName: \"kubernetes.io/projected/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kube-api-access-tfqmc\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:17.933861 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:17.933758    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:18.089210 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:18.089184    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"]
Jun 02 16:01:18.091441 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:01:18.091408    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4adb45_42f6_4ca5_ad38_d186e2a7eac3.slice/crio-4628abb4438f503271d7eee06a283e9c1a2ee2c52901ff5a326c0a9d946c4e21 WatchSource:0}: Error finding container 4628abb4438f503271d7eee06a283e9c1a2ee2c52901ff5a326c0a9d946c4e21: Status 404 returned error can't find the container with id 4628abb4438f503271d7eee06a283e9c1a2ee2c52901ff5a326c0a9d946c4e21
Jun 02 16:01:18.233318 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:18.233219    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerStarted","Data":"615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f"}
Jun 02 16:01:18.233318 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:18.233278    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerStarted","Data":"4628abb4438f503271d7eee06a283e9c1a2ee2c52901ff5a326c0a9d946c4e21"}
Jun 02 16:01:19.238672 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:19.238575    2563 generic.go:358] "Generic (PLEG): container finished" podID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerID="615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f" exitCode=0
Jun 02 16:01:19.239163 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:19.238661    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerDied","Data":"615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f"}
Jun 02 16:01:20.248436 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:20.248398    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerStarted","Data":"6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c"}
Jun 02 16:01:20.248436 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:20.248440    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerStarted","Data":"210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d"}
Jun 02 16:01:20.248865 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:20.248510    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:20.281305 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:20.281248    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" podStartSLOduration=3.281233501 podStartE2EDuration="3.281233501s" podCreationTimestamp="2026-06-02 16:01:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:01:20.278288941 +0000 UTC m=+864.902077789" watchObservedRunningTime="2026-06-02 16:01:20.281233501 +0000 UTC m=+864.905022379"
Jun 02 16:01:27.934111 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:27.934055    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:27.934111 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:27.934096    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:27.936944 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:27.936920    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:28.282080 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:28.281980    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:33.622329 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.622244    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"]
Jun 02 16:01:33.625502 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.625480    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.628747 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.628722    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs\""
Jun 02 16:01:33.629265 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.629243    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-sa-dockercfg-9tzct\""
Jun 02 16:01:33.639448 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.639423    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"]
Jun 02 16:01:33.704434 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.704397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.704595 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.704527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb1818d-c656-4e30-87e1-1113b7aa342b-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.704595 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.704554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.704595 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.704590    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.704706 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.704616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbcpq\" (UniqueName: \"kubernetes.io/projected/4bb1818d-c656-4e30-87e1-1113b7aa342b-kube-api-access-bbcpq\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.704706 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.704649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.805880 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.805838    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806104 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.805942    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb1818d-c656-4e30-87e1-1113b7aa342b-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806104 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.805967    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806104 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806000    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806299 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806183    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbcpq\" (UniqueName: \"kubernetes.io/projected/4bb1818d-c656-4e30-87e1-1113b7aa342b-kube-api-access-bbcpq\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806299 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806234    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806380 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806343    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-tmp\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806380 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-uds\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806552 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806529    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-cache\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.806596 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.806586    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-kserve-provision-location\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.808672 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.808653    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb1818d-c656-4e30-87e1-1113b7aa342b-tls-certs\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.816335 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.816305    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbcpq\" (UniqueName: \"kubernetes.io/projected/4bb1818d-c656-4e30-87e1-1113b7aa342b-kube-api-access-bbcpq\") pod \"llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") " pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:33.959505 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:33.959410    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:34.123451 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:34.123423    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"]
Jun 02 16:01:34.125228 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:01:34.125198    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb1818d_c656_4e30_87e1_1113b7aa342b.slice/crio-2b7bb1e6b818208298c036a8a0b1c4bd3795dd9659e7e507c935740be6ba88ff WatchSource:0}: Error finding container 2b7bb1e6b818208298c036a8a0b1c4bd3795dd9659e7e507c935740be6ba88ff: Status 404 returned error can't find the container with id 2b7bb1e6b818208298c036a8a0b1c4bd3795dd9659e7e507c935740be6ba88ff
Jun 02 16:01:34.312052 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:34.311986    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerStarted","Data":"069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc"}
Jun 02 16:01:34.312233 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:34.312061    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerStarted","Data":"2b7bb1e6b818208298c036a8a0b1c4bd3795dd9659e7e507c935740be6ba88ff"}
Jun 02 16:01:35.317227 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:35.317195    2563 generic.go:358] "Generic (PLEG): container finished" podID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerID="069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc" exitCode=0
Jun 02 16:01:35.317607 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:35.317297    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerDied","Data":"069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc"}
Jun 02 16:01:36.323718 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:36.323684    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerStarted","Data":"1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467"}
Jun 02 16:01:36.324182 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:36.323726    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerStarted","Data":"aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a"}
Jun 02 16:01:36.324182 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:36.323822    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:36.349844 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:36.349787    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" podStartSLOduration=3.349773586 podStartE2EDuration="3.349773586s" podCreationTimestamp="2026-06-02 16:01:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:01:36.3469621 +0000 UTC m=+880.970750945" watchObservedRunningTime="2026-06-02 16:01:36.349773586 +0000 UTC m=+880.973562432"
Jun 02 16:01:43.960538 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:43.960452    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:43.960538 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:43.960497    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:43.963227 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:43.963200    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:44.357856 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:44.357826    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:01:49.287353 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:49.287324    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:50.272424 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:50.272387    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"]
Jun 02 16:01:50.272753 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:50.272719    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="main" containerID="cri-o://210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d" gracePeriod=30
Jun 02 16:01:50.272887 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:50.272768    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="tokenizer" containerID="cri-o://6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c" gracePeriod=30
Jun 02 16:01:51.388698 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.388662    2563 generic.go:358] "Generic (PLEG): container finished" podID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerID="210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d" exitCode=0
Jun 02 16:01:51.388698 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.388702    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerDied","Data":"210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d"}
Jun 02 16:01:51.628320 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.628294    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:51.677160 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677069    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-tmp\") pod \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") "
Jun 02 16:01:51.677305 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677168    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfqmc\" (UniqueName: \"kubernetes.io/projected/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kube-api-access-tfqmc\") pod \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") "
Jun 02 16:01:51.677305 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677200    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tls-certs\") pod \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") "
Jun 02 16:01:51.677305 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677232    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kserve-provision-location\") pod \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") "
Jun 02 16:01:51.677305 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677263    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-cache\") pod \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") "
Jun 02 16:01:51.677507 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677336    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-uds\") pod \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\" (UID: \"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3\") "
Jun 02 16:01:51.677564 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677506    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" (UID: "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:51.677642 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677608    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" (UID: "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:51.677709 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677639    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-tmp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:51.677709 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677698    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" (UID: "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:51.678040 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.677992    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" (UID: "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:01:51.679579 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.679545    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kube-api-access-tfqmc" (OuterVolumeSpecName: "kube-api-access-tfqmc") pod "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" (UID: "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3"). InnerVolumeSpecName "kube-api-access-tfqmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:01:51.679661 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.679574    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" (UID: "cf4adb45-42f6-4ca5-ad38-d186e2a7eac3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:01:51.778481 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.778446    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-uds\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:51.778481 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.778474    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfqmc\" (UniqueName: \"kubernetes.io/projected/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kube-api-access-tfqmc\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:51.778481 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.778485    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:51.778720 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.778497    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:51.778720 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:51.778506    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3-tokenizer-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:01:52.394565 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.394526    2563 generic.go:358] "Generic (PLEG): container finished" podID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerID="6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c" exitCode=0
Jun 02 16:01:52.395066 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.394605    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"
Jun 02 16:01:52.395066 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.394613    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerDied","Data":"6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c"}
Jun 02 16:01:52.395066 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.394652    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv" event={"ID":"cf4adb45-42f6-4ca5-ad38-d186e2a7eac3","Type":"ContainerDied","Data":"4628abb4438f503271d7eee06a283e9c1a2ee2c52901ff5a326c0a9d946c4e21"}
Jun 02 16:01:52.395066 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.394670    2563 scope.go:117] "RemoveContainer" containerID="6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c"
Jun 02 16:01:52.403727 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.403707    2563 scope.go:117] "RemoveContainer" containerID="210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d"
Jun 02 16:01:52.411981 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.411964    2563 scope.go:117] "RemoveContainer" containerID="615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f"
Jun 02 16:01:52.416531 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.416508    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"]
Jun 02 16:01:52.421177 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.421151    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-6c7ffx2nfv"]
Jun 02 16:01:52.422798 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.422775    2563 scope.go:117] "RemoveContainer" containerID="6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c"
Jun 02 16:01:52.423108 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:01:52.423088    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c\": container with ID starting with 6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c not found: ID does not exist" containerID="6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c"
Jun 02 16:01:52.423170 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.423117    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c"} err="failed to get container status \"6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c\": rpc error: code = NotFound desc = could not find container \"6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c\": container with ID starting with 6bcd37283404ed04b608daa079c885aaf378e815245a8cbf4e96d39743298e2c not found: ID does not exist"
Jun 02 16:01:52.423170 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.423137    2563 scope.go:117] "RemoveContainer" containerID="210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d"
Jun 02 16:01:52.423373 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:01:52.423354    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d\": container with ID starting with 210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d not found: ID does not exist" containerID="210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d"
Jun 02 16:01:52.423412 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.423381    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d"} err="failed to get container status \"210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d\": rpc error: code = NotFound desc = could not find container \"210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d\": container with ID starting with 210a34431847723f45531c01b24063d9337a08ea2c54a53eb55c88e22763f38d not found: ID does not exist"
Jun 02 16:01:52.423412 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.423398    2563 scope.go:117] "RemoveContainer" containerID="615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f"
Jun 02 16:01:52.423598 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:01:52.423577    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f\": container with ID starting with 615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f not found: ID does not exist" containerID="615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f"
Jun 02 16:01:52.423637 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:52.423605    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f"} err="failed to get container status \"615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f\": rpc error: code = NotFound desc = could not find container \"615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f\": container with ID starting with 615d10caeba0c020f12e3d376fec1bb0fe7b55bf7328c643efba1133b185226f not found: ID does not exist"
Jun 02 16:01:53.981906 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:53.981871    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" path="/var/lib/kubelet/pods/cf4adb45-42f6-4ca5-ad38-d186e2a7eac3/volumes"
Jun 02 16:01:59.317044 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.316984    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"]
Jun 02 16:01:59.317490 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317431    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="storage-initializer"
Jun 02 16:01:59.317490 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317448    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="storage-initializer"
Jun 02 16:01:59.317490 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317468    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="tokenizer"
Jun 02 16:01:59.317490 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317473    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="tokenizer"
Jun 02 16:01:59.317490 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317492    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="main"
Jun 02 16:01:59.317645 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317497    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="main"
Jun 02 16:01:59.317645 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317562    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="main"
Jun 02 16:01:59.317645 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.317575    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="cf4adb45-42f6-4ca5-ad38-d186e2a7eac3" containerName="tokenizer"
Jun 02 16:01:59.321211 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.321186    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.324192 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.324167    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jun 02 16:01:59.324318 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.324167    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-89dkc\""
Jun 02 16:01:59.334689 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.334663    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"]
Jun 02 16:01:59.346046 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.346002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.346196 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.346089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.346250 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.346198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.346298 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.346260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8m2h\" (UniqueName: \"kubernetes.io/projected/c53ed204-bc3d-4700-a474-38bf4bf3254c-kube-api-access-z8m2h\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.346346 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.346302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.346410 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.346388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53ed204-bc3d-4700-a474-38bf4bf3254c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447180 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447138    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53ed204-bc3d-4700-a474-38bf4bf3254c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447180 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447189    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447441 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447208    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447441 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447245    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447441 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447285    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8m2h\" (UniqueName: \"kubernetes.io/projected/c53ed204-bc3d-4700-a474-38bf4bf3254c-kube-api-access-z8m2h\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447441 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447312    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447647 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447608    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447647 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447618    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447735 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447699    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.447735 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.447703    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.449837 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.449812    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53ed204-bc3d-4700-a474-38bf4bf3254c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.456283 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.456260    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8m2h\" (UniqueName: \"kubernetes.io/projected/c53ed204-bc3d-4700-a474-38bf4bf3254c-kube-api-access-z8m2h\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.633188 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.633091    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:01:59.789130 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:01:59.789095    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"]
Jun 02 16:01:59.790790 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:01:59.790764    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53ed204_bc3d_4700_a474_38bf4bf3254c.slice/crio-330c6691086078d85fadcf4af6b554c3099472c92bb5b996ca5ccb9fd0970e4c WatchSource:0}: Error finding container 330c6691086078d85fadcf4af6b554c3099472c92bb5b996ca5ccb9fd0970e4c: Status 404 returned error can't find the container with id 330c6691086078d85fadcf4af6b554c3099472c92bb5b996ca5ccb9fd0970e4c
Jun 02 16:02:00.426340 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:00.426301    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerStarted","Data":"43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae"}
Jun 02 16:02:00.426340 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:00.426343    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerStarted","Data":"330c6691086078d85fadcf4af6b554c3099472c92bb5b996ca5ccb9fd0970e4c"}
Jun 02 16:02:01.432187 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:01.432147    2563 generic.go:358] "Generic (PLEG): container finished" podID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerID="43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae" exitCode=0
Jun 02 16:02:01.432593 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:01.432225    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerDied","Data":"43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae"}
Jun 02 16:02:02.438525 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:02.438485    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerStarted","Data":"d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e"}
Jun 02 16:02:02.438525 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:02.438529    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerStarted","Data":"096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a"}
Jun 02 16:02:02.438934 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:02.438625    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:02.465979 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:02.465922    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" podStartSLOduration=3.465905471 podStartE2EDuration="3.465905471s" podCreationTimestamp="2026-06-02 16:01:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:02:02.463274723 +0000 UTC m=+907.087063569" watchObservedRunningTime="2026-06-02 16:02:02.465905471 +0000 UTC m=+907.089694336"
Jun 02 16:02:05.361794 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:05.361759    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:02:09.634118 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:09.634067    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:09.634118 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:09.634120    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:09.636975 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:09.636949    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:10.471056 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:10.471027    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:31.475734 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:31.475699    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:33.022454 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:33.022409    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"]
Jun 02 16:02:33.022890 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:33.022800    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="main" containerID="cri-o://096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a" gracePeriod=30
Jun 02 16:02:33.023364 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:33.022936    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="tokenizer" containerID="cri-o://d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e" gracePeriod=30
Jun 02 16:02:33.568459 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:33.568425    2563 generic.go:358] "Generic (PLEG): container finished" podID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerID="096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a" exitCode=0
Jun 02 16:02:33.568605 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:33.568496    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerDied","Data":"096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a"}
Jun 02 16:02:34.378399 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.378370    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:34.575123 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575097    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53ed204-bc3d-4700-a474-38bf4bf3254c-tls-certs\") pod \"c53ed204-bc3d-4700-a474-38bf4bf3254c\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") "
Jun 02 16:02:34.575313 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575151    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-cache\") pod \"c53ed204-bc3d-4700-a474-38bf4bf3254c\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") "
Jun 02 16:02:34.575313 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575189    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-uds\") pod \"c53ed204-bc3d-4700-a474-38bf4bf3254c\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") "
Jun 02 16:02:34.575313 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575206    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-tmp\") pod \"c53ed204-bc3d-4700-a474-38bf4bf3254c\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") "
Jun 02 16:02:34.575313 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575228    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8m2h\" (UniqueName: \"kubernetes.io/projected/c53ed204-bc3d-4700-a474-38bf4bf3254c-kube-api-access-z8m2h\") pod \"c53ed204-bc3d-4700-a474-38bf4bf3254c\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") "
Jun 02 16:02:34.575313 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575274    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-kserve-provision-location\") pod \"c53ed204-bc3d-4700-a474-38bf4bf3254c\" (UID: \"c53ed204-bc3d-4700-a474-38bf4bf3254c\") "
Jun 02 16:02:34.575600 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575389    2563 generic.go:358] "Generic (PLEG): container finished" podID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerID="d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e" exitCode=0
Jun 02 16:02:34.575600 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575436    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "c53ed204-bc3d-4700-a474-38bf4bf3254c" (UID: "c53ed204-bc3d-4700-a474-38bf4bf3254c"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:02:34.575600 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575489    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "c53ed204-bc3d-4700-a474-38bf4bf3254c" (UID: "c53ed204-bc3d-4700-a474-38bf4bf3254c"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:02:34.575600 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575562    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerDied","Data":"d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e"}
Jun 02 16:02:34.575600 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575587    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"
Jun 02 16:02:34.575600 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575596    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77" event={"ID":"c53ed204-bc3d-4700-a474-38bf4bf3254c","Type":"ContainerDied","Data":"330c6691086078d85fadcf4af6b554c3099472c92bb5b996ca5ccb9fd0970e4c"}
Jun 02 16:02:34.576090 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575618    2563 scope.go:117] "RemoveContainer" containerID="d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e"
Jun 02 16:02:34.576090 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.575624    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "c53ed204-bc3d-4700-a474-38bf4bf3254c" (UID: "c53ed204-bc3d-4700-a474-38bf4bf3254c"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:02:34.576212 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.576161    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:02:34.576212 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.576185    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-uds\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:02:34.576318 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.576202    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-tokenizer-tmp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:02:34.576318 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.576284    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c53ed204-bc3d-4700-a474-38bf4bf3254c" (UID: "c53ed204-bc3d-4700-a474-38bf4bf3254c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:02:34.577694 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.577671    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c53ed204-bc3d-4700-a474-38bf4bf3254c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c53ed204-bc3d-4700-a474-38bf4bf3254c" (UID: "c53ed204-bc3d-4700-a474-38bf4bf3254c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:02:34.577811 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.577748    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c53ed204-bc3d-4700-a474-38bf4bf3254c-kube-api-access-z8m2h" (OuterVolumeSpecName: "kube-api-access-z8m2h") pod "c53ed204-bc3d-4700-a474-38bf4bf3254c" (UID: "c53ed204-bc3d-4700-a474-38bf4bf3254c"). InnerVolumeSpecName "kube-api-access-z8m2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:02:34.595930 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.595906    2563 scope.go:117] "RemoveContainer" containerID="096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a"
Jun 02 16:02:34.604649 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.604629    2563 scope.go:117] "RemoveContainer" containerID="43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae"
Jun 02 16:02:34.613221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.613119    2563 scope.go:117] "RemoveContainer" containerID="d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e"
Jun 02 16:02:34.613465 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:02:34.613444    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e\": container with ID starting with d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e not found: ID does not exist" containerID="d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e"
Jun 02 16:02:34.613542 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.613479    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e"} err="failed to get container status \"d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e\": rpc error: code = NotFound desc = could not find container \"d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e\": container with ID starting with d63852230198acb400a036707273def28c7f5919da314dbbc5232f58e229af7e not found: ID does not exist"
Jun 02 16:02:34.613542 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.613505    2563 scope.go:117] "RemoveContainer" containerID="096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a"
Jun 02 16:02:34.613785 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:02:34.613767    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a\": container with ID starting with 096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a not found: ID does not exist" containerID="096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a"
Jun 02 16:02:34.613847 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.613792    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a"} err="failed to get container status \"096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a\": rpc error: code = NotFound desc = could not find container \"096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a\": container with ID starting with 096850e7d14ce2281b19764f0e8803d0deb158e3474749a8a3d3fbef6961911a not found: ID does not exist"
Jun 02 16:02:34.613847 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.613814    2563 scope.go:117] "RemoveContainer" containerID="43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae"
Jun 02 16:02:34.614074 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:02:34.614052    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae\": container with ID starting with 43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae not found: ID does not exist" containerID="43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae"
Jun 02 16:02:34.614190 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.614084    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae"} err="failed to get container status \"43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae\": rpc error: code = NotFound desc = could not find container \"43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae\": container with ID starting with 43547cfd8d23333ede5b5f135311288117d39d04758e107149c100678ae534ae not found: ID does not exist"
Jun 02 16:02:34.676942 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.676901    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c53ed204-bc3d-4700-a474-38bf4bf3254c-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:02:34.676942 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.676935    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8m2h\" (UniqueName: \"kubernetes.io/projected/c53ed204-bc3d-4700-a474-38bf4bf3254c-kube-api-access-z8m2h\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:02:34.676942 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.676946    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c53ed204-bc3d-4700-a474-38bf4bf3254c-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:02:34.903546 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.903510    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"]
Jun 02 16:02:34.907988 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:34.907957    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-bbc5cbf2vt77"]
Jun 02 16:02:35.982073 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:35.982041    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" path="/var/lib/kubelet/pods/c53ed204-bc3d-4700-a474-38bf4bf3254c/volumes"
Jun 02 16:02:36.986451 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.986413    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"]
Jun 02 16:02:36.987070 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987040    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="storage-initializer"
Jun 02 16:02:36.987070 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987062    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="storage-initializer"
Jun 02 16:02:36.987172 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987079    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="tokenizer"
Jun 02 16:02:36.987172 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987088    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="tokenizer"
Jun 02 16:02:36.987172 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987096    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="main"
Jun 02 16:02:36.987172 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987104    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="main"
Jun 02 16:02:36.987294 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987199    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="main"
Jun 02 16:02:36.987294 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.987215    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="c53ed204-bc3d-4700-a474-38bf4bf3254c" containerName="tokenizer"
Jun 02 16:02:36.992226 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.992207    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.995605 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.995573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd832137-4419-4f9c-bbd5-04eda5410466-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.995743 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.995636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.995809 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.995777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpsz2\" (UniqueName: \"kubernetes.io/projected/cd832137-4419-4f9c-bbd5-04eda5410466-kube-api-access-wpsz2\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.995862 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.995830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.995921 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.995867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.995921 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.995899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:36.997063 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.997037    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-d9mnt\""
Jun 02 16:02:36.997196 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:36.997141    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
Jun 02 16:02:37.003892 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.003864    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"]
Jun 02 16:02:37.096898 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.096857    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097119 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.096927    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd832137-4419-4f9c-bbd5-04eda5410466-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097119 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.096955    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097119 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.096993    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wpsz2\" (UniqueName: \"kubernetes.io/projected/cd832137-4419-4f9c-bbd5-04eda5410466-kube-api-access-wpsz2\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097119 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.097026    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097119 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.097047    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097397 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.097357    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097447 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.097376    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097447 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.097432    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.097513 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.097462    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.099858 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.099834    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd832137-4419-4f9c-bbd5-04eda5410466-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.106796 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.106768    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpsz2\" (UniqueName: \"kubernetes.io/projected/cd832137-4419-4f9c-bbd5-04eda5410466-kube-api-access-wpsz2\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.303612 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.303575    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:37.459461 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.459430    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"]
Jun 02 16:02:37.462320 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:02:37.462259    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd832137_4419_4f9c_bbd5_04eda5410466.slice/crio-1d4acb194eb5094e0f2d9e89a381a5cd35550322381ed0ba65566c8ae138f77e WatchSource:0}: Error finding container 1d4acb194eb5094e0f2d9e89a381a5cd35550322381ed0ba65566c8ae138f77e: Status 404 returned error can't find the container with id 1d4acb194eb5094e0f2d9e89a381a5cd35550322381ed0ba65566c8ae138f77e
Jun 02 16:02:37.594901 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.594798    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerStarted","Data":"9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a"}
Jun 02 16:02:37.594901 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:37.594847    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerStarted","Data":"1d4acb194eb5094e0f2d9e89a381a5cd35550322381ed0ba65566c8ae138f77e"}
Jun 02 16:02:38.600315 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:38.600279    2563 generic.go:358] "Generic (PLEG): container finished" podID="cd832137-4419-4f9c-bbd5-04eda5410466" containerID="9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a" exitCode=0
Jun 02 16:02:38.600767 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:38.600378    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerDied","Data":"9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a"}
Jun 02 16:02:38.601626 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:38.601609    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 16:02:39.607489 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:39.607451    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerStarted","Data":"e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096"}
Jun 02 16:02:39.607489 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:39.607489    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerStarted","Data":"fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a"}
Jun 02 16:02:39.607917 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:39.607651    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:39.634749 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:39.634697    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" podStartSLOduration=3.634683637 podStartE2EDuration="3.634683637s" podCreationTimestamp="2026-06-02 16:02:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:02:39.631309422 +0000 UTC m=+944.255098268" watchObservedRunningTime="2026-06-02 16:02:39.634683637 +0000 UTC m=+944.258472483"
Jun 02 16:02:47.304589 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:47.304546    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:47.304589 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:47.304588    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:47.305707 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:02:47.305687    2563 logging.go:55] [core] [Channel #107 SubChannel #108]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.52:9003", ServerName: "10.134.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.52:9003: connect: connection refused"
Jun 02 16:02:47.307078 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:47.307056    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:47.644827 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:47.644738    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:02:48.304946 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:48.304893    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.134.0.52:9003\" within 1s: context deadline exceeded"
Jun 02 16:02:57.304401 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:02:57.304359    2563 logging.go:55] [core] [Channel #115 SubChannel #116]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.52:9003", ServerName: "10.134.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.52:9003: connect: connection refused"
Jun 02 16:02:58.304283 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:02:58.304228    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.134.0.52:9003\" within 1s: context deadline exceeded"
Jun 02 16:03:08.648807 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:03:08.648768    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:16:35.142753 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:35.142712    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"]
Jun 02 16:16:35.145253 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:35.142991    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="main" containerID="cri-o://aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a" gracePeriod=30
Jun 02 16:16:35.145253 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:35.143067    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="tokenizer" containerID="cri-o://1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467" gracePeriod=30
Jun 02 16:16:35.361059 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:16:35.360994    2563 logging.go:55] [core] [Channel #1092 SubChannel #1093]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.50:9003", ServerName: "10.134.0.50:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.50:9003: connect: connection refused"
Jun 02 16:16:35.948431 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:35.948394    2563 generic.go:358] "Generic (PLEG): container finished" podID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerID="aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a" exitCode=0
Jun 02 16:16:35.948595 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:35.948471    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerDied","Data":"aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a"}
Jun 02 16:16:36.361364 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.361322    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.134.0.50:9003\" within 1s: context deadline exceeded"
Jun 02 16:16:36.396783 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.396756    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:16:36.477548 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477515    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-tmp\") pod \"4bb1818d-c656-4e30-87e1-1113b7aa342b\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") "
Jun 02 16:16:36.477747 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477574    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb1818d-c656-4e30-87e1-1113b7aa342b-tls-certs\") pod \"4bb1818d-c656-4e30-87e1-1113b7aa342b\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") "
Jun 02 16:16:36.477747 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477598    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbcpq\" (UniqueName: \"kubernetes.io/projected/4bb1818d-c656-4e30-87e1-1113b7aa342b-kube-api-access-bbcpq\") pod \"4bb1818d-c656-4e30-87e1-1113b7aa342b\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") "
Jun 02 16:16:36.477747 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477617    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-uds\") pod \"4bb1818d-c656-4e30-87e1-1113b7aa342b\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") "
Jun 02 16:16:36.477747 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477659    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-cache\") pod \"4bb1818d-c656-4e30-87e1-1113b7aa342b\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") "
Jun 02 16:16:36.477747 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477693    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-kserve-provision-location\") pod \"4bb1818d-c656-4e30-87e1-1113b7aa342b\" (UID: \"4bb1818d-c656-4e30-87e1-1113b7aa342b\") "
Jun 02 16:16:36.478036 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477921    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "4bb1818d-c656-4e30-87e1-1113b7aa342b" (UID: "4bb1818d-c656-4e30-87e1-1113b7aa342b"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:16:36.478036 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477950    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "4bb1818d-c656-4e30-87e1-1113b7aa342b" (UID: "4bb1818d-c656-4e30-87e1-1113b7aa342b"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:16:36.478036 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.477967    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "4bb1818d-c656-4e30-87e1-1113b7aa342b" (UID: "4bb1818d-c656-4e30-87e1-1113b7aa342b"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:16:36.478489 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.478442    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4bb1818d-c656-4e30-87e1-1113b7aa342b" (UID: "4bb1818d-c656-4e30-87e1-1113b7aa342b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:16:36.479879 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.479856    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bb1818d-c656-4e30-87e1-1113b7aa342b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4bb1818d-c656-4e30-87e1-1113b7aa342b" (UID: "4bb1818d-c656-4e30-87e1-1113b7aa342b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:16:36.479983 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.479966    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb1818d-c656-4e30-87e1-1113b7aa342b-kube-api-access-bbcpq" (OuterVolumeSpecName: "kube-api-access-bbcpq") pod "4bb1818d-c656-4e30-87e1-1113b7aa342b" (UID: "4bb1818d-c656-4e30-87e1-1113b7aa342b"). InnerVolumeSpecName "kube-api-access-bbcpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:16:36.578804 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.578762    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:16:36.578804 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.578798    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-tmp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:16:36.578804 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.578809    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bb1818d-c656-4e30-87e1-1113b7aa342b-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:16:36.579090 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.578818    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbcpq\" (UniqueName: \"kubernetes.io/projected/4bb1818d-c656-4e30-87e1-1113b7aa342b-kube-api-access-bbcpq\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:16:36.579090 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.578827    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-uds\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:16:36.579090 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.578836    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bb1818d-c656-4e30-87e1-1113b7aa342b-tokenizer-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:16:36.954533 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.954492    2563 generic.go:358] "Generic (PLEG): container finished" podID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerID="1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467" exitCode=0
Jun 02 16:16:36.954728 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.954569    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"
Jun 02 16:16:36.954728 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.954597    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerDied","Data":"1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467"}
Jun 02 16:16:36.954728 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.954635    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv" event={"ID":"4bb1818d-c656-4e30-87e1-1113b7aa342b","Type":"ContainerDied","Data":"2b7bb1e6b818208298c036a8a0b1c4bd3795dd9659e7e507c935740be6ba88ff"}
Jun 02 16:16:36.954728 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.954655    2563 scope.go:117] "RemoveContainer" containerID="1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467"
Jun 02 16:16:36.964518 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.964494    2563 scope.go:117] "RemoveContainer" containerID="aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a"
Jun 02 16:16:36.972830 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.972808    2563 scope.go:117] "RemoveContainer" containerID="069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc"
Jun 02 16:16:36.980939 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.980919    2563 scope.go:117] "RemoveContainer" containerID="1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467"
Jun 02 16:16:36.981310 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:16:36.981284    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467\": container with ID starting with 1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467 not found: ID does not exist" containerID="1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467"
Jun 02 16:16:36.981422 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.981318    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467"} err="failed to get container status \"1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467\": rpc error: code = NotFound desc = could not find container \"1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467\": container with ID starting with 1cae50af209d654cfc7bbe9b387eb351abc525523f723a846b8be2753febf467 not found: ID does not exist"
Jun 02 16:16:36.981422 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.981343    2563 scope.go:117] "RemoveContainer" containerID="aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a"
Jun 02 16:16:36.981619 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:16:36.981602    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a\": container with ID starting with aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a not found: ID does not exist" containerID="aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a"
Jun 02 16:16:36.981665 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.981625    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a"} err="failed to get container status \"aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a\": rpc error: code = NotFound desc = could not find container \"aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a\": container with ID starting with aabb4fda6fae20235daa68c7eb64ea91a4de8fd48c459a13f2a092ff47d2660a not found: ID does not exist"
Jun 02 16:16:36.981665 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.981641    2563 scope.go:117] "RemoveContainer" containerID="069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc"
Jun 02 16:16:36.981902 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:16:36.981884    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc\": container with ID starting with 069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc not found: ID does not exist" containerID="069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc"
Jun 02 16:16:36.981954 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.981910    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc"} err="failed to get container status \"069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc\": rpc error: code = NotFound desc = could not find container \"069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc\": container with ID starting with 069ca6a95e995b836221c6a097518d8104555534c3077e1ceeaec5a2634959fc not found: ID does not exist"
Jun 02 16:16:36.993456 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.993423    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"]
Jun 02 16:16:36.996804 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:36.996773    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheqcrtv"]
Jun 02 16:16:37.982677 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:16:37.982641    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" path="/var/lib/kubelet/pods/4bb1818d-c656-4e30-87e1-1113b7aa342b/volumes"
Jun 02 16:17:36.342215 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:36.342180    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"]
Jun 02 16:17:36.342744 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:36.342481    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="main" containerID="cri-o://fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a" gracePeriod=30
Jun 02 16:17:36.342744 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:36.342540    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="tokenizer" containerID="cri-o://e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096" gracePeriod=30
Jun 02 16:17:37.199239 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:37.199205    2563 generic.go:358] "Generic (PLEG): container finished" podID="cd832137-4419-4f9c-bbd5-04eda5410466" containerID="fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a" exitCode=0
Jun 02 16:17:37.199471 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:37.199283    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerDied","Data":"fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a"}
Jun 02 16:17:37.643986 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:37.643939    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="tokenizer" probeResult="failure" output="Get \"http://10.134.0.52:8082/healthz\": dial tcp 10.134.0.52:8082: connect: connection refused"
Jun 02 16:17:37.906671 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:37.906609    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:17:38.010451 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010415    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd832137-4419-4f9c-bbd5-04eda5410466-tls-certs\") pod \"cd832137-4419-4f9c-bbd5-04eda5410466\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") "
Jun 02 16:17:38.010451 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010455    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-cache\") pod \"cd832137-4419-4f9c-bbd5-04eda5410466\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") "
Jun 02 16:17:38.010677 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010481    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpsz2\" (UniqueName: \"kubernetes.io/projected/cd832137-4419-4f9c-bbd5-04eda5410466-kube-api-access-wpsz2\") pod \"cd832137-4419-4f9c-bbd5-04eda5410466\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") "
Jun 02 16:17:38.010677 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010543    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-kserve-provision-location\") pod \"cd832137-4419-4f9c-bbd5-04eda5410466\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") "
Jun 02 16:17:38.010677 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010599    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-uds\") pod \"cd832137-4419-4f9c-bbd5-04eda5410466\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") "
Jun 02 16:17:38.010677 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010630    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-tmp\") pod \"cd832137-4419-4f9c-bbd5-04eda5410466\" (UID: \"cd832137-4419-4f9c-bbd5-04eda5410466\") "
Jun 02 16:17:38.010958 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010903    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "cd832137-4419-4f9c-bbd5-04eda5410466" (UID: "cd832137-4419-4f9c-bbd5-04eda5410466"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:38.010958 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.010935    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "cd832137-4419-4f9c-bbd5-04eda5410466" (UID: "cd832137-4419-4f9c-bbd5-04eda5410466"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:38.011143 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.011038    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "cd832137-4419-4f9c-bbd5-04eda5410466" (UID: "cd832137-4419-4f9c-bbd5-04eda5410466"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:38.011711 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.011683    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cd832137-4419-4f9c-bbd5-04eda5410466" (UID: "cd832137-4419-4f9c-bbd5-04eda5410466"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:38.012806 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.012787    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd832137-4419-4f9c-bbd5-04eda5410466-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd832137-4419-4f9c-bbd5-04eda5410466" (UID: "cd832137-4419-4f9c-bbd5-04eda5410466"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:17:38.012876 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.012836    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd832137-4419-4f9c-bbd5-04eda5410466-kube-api-access-wpsz2" (OuterVolumeSpecName: "kube-api-access-wpsz2") pod "cd832137-4419-4f9c-bbd5-04eda5410466" (UID: "cd832137-4419-4f9c-bbd5-04eda5410466"). InnerVolumeSpecName "kube-api-access-wpsz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:17:38.111594 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.111555    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-tmp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:38.111594 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.111587    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd832137-4419-4f9c-bbd5-04eda5410466-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:38.111594 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.111597    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:38.111830 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.111606    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpsz2\" (UniqueName: \"kubernetes.io/projected/cd832137-4419-4f9c-bbd5-04eda5410466-kube-api-access-wpsz2\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:38.111830 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.111617    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:38.111830 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.111626    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cd832137-4419-4f9c-bbd5-04eda5410466-tokenizer-uds\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:38.204599 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.204519    2563 generic.go:358] "Generic (PLEG): container finished" podID="cd832137-4419-4f9c-bbd5-04eda5410466" containerID="e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096" exitCode=0
Jun 02 16:17:38.204599 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.204589    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"
Jun 02 16:17:38.204772 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.204632    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerDied","Data":"e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096"}
Jun 02 16:17:38.204772 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.204657    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk" event={"ID":"cd832137-4419-4f9c-bbd5-04eda5410466","Type":"ContainerDied","Data":"1d4acb194eb5094e0f2d9e89a381a5cd35550322381ed0ba65566c8ae138f77e"}
Jun 02 16:17:38.204772 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.204675    2563 scope.go:117] "RemoveContainer" containerID="e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096"
Jun 02 16:17:38.214174 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.214156    2563 scope.go:117] "RemoveContainer" containerID="fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a"
Jun 02 16:17:38.228301 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.228262    2563 scope.go:117] "RemoveContainer" containerID="9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a"
Jun 02 16:17:38.231428 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.231399    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"]
Jun 02 16:17:38.237219 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.237197    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5c94dccbvm5lk"]
Jun 02 16:17:38.238088 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.238068    2563 scope.go:117] "RemoveContainer" containerID="e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096"
Jun 02 16:17:38.238337 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:38.238316    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096\": container with ID starting with e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096 not found: ID does not exist" containerID="e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096"
Jun 02 16:17:38.238414 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.238348    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096"} err="failed to get container status \"e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096\": rpc error: code = NotFound desc = could not find container \"e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096\": container with ID starting with e9f57b5a092da8b3f87e66808b2e6ecdf32ba00c616427c481cd81ca87c6f096 not found: ID does not exist"
Jun 02 16:17:38.238414 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.238372    2563 scope.go:117] "RemoveContainer" containerID="fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a"
Jun 02 16:17:38.238600 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:38.238581    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a\": container with ID starting with fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a not found: ID does not exist" containerID="fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a"
Jun 02 16:17:38.238651 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.238612    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a"} err="failed to get container status \"fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a\": rpc error: code = NotFound desc = could not find container \"fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a\": container with ID starting with fbe3f556546f55de3497a4d053ac5aebb58ad651b23ae03ebad1489cf946e01a not found: ID does not exist"
Jun 02 16:17:38.238651 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.238634    2563 scope.go:117] "RemoveContainer" containerID="9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a"
Jun 02 16:17:38.238882 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:38.238853    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a\": container with ID starting with 9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a not found: ID does not exist" containerID="9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a"
Jun 02 16:17:38.238947 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:38.238889    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a"} err="failed to get container status \"9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a\": rpc error: code = NotFound desc = could not find container \"9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a\": container with ID starting with 9308569d84ac5f5be17baf7a4789cb1c63c08f4101991c14afd2984cf636756a not found: ID does not exist"
Jun 02 16:17:39.506291 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506259    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"]
Jun 02 16:17:39.506744 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506726    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="main"
Jun 02 16:17:39.506744 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506744    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="main"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506766    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="storage-initializer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506775    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="storage-initializer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506787    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="tokenizer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506795    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="tokenizer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506819    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="main"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506827    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="main"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506837    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="storage-initializer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506844    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="storage-initializer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506854    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="tokenizer"
Jun 02 16:17:39.506882 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506861    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="tokenizer"
Jun 02 16:17:39.507314 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506949    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="tokenizer"
Jun 02 16:17:39.507314 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506965    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="main"
Jun 02 16:17:39.507314 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506976    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="4bb1818d-c656-4e30-87e1-1113b7aa342b" containerName="main"
Jun 02 16:17:39.507314 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.506983    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" containerName="tokenizer"
Jun 02 16:17:39.511935 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.511918    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.515501 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.515480    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"conv-test-lora-crit-kserve-self-signed-certs\""
Jun 02 16:17:39.515601 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.515480    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8729g\""
Jun 02 16:17:39.526532 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.526509    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"]
Jun 02 16:17:39.625415 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-home\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.625597 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-kserve-provision-location\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.625597 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-tmp-dir\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.625597 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f98ffc7-1197-4b56-957c-72c8171d19c3-tls-certs\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.625704 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-dshm\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.625704 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-model-cache\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.625770 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.625701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nsbn\" (UniqueName: \"kubernetes.io/projected/1f98ffc7-1197-4b56-957c-72c8171d19c3-kube-api-access-6nsbn\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726565 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726518    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-home\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726565 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726563    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-kserve-provision-location\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726806 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726584    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-tmp-dir\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726806 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726607    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f98ffc7-1197-4b56-957c-72c8171d19c3-tls-certs\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726806 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726729    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-dshm\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726806 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726754    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-model-cache\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.726806 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726774    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nsbn\" (UniqueName: \"kubernetes.io/projected/1f98ffc7-1197-4b56-957c-72c8171d19c3-kube-api-access-6nsbn\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.727087 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.726994    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-kserve-provision-location\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.727087 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.727056    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-tmp-dir\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.727087 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.727073    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-home\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.727230 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.727181    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-model-cache\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.729194 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.729171    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-dshm\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.729438 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.729424    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f98ffc7-1197-4b56-957c-72c8171d19c3-tls-certs\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.738408 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.738374    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nsbn\" (UniqueName: \"kubernetes.io/projected/1f98ffc7-1197-4b56-957c-72c8171d19c3-kube-api-access-6nsbn\") pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") " pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.821834 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.821800    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:39.982796 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:39.982759    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd832137-4419-4f9c-bbd5-04eda5410466" path="/var/lib/kubelet/pods/cd832137-4419-4f9c-bbd5-04eda5410466/volumes"
Jun 02 16:17:40.015531 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:40.015494    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"]
Jun 02 16:17:40.019515 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:17:40.019483    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f98ffc7_1197_4b56_957c_72c8171d19c3.slice/crio-e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f WatchSource:0}: Error finding container e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f: Status 404 returned error can't find the container with id e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f
Jun 02 16:17:40.021273 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:40.021256    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 16:17:40.215477 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:40.215445    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" event={"ID":"1f98ffc7-1197-4b56-957c-72c8171d19c3","Type":"ContainerStarted","Data":"7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477"}
Jun 02 16:17:40.215477 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:40.215484    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" event={"ID":"1f98ffc7-1197-4b56-957c-72c8171d19c3","Type":"ContainerStarted","Data":"e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f"}
Jun 02 16:17:41.220441 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:41.220412    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_1f98ffc7-1197-4b56-957c-72c8171d19c3/storage-initializer/0.log"
Jun 02 16:17:41.220864 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:41.220454    2563 generic.go:358] "Generic (PLEG): container finished" podID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerID="7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477" exitCode=1
Jun 02 16:17:41.220864 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:41.220550    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" event={"ID":"1f98ffc7-1197-4b56-957c-72c8171d19c3","Type":"ContainerDied","Data":"7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477"}
Jun 02 16:17:42.225615 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:42.225589    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_1f98ffc7-1197-4b56-957c-72c8171d19c3/storage-initializer/1.log"
Jun 02 16:17:42.226353 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:42.225941    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_1f98ffc7-1197-4b56-957c-72c8171d19c3/storage-initializer/0.log"
Jun 02 16:17:42.226353 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:42.225974    2563 generic.go:358] "Generic (PLEG): container finished" podID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerID="abb40663176b747b23a4ccf9acf3a2eab9c05aa5557be61364727743f9fb8700" exitCode=1
Jun 02 16:17:42.226353 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:42.226071    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" event={"ID":"1f98ffc7-1197-4b56-957c-72c8171d19c3","Type":"ContainerDied","Data":"abb40663176b747b23a4ccf9acf3a2eab9c05aa5557be61364727743f9fb8700"}
Jun 02 16:17:42.226353 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:42.226115    2563 scope.go:117] "RemoveContainer" containerID="7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477"
Jun 02 16:17:42.226353 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:42.226344    2563 scope.go:117] "RemoveContainer" containerID="7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477"
Jun 02 16:17:42.238170 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:42.238132    2563 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_kserve-ci-e2e-test_1f98ffc7-1197-4b56-957c-72c8171d19c3_0 in pod sandbox e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f from index: no such id: '7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477'" containerID="7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477"
Jun 02 16:17:42.238247 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:42.238194    2563 kuberuntime_container.go:951] "Unhandled Error" err="failed to remove pod init container \"storage-initializer\": rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_kserve-ci-e2e-test_1f98ffc7-1197-4b56-957c-72c8171d19c3_0 in pod sandbox e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f from index: no such id: '7198faafd411fbf590b76dc3cc1904944769abec379bf66c2794772163318477'; Skipping pod \"conv-test-lora-crit-kserve-88b4dc55f-xdwhw_kserve-ci-e2e-test(1f98ffc7-1197-4b56-957c-72c8171d19c3)\"" logger="UnhandledError"
Jun 02 16:17:42.239529 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:42.239509    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=conv-test-lora-crit-kserve-88b4dc55f-xdwhw_kserve-ci-e2e-test(1f98ffc7-1197-4b56-957c-72c8171d19c3)\"" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3"
Jun 02 16:17:43.231641 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:43.231614    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_1f98ffc7-1197-4b56-957c-72c8171d19c3/storage-initializer/1.log"
Jun 02 16:17:43.232387 ip-10-0-140-193 kubenswrapper[2563]: E0602 16:17:43.232361    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=conv-test-lora-crit-kserve-88b4dc55f-xdwhw_kserve-ci-e2e-test(1f98ffc7-1197-4b56-957c-72c8171d19c3)\"" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3"
Jun 02 16:17:50.859206 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:50.859168    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"]
Jun 02 16:17:50.863289 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:50.863261    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:50.866224 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:50.866189    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jun 02 16:17:50.866224 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:50.866189    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-zvk8p\""
Jun 02 16:17:50.875960 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:50.875929    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"]
Jun 02 16:17:51.020727 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.020688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e269a9-e937-4780-b4f0-756a8511cf2b-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.020884 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.020735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.020884 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.020765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.020884 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.020791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.020884 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.020806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzdwl\" (UniqueName: \"kubernetes.io/projected/42e269a9-e937-4780-b4f0-756a8511cf2b-kube-api-access-hzdwl\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.020884 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.020851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.121951 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.121877    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e269a9-e937-4780-b4f0-756a8511cf2b-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.121951 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.121919    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122368 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.121960    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122368 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122034    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122368 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122070    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hzdwl\" (UniqueName: \"kubernetes.io/projected/42e269a9-e937-4780-b4f0-756a8511cf2b-kube-api-access-hzdwl\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122368 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122101    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122553 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122406    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122553 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122450    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122553 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122473    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.122553 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.122503    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/42e269a9-e937-4780-b4f0-756a8511cf2b-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.124502 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.124481    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e269a9-e937-4780-b4f0-756a8511cf2b-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.132488 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.132467    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzdwl\" (UniqueName: \"kubernetes.io/projected/42e269a9-e937-4780-b4f0-756a8511cf2b-kube-api-access-hzdwl\") pod \"stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts\" (UID: \"42e269a9-e937-4780-b4f0-756a8511cf2b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.175566 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.175521    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:51.342276 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:51.342247    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"]
Jun 02 16:17:52.270931 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:52.270897    2563 generic.go:358] "Generic (PLEG): container finished" podID="42e269a9-e937-4780-b4f0-756a8511cf2b" containerID="686d09a429976ec5b9f286e6a7ebecc91971e3e57e2d69289dd8f488c3ee686d" exitCode=0
Jun 02 16:17:52.271334 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:52.270989    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" event={"ID":"42e269a9-e937-4780-b4f0-756a8511cf2b","Type":"ContainerDied","Data":"686d09a429976ec5b9f286e6a7ebecc91971e3e57e2d69289dd8f488c3ee686d"}
Jun 02 16:17:52.271334 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:52.271038    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" event={"ID":"42e269a9-e937-4780-b4f0-756a8511cf2b","Type":"ContainerStarted","Data":"08a2124e87864663f1f08b4492b45be7fce3793015601a3c8cde53cfc11e711d"}
Jun 02 16:17:53.277856 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.277824    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" event={"ID":"42e269a9-e937-4780-b4f0-756a8511cf2b","Type":"ContainerStarted","Data":"3521ce09ed7ac551fcfbbf300f728b3d2a2dfe7c9413629008b1af5a78798045"}
Jun 02 16:17:53.277856 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.277858    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" event={"ID":"42e269a9-e937-4780-b4f0-756a8511cf2b","Type":"ContainerStarted","Data":"5744bc366ba869c0d1c727cbcf7fed67634873c99e36dd8a33c7744bb05db4e5"}
Jun 02 16:17:53.278430 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.277986    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:17:53.305086 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.305041    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" podStartSLOduration=3.3050053999999998 podStartE2EDuration="3.3050054s" podCreationTimestamp="2026-06-02 16:17:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:17:53.301168502 +0000 UTC m=+1857.924957349" watchObservedRunningTime="2026-06-02 16:17:53.3050054 +0000 UTC m=+1857.928794247"
Jun 02 16:17:53.752226 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.752190    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"]
Jun 02 16:17:53.896718 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.896696    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_1f98ffc7-1197-4b56-957c-72c8171d19c3/storage-initializer/1.log"
Jun 02 16:17:53.896853 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:53.896771    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:54.051457 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051429    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nsbn\" (UniqueName: \"kubernetes.io/projected/1f98ffc7-1197-4b56-957c-72c8171d19c3-kube-api-access-6nsbn\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051617 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051492    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-home\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051617 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051523    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-dshm\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051617 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051550    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-model-cache\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051617 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051588    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-kserve-provision-location\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051617 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051611    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f98ffc7-1197-4b56-957c-72c8171d19c3-tls-certs\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051887 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051658    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-tmp-dir\") pod \"1f98ffc7-1197-4b56-957c-72c8171d19c3\" (UID: \"1f98ffc7-1197-4b56-957c-72c8171d19c3\") "
Jun 02 16:17:54.051887 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051723    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-home" (OuterVolumeSpecName: "home") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:54.051887 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051825    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:54.052081 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051928    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-home\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.052081 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051947    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-model-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.052081 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051967    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:54.052081 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.051985    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:54.053784 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.053759    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f98ffc7-1197-4b56-957c-72c8171d19c3-kube-api-access-6nsbn" (OuterVolumeSpecName: "kube-api-access-6nsbn") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "kube-api-access-6nsbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:17:54.053896 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.053812    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-dshm" (OuterVolumeSpecName: "dshm") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:17:54.053896 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.053867    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f98ffc7-1197-4b56-957c-72c8171d19c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f98ffc7-1197-4b56-957c-72c8171d19c3" (UID: "1f98ffc7-1197-4b56-957c-72c8171d19c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:17:54.153194 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.153155    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-dshm\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.153194 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.153188    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.153194 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.153202    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f98ffc7-1197-4b56-957c-72c8171d19c3-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.153429 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.153214    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f98ffc7-1197-4b56-957c-72c8171d19c3-tmp-dir\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.153429 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.153226    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nsbn\" (UniqueName: \"kubernetes.io/projected/1f98ffc7-1197-4b56-957c-72c8171d19c3-kube-api-access-6nsbn\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:17:54.283126 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.283095    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_conv-test-lora-crit-kserve-88b4dc55f-xdwhw_1f98ffc7-1197-4b56-957c-72c8171d19c3/storage-initializer/1.log"
Jun 02 16:17:54.283518 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.283226    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"
Jun 02 16:17:54.283518 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.283238    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw" event={"ID":"1f98ffc7-1197-4b56-957c-72c8171d19c3","Type":"ContainerDied","Data":"e8396e936f417927fff6dc176da97006be39c3bab8efad12f744ec8a30e37e5f"}
Jun 02 16:17:54.283518 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.283287    2563 scope.go:117] "RemoveContainer" containerID="abb40663176b747b23a4ccf9acf3a2eab9c05aa5557be61364727743f9fb8700"
Jun 02 16:17:54.324782 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.324700    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"]
Jun 02 16:17:54.328400 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:54.328372    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/conv-test-lora-crit-kserve-88b4dc55f-xdwhw"]
Jun 02 16:17:55.981702 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:17:55.981666    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" path="/var/lib/kubelet/pods/1f98ffc7-1197-4b56-957c-72c8171d19c3/volumes"
Jun 02 16:18:01.175865 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:18:01.175821    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:18:01.176324 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:18:01.175989    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:18:01.178561 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:18:01.178533    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:18:01.312708 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:18:01.312682    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:18:23.322238 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:18:23.322164    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts"
Jun 02 16:20:56.976959 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:20:56.976923    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:22:20.977310 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:22:20.977276    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:23:44.977566 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:23:44.977535    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:24:53.977143 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:24:53.977108    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:26:03.977033 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:26:03.976917    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:27:06.977172 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:27:06.977138    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:28:25.979872 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:28:25.979835    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:29:26.977144 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:29:26.977107    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:30:56.977311 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:30:56.977275    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:32:02.977158 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:32:02.977123    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:33:19.977688 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:33:19.977601    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:34:49.976839 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:34:49.976740    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:36:02.977686 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:36:02.977650    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:37:04.976669 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:37:04.976631    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:38:33.981853 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:38:33.981812    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:40:03.977743 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:40:03.977703    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:41:05.980536 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:41:05.980449    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:42:31.977657 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:42:31.977622    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:43:46.977236 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:43:46.977151    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:44:52.977245 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:44:52.977214    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:46:01.977331 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:46:01.977290    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:47:02.977780 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:47:02.977748    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:48:05.981708 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:48:05.981674    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:49:25.982169 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:49:25.982135    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:50:29.977538 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:50:29.977499    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:51:24.246311 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.246232    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"]
Jun 02 16:51:24.246751 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.246606    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerName="storage-initializer"
Jun 02 16:51:24.246751 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.246618    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerName="storage-initializer"
Jun 02 16:51:24.246751 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.246630    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerName="storage-initializer"
Jun 02 16:51:24.246751 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.246635    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerName="storage-initializer"
Jun 02 16:51:24.250183 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.250146    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerName="storage-initializer"
Jun 02 16:51:24.250346 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.250202    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f98ffc7-1197-4b56-957c-72c8171d19c3" containerName="storage-initializer"
Jun 02 16:51:24.255605 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.255574    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.258717 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.258667    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa-dockercfg-w7spk\""
Jun 02 16:51:24.258717 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.258699    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\""
Jun 02 16:51:24.260319 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.260297    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"]
Jun 02 16:51:24.380422 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.380375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.380422 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.380421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pstk5\" (UniqueName: \"kubernetes.io/projected/1445060d-06de-4379-beb0-d2a9929cbcff-kube-api-access-pstk5\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.380685 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.380507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.380685 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.380532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.380685 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.380584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.380685 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.380660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1445060d-06de-4379-beb0-d2a9929cbcff-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.481831 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.481783    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.481831 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.481834    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482161 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.481887    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482161 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482058    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1445060d-06de-4379-beb0-d2a9929cbcff-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482161 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482148    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482341 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482178    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pstk5\" (UniqueName: \"kubernetes.io/projected/1445060d-06de-4379-beb0-d2a9929cbcff-kube-api-access-pstk5\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482341 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482228    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482341 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482278    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482341 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482320    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.482490 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.482421    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.484647 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.484630    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1445060d-06de-4379-beb0-d2a9929cbcff-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.491314 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.491292    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pstk5\" (UniqueName: \"kubernetes.io/projected/1445060d-06de-4379-beb0-d2a9929cbcff-kube-api-access-pstk5\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.567611 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.567566    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:24.710700 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.710673    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"]
Jun 02 16:51:24.713103 ip-10-0-140-193 kubenswrapper[2563]: W0602 16:51:24.713067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1445060d_06de_4379_beb0_d2a9929cbcff.slice/crio-e42a4f6c5a53e19c49486da9593fdd658958a69d84ba55a80d742f01e9238706 WatchSource:0}: Error finding container e42a4f6c5a53e19c49486da9593fdd658958a69d84ba55a80d742f01e9238706: Status 404 returned error can't find the container with id e42a4f6c5a53e19c49486da9593fdd658958a69d84ba55a80d742f01e9238706
Jun 02 16:51:24.714946 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:24.714932    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 16:51:25.221960 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:25.221918    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerStarted","Data":"189f8f2c25fe90c9ce2dfbda1cce1254e275d97919e5f2c8195686050c81a844"}
Jun 02 16:51:25.221960 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:25.221958    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerStarted","Data":"e42a4f6c5a53e19c49486da9593fdd658958a69d84ba55a80d742f01e9238706"}
Jun 02 16:51:26.227333 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:26.227300    2563 generic.go:358] "Generic (PLEG): container finished" podID="1445060d-06de-4379-beb0-d2a9929cbcff" containerID="189f8f2c25fe90c9ce2dfbda1cce1254e275d97919e5f2c8195686050c81a844" exitCode=0
Jun 02 16:51:26.227766 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:26.227392    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerDied","Data":"189f8f2c25fe90c9ce2dfbda1cce1254e275d97919e5f2c8195686050c81a844"}
Jun 02 16:51:27.233890 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:27.233856    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerStarted","Data":"12a15e2417337ca80da8cd9ec5b6a619b92a82d77628a39fafb24c45fd0b7b9b"}
Jun 02 16:51:27.233890 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:27.233894    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerStarted","Data":"752b568859dfadfe7ad6469f794adbcb79931c07b1d81759693f5ec8ff818cf8"}
Jun 02 16:51:27.234471 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:27.233985    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:27.260221 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:27.260166    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" podStartSLOduration=3.260151079 podStartE2EDuration="3.260151079s" podCreationTimestamp="2026-06-02 16:51:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:51:27.257006426 +0000 UTC m=+3871.880795275" watchObservedRunningTime="2026-06-02 16:51:27.260151079 +0000 UTC m=+3871.883939925"
Jun 02 16:51:34.568670 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:34.568625    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:34.568670 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:34.568674    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:34.571349 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:34.571323    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:35.267938 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:35.267906    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:51:55.980940 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:55.980636    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:51:56.272703 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:51:56.272626    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:53:24.977628 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:53:24.977596    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:54:02.727620 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:02.727588    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"]
Jun 02 16:54:02.728233 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:02.727995    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="main" containerID="cri-o://752b568859dfadfe7ad6469f794adbcb79931c07b1d81759693f5ec8ff818cf8" gracePeriod=30
Jun 02 16:54:02.728233 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:02.728153    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="tokenizer" containerID="cri-o://12a15e2417337ca80da8cd9ec5b6a619b92a82d77628a39fafb24c45fd0b7b9b" gracePeriod=30
Jun 02 16:54:02.858683 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:02.858652    2563 generic.go:358] "Generic (PLEG): container finished" podID="1445060d-06de-4379-beb0-d2a9929cbcff" containerID="752b568859dfadfe7ad6469f794adbcb79931c07b1d81759693f5ec8ff818cf8" exitCode=0
Jun 02 16:54:02.858824 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:02.858692    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerDied","Data":"752b568859dfadfe7ad6469f794adbcb79931c07b1d81759693f5ec8ff818cf8"}
Jun 02 16:54:03.864668 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.864631    2563 generic.go:358] "Generic (PLEG): container finished" podID="1445060d-06de-4379-beb0-d2a9929cbcff" containerID="12a15e2417337ca80da8cd9ec5b6a619b92a82d77628a39fafb24c45fd0b7b9b" exitCode=0
Jun 02 16:54:03.865039 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.864675    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerDied","Data":"12a15e2417337ca80da8cd9ec5b6a619b92a82d77628a39fafb24c45fd0b7b9b"}
Jun 02 16:54:03.891474 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.891447    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:54:03.894694 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894675    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-uds\") pod \"1445060d-06de-4379-beb0-d2a9929cbcff\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") "
Jun 02 16:54:03.894762 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894706    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-cache\") pod \"1445060d-06de-4379-beb0-d2a9929cbcff\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") "
Jun 02 16:54:03.894762 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894745    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pstk5\" (UniqueName: \"kubernetes.io/projected/1445060d-06de-4379-beb0-d2a9929cbcff-kube-api-access-pstk5\") pod \"1445060d-06de-4379-beb0-d2a9929cbcff\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") "
Jun 02 16:54:03.894840 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894782    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-tmp\") pod \"1445060d-06de-4379-beb0-d2a9929cbcff\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") "
Jun 02 16:54:03.894926 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894907    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-kserve-provision-location\") pod \"1445060d-06de-4379-beb0-d2a9929cbcff\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") "
Jun 02 16:54:03.895033 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894942    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "1445060d-06de-4379-beb0-d2a9929cbcff" (UID: "1445060d-06de-4379-beb0-d2a9929cbcff"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:54:03.895033 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.894962    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "1445060d-06de-4379-beb0-d2a9929cbcff" (UID: "1445060d-06de-4379-beb0-d2a9929cbcff"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:54:03.895151 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.895054    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1445060d-06de-4379-beb0-d2a9929cbcff-tls-certs\") pod \"1445060d-06de-4379-beb0-d2a9929cbcff\" (UID: \"1445060d-06de-4379-beb0-d2a9929cbcff\") "
Jun 02 16:54:03.895249 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.895224    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "1445060d-06de-4379-beb0-d2a9929cbcff" (UID: "1445060d-06de-4379-beb0-d2a9929cbcff"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:54:03.895336 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.895317    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-tmp\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:54:03.895379 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.895344    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-uds\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:54:03.895379 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.895357    2563 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-tokenizer-cache\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:54:03.895687 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.895663    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1445060d-06de-4379-beb0-d2a9929cbcff" (UID: "1445060d-06de-4379-beb0-d2a9929cbcff"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 16:54:03.896966 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.896940    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1445060d-06de-4379-beb0-d2a9929cbcff-kube-api-access-pstk5" (OuterVolumeSpecName: "kube-api-access-pstk5") pod "1445060d-06de-4379-beb0-d2a9929cbcff" (UID: "1445060d-06de-4379-beb0-d2a9929cbcff"). InnerVolumeSpecName "kube-api-access-pstk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 16:54:03.897250 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.897232    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1445060d-06de-4379-beb0-d2a9929cbcff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1445060d-06de-4379-beb0-d2a9929cbcff" (UID: "1445060d-06de-4379-beb0-d2a9929cbcff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 16:54:03.995816 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.995791    2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1445060d-06de-4379-beb0-d2a9929cbcff-kserve-provision-location\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:54:03.995816 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.995816    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1445060d-06de-4379-beb0-d2a9929cbcff-tls-certs\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:54:03.995968 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:03.995826    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pstk5\" (UniqueName: \"kubernetes.io/projected/1445060d-06de-4379-beb0-d2a9929cbcff-kube-api-access-pstk5\") on node \"ip-10-0-140-193.ec2.internal\" DevicePath \"\""
Jun 02 16:54:04.869903 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.869868    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp" event={"ID":"1445060d-06de-4379-beb0-d2a9929cbcff","Type":"ContainerDied","Data":"e42a4f6c5a53e19c49486da9593fdd658958a69d84ba55a80d742f01e9238706"}
Jun 02 16:54:04.869903 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.869905    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"
Jun 02 16:54:04.870407 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.869912    2563 scope.go:117] "RemoveContainer" containerID="12a15e2417337ca80da8cd9ec5b6a619b92a82d77628a39fafb24c45fd0b7b9b"
Jun 02 16:54:04.879235 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.879216    2563 scope.go:117] "RemoveContainer" containerID="752b568859dfadfe7ad6469f794adbcb79931c07b1d81759693f5ec8ff818cf8"
Jun 02 16:54:04.887742 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.887701    2563 scope.go:117] "RemoveContainer" containerID="189f8f2c25fe90c9ce2dfbda1cce1254e275d97919e5f2c8195686050c81a844"
Jun 02 16:54:04.891173 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.891150    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"]
Jun 02 16:54:04.898093 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:04.898069    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schegfldp"]
Jun 02 16:54:05.981901 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:05.981863    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" path="/var/lib/kubelet/pods/1445060d-06de-4379-beb0-d2a9929cbcff/volumes"
Jun 02 16:54:33.976748 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:54:33.976709    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:55:35.979979 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:55:35.979944    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:56:44.976705 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:56:44.976673    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:57:46.977360 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:57:46.977322    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 16:58:53.977172 ip-10-0-140-193 kubenswrapper[2563]: I0602 16:58:53.977089    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:00:16.977605 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:00:16.977529    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:01:35.980039 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:01:35.979984    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:02:54.977640 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:02:54.977603    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:03:56.976685 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:03:56.976646    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:05:16.976788 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:05:16.976753    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:06:41.976921 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:06:41.976880    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:07:59.358097 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:07:59.358001    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:07:59.530242 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:07:59.530209    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:07:59.546728 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:07:59.546702    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:07:59.552814 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:07:59.552794    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:00.492880 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:00.492853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:00.628746 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:00.628708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:00.647868 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:00.647833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:00.655210 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:00.655185    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:01.621521 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:01.621486    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:01.754148 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:01.754119    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:01.770168 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:01.770140    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:01.776467 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:01.776442    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:02.730348 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:02.730313    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:02.895905 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:02.895867    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:02.911425 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:02.911392    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:02.918495 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:02.918476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:03.849388 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:03.849355    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:03.973407 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:03.973364    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:03.988974 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:03.988945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:03.994841 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:03.994821    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:04.922249 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:04.922219    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:05.052093 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:05.052058    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:05.066829 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:05.066801    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:05.073930 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:05.073908    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:06.016605 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:06.016573    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:06.149348 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:06.149315    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:06.164780 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:06.164750    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:06.171109 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:06.171087    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:07.120539 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:07.120513    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:07.256913 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:07.256884    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:07.271079 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:07.271060    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:07.277345 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:07.277327    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:08.232210 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:08.232178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:08.378200 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:08.378170    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:08.397643 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:08.397616    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:08.405510 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:08.405486    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:08.976815 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:08.976777    2563 kubelet_pods.go:1019] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-zvk8p\" not found"
Jun 02 17:08:09.406858 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:09.406830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:09.548281 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:09.548253    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:09.568399 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:09.568366    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:09.575800 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:09.575776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:10.585245 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:10.585218    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:10.733474 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:10.733448    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:10.753252 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:10.753225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:10.761963 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:10.761943    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:11.778285 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:11.778251    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:11.912289 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:11.912260    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:11.925723 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:11.925687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:11.931967 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:11.931949    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:12.911138 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:12.911104    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:13.060879 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:13.060847    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:13.075283 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:13.075252    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:13.081428 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:13.081392    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:14.027236 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:14.027208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-jdt24_1104eedb-d3d9-41ec-b3b9-b13de6b78f1e/istio-proxy/0.log"
Jun 02 17:08:14.089393 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:14.089360    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/main/0.log"
Jun 02 17:08:14.105973 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:14.105940    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/tokenizer/0.log"
Jun 02 17:08:14.112622 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:14.112603    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_stop-feature-test-kserve-router-scheduler-7947f646c8-w52ts_42e269a9-e937-4780-b4f0-756a8511cf2b/storage-initializer/0.log"
Jun 02 17:08:15.107311 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:15.107278    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-2xdkj_88e65e53-7f53-4667-baf1-c9966e69e948/discovery/0.log"
Jun 02 17:08:15.127967 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:15.127931    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t_33a10624-6046-465c-9827-acc53f72274a/istio-proxy/0.log"
Jun 02 17:08:15.926720 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:15.926684    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-2xdkj_88e65e53-7f53-4667-baf1-c9966e69e948/discovery/0.log"
Jun 02 17:08:15.940878 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:15.940845    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t_33a10624-6046-465c-9827-acc53f72274a/istio-proxy/0.log"
Jun 02 17:08:16.739224 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:16.739182    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-2xdkj_88e65e53-7f53-4667-baf1-c9966e69e948/discovery/0.log"
Jun 02 17:08:16.753656 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:16.753627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t_33a10624-6046-465c-9827-acc53f72274a/istio-proxy/0.log"
Jun 02 17:08:17.533843 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:17.533809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-mcvg7_3f836081-ea4c-49e0-afdc-579993710b48/authorino/0.log"
Jun 02 17:08:17.548653 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:17.548619    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-xccdr_c5d7ff2e-3462-4732-84c8-a5b54878b1eb/manager/0.log"
Jun 02 17:08:17.652621 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:17.652590    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-c7fb4c8d5-ng5cv_bf180035-fcec-4771-8f3b-e8c884266cd4/manager/0.log"
Jun 02 17:08:18.441326 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:18.441286    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-mcvg7_3f836081-ea4c-49e0-afdc-579993710b48/authorino/0.log"
Jun 02 17:08:18.453908 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:18.453879    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-xccdr_c5d7ff2e-3462-4732-84c8-a5b54878b1eb/manager/0.log"
Jun 02 17:08:18.559059 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:18.559027    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-c7fb4c8d5-ng5cv_bf180035-fcec-4771-8f3b-e8c884266cd4/manager/0.log"
Jun 02 17:08:19.369029 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:19.368981    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-mcvg7_3f836081-ea4c-49e0-afdc-579993710b48/authorino/0.log"
Jun 02 17:08:19.384296 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:19.384267    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-xccdr_c5d7ff2e-3462-4732-84c8-a5b54878b1eb/manager/0.log"
Jun 02 17:08:19.488083 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:19.488055    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-c7fb4c8d5-ng5cv_bf180035-fcec-4771-8f3b-e8c884266cd4/manager/0.log"
Jun 02 17:08:20.270616 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:20.270583    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-mcvg7_3f836081-ea4c-49e0-afdc-579993710b48/authorino/0.log"
Jun 02 17:08:20.285648 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:20.285619    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-xccdr_c5d7ff2e-3462-4732-84c8-a5b54878b1eb/manager/0.log"
Jun 02 17:08:20.399999 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:20.399967    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-c7fb4c8d5-ng5cv_bf180035-fcec-4771-8f3b-e8c884266cd4/manager/0.log"
Jun 02 17:08:21.188174 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:21.188145    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-mcvg7_3f836081-ea4c-49e0-afdc-579993710b48/authorino/0.log"
Jun 02 17:08:21.201145 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:21.201124    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-xccdr_c5d7ff2e-3462-4732-84c8-a5b54878b1eb/manager/0.log"
Jun 02 17:08:21.310079 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:21.310045    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-c7fb4c8d5-ng5cv_bf180035-fcec-4771-8f3b-e8c884266cd4/manager/0.log"
Jun 02 17:08:26.590568 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:26.590537    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vrpfz_21e6a21a-a998-48cb-aac3-b9ab616597d2/global-pull-secret-syncer/0.log"
Jun 02 17:08:26.656867 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:26.656837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-66b65_461355bd-6507-41f6-a7cf-0c62360f9a90/konnectivity-agent/0.log"
Jun 02 17:08:26.820483 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:26.820448    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-193.ec2.internal_2cc8efb4d7faafb9447182ed8c6f6fdb/haproxy/0.log"
Jun 02 17:08:30.863562 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:30.863533    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-mcvg7_3f836081-ea4c-49e0-afdc-579993710b48/authorino/0.log"
Jun 02 17:08:30.910270 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:30.910242    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-xccdr_c5d7ff2e-3462-4732-84c8-a5b54878b1eb/manager/0.log"
Jun 02 17:08:31.114750 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:31.114662    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-c7fb4c8d5-ng5cv_bf180035-fcec-4771-8f3b-e8c884266cd4/manager/0.log"
Jun 02 17:08:32.188840 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.188809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-t5n2c_e07e5347-9398-4ad3-9d41-e7948879b7b4/cluster-monitoring-operator/0.log"
Jun 02 17:08:32.330460 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.330430    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-6pj6d_202eeb56-5528-4d44-823e-363bdc40f087/monitoring-plugin/0.log"
Jun 02 17:08:32.470051 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.469954    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjnws_6dd45487-bc47-4833-82f9-1802d25e487b/node-exporter/0.log"
Jun 02 17:08:32.497154 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.497131    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjnws_6dd45487-bc47-4833-82f9-1802d25e487b/kube-rbac-proxy/0.log"
Jun 02 17:08:32.524184 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.524160    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjnws_6dd45487-bc47-4833-82f9-1802d25e487b/init-textfile/0.log"
Jun 02 17:08:32.620374 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.620341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-lsmgx_f64c06dd-bb9b-426d-b49f-b8f412d3093e/kube-rbac-proxy-main/0.log"
Jun 02 17:08:32.642525 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.642496    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-lsmgx_f64c06dd-bb9b-426d-b49f-b8f412d3093e/kube-rbac-proxy-self/0.log"
Jun 02 17:08:32.664886 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.664865    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-lsmgx_f64c06dd-bb9b-426d-b49f-b8f412d3093e/openshift-state-metrics/0.log"
Jun 02 17:08:32.953517 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:32.953487    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-xv5tg_a2c4c393-af6c-4c29-8487-ce43eca6725b/prometheus-operator-admission-webhook/0.log"
Jun 02 17:08:33.081156 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:33.081131    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8676877756-zk85p_a25e2b5c-21e7-450c-b856-ec1639508495/thanos-query/0.log"
Jun 02 17:08:33.110503 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:33.110476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8676877756-zk85p_a25e2b5c-21e7-450c-b856-ec1639508495/kube-rbac-proxy-web/0.log"
Jun 02 17:08:33.134180 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:33.134151    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8676877756-zk85p_a25e2b5c-21e7-450c-b856-ec1639508495/kube-rbac-proxy/0.log"
Jun 02 17:08:33.154426 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:33.154396    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8676877756-zk85p_a25e2b5c-21e7-450c-b856-ec1639508495/prom-label-proxy/0.log"
Jun 02 17:08:33.193713 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:33.193691    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8676877756-zk85p_a25e2b5c-21e7-450c-b856-ec1639508495/kube-rbac-proxy-rules/0.log"
Jun 02 17:08:33.216052 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:33.215968    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8676877756-zk85p_a25e2b5c-21e7-450c-b856-ec1639508495/kube-rbac-proxy-metrics/0.log"
Jun 02 17:08:34.370663 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:34.370631    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-9q855_bb28d427-0780-43ac-a3f5-eef65787cb92/networking-console-plugin/0.log"
Jun 02 17:08:35.362186 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.362155    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-j2jg5_fdd2a710-f503-42f2-8397-72f3106a2630/download-server/0.log"
Jun 02 17:08:35.447564 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.447529    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"]
Jun 02 17:08:35.447959 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.447944    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="tokenizer"
Jun 02 17:08:35.448004 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.447961    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="tokenizer"
Jun 02 17:08:35.448004 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.447983    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="main"
Jun 02 17:08:35.448004 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.447990    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="main"
Jun 02 17:08:35.448004 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.447999    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="storage-initializer"
Jun 02 17:08:35.448004 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.448004    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="storage-initializer"
Jun 02 17:08:35.448199 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.448100    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="tokenizer"
Jun 02 17:08:35.448199 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.448110    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="1445060d-06de-4379-beb0-d2a9929cbcff" containerName="main"
Jun 02 17:08:35.450402 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.450382    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.453207 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.453187    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nxgdw\"/\"openshift-service-ca.crt\""
Jun 02 17:08:35.453334 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.453215    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nxgdw\"/\"default-dockercfg-xs2rl\""
Jun 02 17:08:35.453334 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.453313    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nxgdw\"/\"kube-root-ca.crt\""
Jun 02 17:08:35.460076 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.460053    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"]
Jun 02 17:08:35.569367 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.569331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-podres\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.569367 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.569370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8z27\" (UniqueName: \"kubernetes.io/projected/2c38b65c-7839-48cb-b3e1-8fcef1d89622-kube-api-access-c8z27\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.569568 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.569459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-sys\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.569568 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.569501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-proc\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.569568 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.569541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-lib-modules\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670627 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670524    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-sys\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670627 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670585    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-proc\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670627 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670617    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-lib-modules\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670899 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670668    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-podres\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670899 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-sys\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670899 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670693    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c8z27\" (UniqueName: \"kubernetes.io/projected/2c38b65c-7839-48cb-b3e1-8fcef1d89622-kube-api-access-c8z27\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670899 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670716    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-proc\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670899 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670817    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-lib-modules\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.670899 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.670824    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c38b65c-7839-48cb-b3e1-8fcef1d89622-podres\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.678600 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.678582    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8z27\" (UniqueName: \"kubernetes.io/projected/2c38b65c-7839-48cb-b3e1-8fcef1d89622-kube-api-access-c8z27\") pod \"perf-node-gather-daemonset-bnzgw\" (UID: \"2c38b65c-7839-48cb-b3e1-8fcef1d89622\") " pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.761590 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.761555    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:35.830365 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.830339    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-gs7nr_c673ec07-8d69-4136-8f96-eefd17d29c83/volume-data-source-validator/0.log"
Jun 02 17:08:35.891969 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.891943    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"]
Jun 02 17:08:35.893873 ip-10-0-140-193 kubenswrapper[2563]: W0602 17:08:35.893848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2c38b65c_7839_48cb_b3e1_8fcef1d89622.slice/crio-c967eb9876606e4930e9096268788235391e47586e8fbb4bc3bb97133f0f7e67 WatchSource:0}: Error finding container c967eb9876606e4930e9096268788235391e47586e8fbb4bc3bb97133f0f7e67: Status 404 returned error can't find the container with id c967eb9876606e4930e9096268788235391e47586e8fbb4bc3bb97133f0f7e67
Jun 02 17:08:35.895485 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:35.895468    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 17:08:36.301207 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.301180    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw" event={"ID":"2c38b65c-7839-48cb-b3e1-8fcef1d89622","Type":"ContainerStarted","Data":"300a81b8db94193c619d13faa42858b885c73cd19878595a01dca1f554e95883"}
Jun 02 17:08:36.301396 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.301214    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw" event={"ID":"2c38b65c-7839-48cb-b3e1-8fcef1d89622","Type":"ContainerStarted","Data":"c967eb9876606e4930e9096268788235391e47586e8fbb4bc3bb97133f0f7e67"}
Jun 02 17:08:36.301396 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.301280    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:36.318661 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.318615    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw" podStartSLOduration=1.318602157 podStartE2EDuration="1.318602157s" podCreationTimestamp="2026-06-02 17:08:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 17:08:36.315919824 +0000 UTC m=+4900.939708671" watchObservedRunningTime="2026-06-02 17:08:36.318602157 +0000 UTC m=+4900.942391004"
Jun 02 17:08:36.602422 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.602347    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-88dbl_14fc5c79-98d8-47a1-8c19-2cb07454b83e/dns/0.log"
Jun 02 17:08:36.621965 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.621921    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-88dbl_14fc5c79-98d8-47a1-8c19-2cb07454b83e/kube-rbac-proxy/0.log"
Jun 02 17:08:36.644093 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:36.644071    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2ghc2_1992ad1c-cbee-4529-a8a8-b05b61961292/dns-node-resolver/0.log"
Jun 02 17:08:37.144030 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:37.143994    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-fdd9c8df7-f9twg_6969dfd7-09e7-4fec-baba-a90816d3f83a/registry/0.log"
Jun 02 17:08:37.166679 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:37.166648    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6glkd_d60aee99-f527-46fb-8992-2945e473f6d4/node-ca/0.log"
Jun 02 17:08:38.047339 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:38.047304    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-2xdkj_88e65e53-7f53-4667-baf1-c9966e69e948/discovery/0.log"
Jun 02 17:08:38.070385 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:38.070360    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-nqw6t_33a10624-6046-465c-9827-acc53f72274a/istio-proxy/0.log"
Jun 02 17:08:38.528425 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:38.528398    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7fbbj_09c32fc9-879b-4dd1-9b1a-6b16d9972168/serve-healthcheck-canary/0.log"
Jun 02 17:08:39.035232 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:39.035185    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d647j_72088158-d1c4-4d2f-be92-bb3f772aa99c/kube-rbac-proxy/0.log"
Jun 02 17:08:39.059872 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:39.059847    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d647j_72088158-d1c4-4d2f-be92-bb3f772aa99c/exporter/0.log"
Jun 02 17:08:39.086752 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:39.086731    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d647j_72088158-d1c4-4d2f-be92-bb3f772aa99c/extractor/0.log"
Jun 02 17:08:42.258597 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:42.258553    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7b649f477f-7dx4d_73a342c0-d213-40ce-a603-77ebce7c4354/manager/0.log"
Jun 02 17:08:42.315596 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:42.315565    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nxgdw/perf-node-gather-daemonset-bnzgw"
Jun 02 17:08:42.518898 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:42.518817    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-4j6g6_cb1fd274-e48c-4c1b-b5ce-65fe8e74753d/manager/0.log"
Jun 02 17:08:42.562975 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:42.562936    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-79f457656b-kcmhm_ab5e627c-b245-423e-9d5c-ed89d7bbf8f5/seaweedfs/0.log"
Jun 02 17:08:47.165923 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:47.165892    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-8jslt_b74f94ba-6a90-4d66-8c84-b71e9f1f2644/migrator/0.log"
Jun 02 17:08:47.186631 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:47.186606    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-8jslt_b74f94ba-6a90-4d66-8c84-b71e9f1f2644/graceful-termination/0.log"
Jun 02 17:08:48.577266 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.577235    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/kube-multus-additional-cni-plugins/0.log"
Jun 02 17:08:48.598628 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.598602    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/egress-router-binary-copy/0.log"
Jun 02 17:08:48.621106 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.621080    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/cni-plugins/0.log"
Jun 02 17:08:48.646855 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.646832    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/bond-cni-plugin/0.log"
Jun 02 17:08:48.667731 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.667708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/routeoverride-cni/0.log"
Jun 02 17:08:48.690091 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.690071    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/whereabouts-cni-bincopy/0.log"
Jun 02 17:08:48.713689 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.713660    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jvn72_a411c79a-4777-4a0e-a335-572999714887/whereabouts-cni/0.log"
Jun 02 17:08:48.928533 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:48.928457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vqrnf_ec80ae08-6054-4495-a85d-c88fc5f066a8/kube-multus/0.log"
Jun 02 17:08:49.020352 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.020323    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5r476_aca2f9e4-d1e9-470c-b700-d6cc6f3be66c/network-metrics-daemon/0.log"
Jun 02 17:08:49.041732 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.041708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5r476_aca2f9e4-d1e9-470c-b700-d6cc6f3be66c/kube-rbac-proxy/0.log"
Jun 02 17:08:49.880841 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.880813    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/ovn-controller/0.log"
Jun 02 17:08:49.921474 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.921448    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/ovn-acl-logging/0.log"
Jun 02 17:08:49.944025 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.943991    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/kube-rbac-proxy-node/0.log"
Jun 02 17:08:49.966233 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.966208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/kube-rbac-proxy-ovn-metrics/0.log"
Jun 02 17:08:49.984396 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:49.984376    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/northd/0.log"
Jun 02 17:08:50.004536 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:50.004512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/nbdb/0.log"
Jun 02 17:08:50.027552 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:50.027530    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/sbdb/0.log"
Jun 02 17:08:50.143633 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:50.143556    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9p9h9_560b50bf-edb4-4b38-950c-a898eb7bc4b9/ovnkube-controller/0.log"
Jun 02 17:08:51.978627 ip-10-0-140-193 kubenswrapper[2563]: I0602 17:08:51.978592    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-kpzkn_73906112-b7ef-4e92-865a-6fbf041dd5bf/network-check-target-container/0.log"